So in this post we are going to talk about how search engine works. Hello friends I am Abhishek going welcome in seo-stuck. So let's start with the new topic on seo module that is how search engine work.
In this module we are going to discuss about what is search engine. What is crawling what is indexing what is your crawl budget. And how to optimize your product okay. So let's start with a brief definition on search engine.
What search engine
What search engine is? So a search engine is a software program. That helps people find the information; they are looking for online using keyword or the phrases okay. Example of the search engines are just like your Google, Bing, yahoo.
You can use these search engines. To figure out any particular information you want to get okay. So these search engines can store information. About brilliance of the web pages in an organized format okay.
So basically that is the search engine. And now we have to know how the search engine works. When you search any particular query. Then what are the backend activities happening on the search engine.
How search engine work
Because just by knowing that. Here we can work on this way now we have three component of search engine.
1. First one is your query engine. Which proceeds your query to your crawler? Which proceed your query to your search engine database?
2. Just to figure out that information next is your crawler. Which can read every single information from your website?
3. And next one is indexer which can index your website.
What is this crawling?
Okay next is your crawling what is this crawling here. Crawling basically means crawling is reading content on the web pages. And storing that information in the database. So what happened here let’s suppose?
This is your website okay. And on your website you can add multiple information. There information in the form of text in the form of images in the form of posts okay. So whatever with the information you added in your website.
Here we have one web crawler. Or you can also say web spider okay. So your web crawler or web spider visits on your website. Or any particular seed url you mentioned there any particular url. Queue in your web web page okay.
They can visit there and crawl every single information patch. With the help of internet here and then extract that information and save this information in your database okay. So this is the use of crawling.
Here in case of crawling let's suppose if any web link is available in your web page okay. So how your Google crawler is going to work. They first visit your website read your content. Again they found one link here.
Then they again visit on that particular link page. And also read all the information available here. So this is the functioning of crawler here.
Policies of crawler (How search engine work)
Now the back crawler explores the work page based on some policies. What are these policies let's figure out. First of all we have one selection policy according to the selection policy. Crawler decides which page it should download.
And which it should not next we have revisit policy. Revisit policy means that crawler schedule the time. When it should reopen the workface and added the changes in its database okay. Let's suppose.
If this is your website web crawler visit to your website. And read every single information okay. And after some days. If you do any change in your web page in that case. Here Google crawler again visits to your website.
And again crawl all the recommended average changes. Which you've done in your website and store that changes in their database okay. Next we have parallelization policy in case of parallelization policy.
Crawlers use multiple processes at once to explore the links. Known as distributed chronicle. So what happened in that case? Here let's suppose this is your website okay. And your website a link is mentioned in website.
Here and again when web crawler read information. Okay next we have politeness policy in case of politeness. What happened here we have our term? Known as crawl delay okay. So what is this crawl delay means?
when Google crawlers download any information from your webpage. Then they just take a post of some billy second. And that millisecond pose is known as crawl delay here okay. That is implemented in which the crawler has to wait for a few seconds.
After it download some data from the website okay. So these are the policies we have in your in case of crawling.
Crawler work
Next we have how the crawler works. See we have a search engine here let us suppose Google. Is your search engine when someone put any query here any query in the search engine okay? Then what happened.
Here your web spider or your web crawler. With it to your website read every single web page crawl your website. Copy the information in your website and store that information in the database okay.
And when user put any information there. Then that information is figure out from this database. And display the result from this database to the user. So this is how your crawler works okay. After crawling next.
Indexing of search engine (How search engine work)
We have indexing okay as Google crawler. Or your search engine crawler crawl every information from your website. That after crawling your information now the next step is to do index. What indexing means. Indexing is just like you can do indexing of your work okay. Let's suppose if you want to figure out any particular topic from this book okay. Now what approach you can do first. you will start from your search page.
And start continues reading all the topics. Here or you can simply look at your index figure out that particular topic. Corresponding to this figure out your page simply visit that page. Number and find your information okay.
So this is the easiest step here like simply look at your index. And visit to that particular page. The same thing happens in the Google in your search engine. In your search engine database we have lot of information there okay.
So just to fetch your users query from that particular database. It is really difficult so that's why here search engine do index. Indexing on the basis of keywords here okay. When someone search for any particular query.
Then corresponding to their worry we can check their relevant keywords. And then that particular query or information. Will reflect to your user okay so after the baby search engine crawl. All over the internet it creates a index.
Conclusion
Okay so in this post we will cover how your search engine working. And in our more upcoming post we will learn more about seo so thank you so much for reading this post