You must be thinking at times, how Google is able to give out the correct or the required result in such a short span of time? I myself use to ponder over it for a long time.

Let me explain you the process behind the search result delivered in Google.

The life span of a Google search query normally lasts less than half a second, yet involves a number of different steps that must be completed, before results can be delivered to a person seeking information.

How Google spider crawls:

In Google, the web crawling (downloading of web pages) is done by several distributed crawlers ( called Googlebot). There is a URL server that sends lists of URL's to be fetched to these crawlers. The web pages that are fetched by the Googlebot are then sent to the store server. The store server then compresses and stores the web pages into a repository (a kind of storage facility). Every web page has an associated ID number called a doc ID that is assigned whenever a new URL is parsed (analyzed) out of a web page.

The indexing function is performed by the indexer and the sorter. The indexer performs a number of functions. It reads the repository, uncompress the documents, and parses (analyzes) them. Each document is then converted into a set of word occurrences called hits. The hits record the word, position in the document, an approximation of font size and capitalization.

The indexer distributes these hits into a set of "barrels", creating a partially sorted forward index. The indexer performs another important function. It parses (analyzes) out all the links in every web page and stores all the important information about them in an anchor file. This file contains enough information to determine where each link points from and to, and the text of the link.

The URL resolver reads the anchor file and converts relative URL's into absolute URL's and in turn into doc ID's. It puts the anchor text into the forward index, associated with the doc ID that the
anchor points to. It also generates a database of links that are pairs of doc ID's.

The links database is used to compute page ranks for all the documents. The pages that get into the barrel first will be most likely to be included in the search results. The question is, how does Google decide which pages go into the barrel first? The answer (as some of you probably would have already guessed) is based on the fact that the doc ID is an integer.

For viewing how Google search engine works,

Click here