skip to content

How do search engines work?

In order to better understand SEO and link popularity it is helpful to understand something about how the search engines work.

Search Engines use automated software (spiders or bots) to "crawl" through the web using the (hyper) link structure. When a "crawler" visits a website, it reads the content of the site, reads the website's meta tags and also follows any links that the site connects to performing indexing on all linked websites as well.

Once a page has been crawled, it's contents can be stored in a database of documents that makes up a search engine's "index". When a user searches it is these indexed documents that are retrieved.

When a request for information comes into the search engine, the engine retrieves from it's index all the documents that match the query. A match is determined if the search term or phrase is found on the page in the manner specified by the user.

Once the search engine has determined which results are a match for the query, it runs calculations on each of the results to determine which are most relevant. These are sorted on the results pages in order from most relevant to least so that users can make a choice about which to select. This is the page ranking for a particular query or search term.

The search engines generally use 2 criteria to rank pages for a particular search term; Relevance and Popularity.

  1. Popularity through link analysis. The search engines view links to a site as a "vote". For instance if Site A links to Site B then Site A is considered to have voted for Site B. Sites with more links to them are considered to be more popular and therefore more relevant. However, not all "votes" are equal. Links from popular sites carry much more weight than links from less popular sites.
  2. Relevancy through document analysis. Search engines look at whether the search terms are found in important areas of the document; the page title, the headings, and the text content. They also attempt to automatically measure the quality of the document. The more often (to a point) that the search terms appear in the title, headings and page text the more relevant the page is considered to be.

See the articles What is SEO? and link popularity to understand how this knowledge about how search engines work can be used to improve search result page rankings.