Search Engines 101

Search engines help a person find web pages on a subject that is specified by the person. The person specifies the subject by providing specific words that “define” or are important to that subject.

If you are interested in, say, stamp collecting, you can enter the words “stamp” and “collecting”, and the search engine will return a list of web pages that contain those words. And it will return the results in a matter of seconds.

How does it do this?

By maintaining a database of web pages. The database is indexed based on the words used on those web pages. So if, in our example, the search used the words “stamp” and “collecting”, behind the scenes the database is scanned for the web pages that use those words, and the pages that are found are presented as the results of the search.

Spiders and Robots

In order to create and maintain the database, search engines use software called “spiders” or “robots” to visit web sites and collect information regarding the words used on the web pages. The spiders collect other information that is used in categorizing the web pages, and for ranking the relative importance of those pages to their subject.

Although these spiders visit web sites daily, the databases are not updated with new results on a daily basis. Sometimes in takes months for a new website to appear in the database and begin getting displayed in search results.

Search Results

When the result of a search is presented, the web pages that are “found” are listed in their order of relevance and importance to the subject of the search.

The exact algorithm used to categorize and rank web pages’ relevance and importance is a closely guarded secret, since this provides the primary means a search engine can differentiate itself from its competitors. Presumably, if the search results give you what you want, you will keep going back the next time you want to search for information.

The Big Three

There are literally hundred of web sites providing search. The three most popular – Google, MSN Search, and Yahoo – are aggressively competing with one another for your business.

How do they make money?

By selling advertising. Whenever you perform a search, the results that are returned also present what are called “Sponsored Links” by Google, “Sponsored Sites” by MSN Search, and “Sponsored Results” by Yahoo Search.

The sponsored sites are web sites whose owners have paid for their sites to be displayed on the page of results for those specific search terms. The cost of this advertising is determined by the frequency of searches on the subject and whether or not there is strong competition for business.

In other words, sponsors for “used cars” will pay considerably more than sponsors for “stamp collecting”. Try doing a search on “training aardvarks” and you will find – to no one’s surprise – that there are no sponsored sites.

Preferences and Advanced Search

There are two features that you should take advantage of – Preferences and Advanced Search. Preferences allows you to specify how you want search results displayed (for example, how many web pages you’d like listed on a page of results). Advanced search permits you to fine-tune your search, honing in on exactly the information you are most interested in.

Improving Your Search

There are a number of ways of entering search terms that will help filter out unwanted results and increase focus on the exact information you want. Let’s return to our stamp collecting example. Entering

stamp collecting “united states” -coin

will return a list of web pages that contain the words “stamp” and “collecting”, contain the exact phrase “united states”, and do not contain the word “coin”.

A good way to discover these tricks is to investigate the Advanced Search option.

Toolbars - the tool that binds

There is another feature of search sites that is quite prevalent at the moment – Toolbars. Google, Yahoo, MSN Search, and many other sites offer you download of a free toolbar that you can install into Internet Explorer. The toolbar provides various useful features, such as the ability to initiate a search from the toolbar, stopping pop-ups, and automatically filling online forms with your personal information.

Toolbars are an attempt to “get your business” by eliminating the need for you to go to their website – you can search directly from your toolbar. Very convenient, but it could result in your missing out on interesting developments by competitors.

Because the competition for your attention is so great, other web search sites are looking for a means of separating themselves from the crowd by making your search easier, more accurate, and easier to understand.

Try a search at Kartoo and you will see a very different approach to displaying search results. Or try Snap and you will see results appear as you type in the search terms themselves.

We can expect other interesting developments as these online businesses continue to compete for our attention.