Crawling
A search engine uses software called a "web spider" or "web crawler". This is a type of "bot" - short for "robot" - which is software that automatically does repetitive tasks.
The spider browses the world wide web at regular intervals. It starts at heavily used sites and follows links to other sites. The spider "crawls" from site to site, reading their content.
Indexing
The spider builds a list of web pages and words from which the search engine creates a database called an index.
When you search, the engine compares your keyword or phrase to its index and compiles a list of links to pages that match your enquiry. This is what you see as your search results.
Ranking
Search engines use sophisticated rules for ranking the results. They all do it a bit differently but there are some common principles. For every indexed web page, they analyse data about specific internal and external factors.
Internal factors include behind-the-scenes information such as keywords, page descriptions and page titles. Because of tricks like keyword stuffing - attempting to influence results by cramming a site with a variety of keywords - search engines look at the content of each page to assess what it is really about.
External factors include links from other sites. The idea is that other sites will link to a web page if it contains helpful, relevant information. The more links into a page from other sites, the higher ranking it deserves. Again, people play tricky games - like registering on paid directories that have no purpose other than to create inbound links - so search engines analyse where links come from. A link from a popular site with few links has more weight than one from a directory with many links.
Tips for better search engine rankings
Publish content that is really helpful and useful. The better your content, the more other sites will link to it. Links get you recognition from search engines.
Have one main topic or subject per page. This allows clearer matches with searches.
Put each page's main idea into a heading and the first paragraph. Search engines give more weight to words at the beginning of a page.
Include in the content of each page the kind of words and phrases with which people will search. Avoid industry jargon. Use your customers' language.
Minimise or avoid the use of Flash visual effects. Search engines can have a hard time looking inside them.
Include ALT text for images that communicate information (including logos and pictures that act as links). ALT text is a description that appears in a text box when you put the cursor over the image. Search engines access it like other text on your site.
Build your site with descriptive page titles, page descriptions and keywords.
Be patient, get good advice and work at it. Doing well in search engine results is something to tackle as part of an overall plan for managing your site.
