|
Spring 1999
|
In this article, Elkie Holland offers insights on how Internet search engines work, what criteria they use to rank web pages, and some tips to help ensure your web page gets the best ranking - and by extension, more visitors.
Many web sites appear poorly or may not even be listed at all as their design fails to consider how search engines work. Knowledge of “search engine design” can help many of these sites. It can ensure that your web pages are accessible to search engines and focused in ways that help improve their chances of being found.
Directories
Hybrid Search Engines
Metacrawlers
Search engines have 3 major elements.
1) The Spider: Also called the crawler. The spider visits a web page, reads it, and then follows links to other pages within the site. The spider returns to the site on a regular basis, such as every month or two, to look for changes.
2) The Index: Everything the spider finds goes into the second part of a search engine, the index. The index, sometimes called the catalog, is like a huge book containing a copy of every web page that the spider finds. If a web page changes, then this book is updated new information. It can take a while for new pages or changes to be added to the index.
3) Search engine software: This is the program that sifts through the millions of pages recorded in the index to find matches to a search and rank them in order of what it believes is most relevant.
Search engines follow a set of rules to go about determining relevancy. The main rules involve the location and frequency of keywords on a web page.
Pages with keywords in the title are assumed to be more relevant than others to the topic. Search engines will also check to see if the keywords appear near the top of a web page, such as in the headline or in the first few paragraphs of text. They assume that any page relevant to the topic will mention those words right from the beginning.
Frequency is the other major factor in how search engines determine relevancy. A search engine will analyse how often keywords appear in relation to other words in a web page. Those with a higher frequency are often deemed more relevant.
Sounds simple - if only <g>! All the major search engines use the location/frequency method to some degree but they each like to add a few nuances. No search engine does it exactly the same way which is why we get different results for identical searches on different engines. Another reason for the different results from the different engines is that some index more web pages than others. Some search engines also index web pages more often than others. No two search engines have the exact same collection of web pages to search through.
Tip 1: Pick your strategic keywords
How do you think people will search for your web page? The words you imagine
them typing into the search box are your strategic keywords. Each page in your
web site will have different strategic keywords that reflect the page’s
content. Your strategic keywords should always be at least two or more words
long.
Tip 2: Position your keywords
Make sure that your strategic keywords appear in the crucial locations on your
web pages. The page title is most important. Search engines also like pages
where keywords appear “high” on the page. To accommodate this, use
your strategic keywords for your page headline, if possible. Have them also
appear in the first paragraphs of your web page.
Tip 3: Beware of Tables
Keep in mind that tables can “push” your text further down the page,
making keywords less relevant because they appear lower on the page. This is
because tables break apart when search engines read them. Don’t stop using
tables but think about how they can affect search engine results.
Tip 4: Beware of JavaScript
JavaScript can also have the same effect as tables. The search engine reads
this information first, causing normal HTML text to appear lower on the page.
Place your script further down on the page if possible.
Tip 5: MetaTags
These can assist you with overcoming problems with tables and frames. Meta tags
will also help you control your site’s description in engines that support
them. You should use meta tags, but keep in mind that they are not a guarantee
that your site will appear first. Adding some meta tag code is not a magic bullet
that cures your site of dismal rankings but it sure may help.
There are several meta tags, but the most important for search engine indexing are the description and keywords tags. The description tag returns a description of the page in place of the summary the search engine would ordinarily create. The keywords tag provides keywords for the search engine to associate with your page. Meta tags are what many web designers mistakenly assume are the “secret” to propelling their web pages to the top of the rankings. They can be part of the recipe for high rankings but they are not necessarily the secret ingredient. Example:
<HEAD>
<TITLE>My World</TITLE>
<META name=“description” content=“Everything you wanted to
know about stamps, from prices to history.”>
<META name=“keywords” content=“stamps, stamp collecting, stamp
history, prices, stamps for sale”>
</HEAD>
Tip 6: Relevant Content
Changing page titles and adding meta tags is not necessarily going to help your
page do well for your strategic keywords if the page has nothing to do with
the topic. Your keywords need to be reflected in the page’s content. This
means you need HTML text on your page.
Tip 7: Text as Graphics
If you present large sections of text via graphics, it looks pretty, but search
engines can’t read them! That means they miss out on text that might make
your site more relevant. Some engines will index ALT text and comments, along
with meta tags. But to be safe, use HTML text whenever possible.
Tip 8: Visible text
Be sure that your HTML text is “visible”. Search engines are now catching
on to designers who try to “spam” search engines by repeating keywords
in a tiny font or in the same colour as the background to make the text invisible
to browsers.
Tip 9: Expand references
Expand your text references where appropriate e.g. a stamp collecting page might
have references to “collectors” and “collecting”. Expanding
these references to “stamp collectors” and “stamp collecting”
reinforces your strategic keywords in a legitimate and natural manner.
Tip 10: Image maps
Some search engines see the web the way someone using a very old browser might
and may not read image maps which means that some engines may not index any
of your web pages. Make an additional web page with a site map with text links
to everything in your web site. This page can then act as a door for search
engines to your site.
Tip 11: Frames
Frames can also cause a problem for search engines. They cannot always follow
frame links. Again ensure there is an alternative door to your site to allow
these engines to enter and index your site. Frames can either prevent search
engines from finding pages within a site, or cause them to send visitors into
a site without the proper frame “context” being established.
One solution to this problem is to add meta tags to the master page. However, meta tags are only a partial solution, because not all the search engines support them. If you want to know more visit: http://www.w3.org/TR/REChtml40/present/frames.html. This is the official HTML 4.0 specs on frames.
Tip 12: HTML links
Ensure your site has a page with HTML hyperlinks to assist with solving the
image maps and frames problem. This enables search engines to navigate around
your site. If you put them at the bottom of a page, this won’t ruin the
design.
Tip 13: Dynamic Doorblock
Generating pages via CGI or database-delivery? Once again expect that some of
the search engines won’t be able to index them. Consider creating static
pages whenever possible (or in addition to), perhaps using the database to update
the pages, not to generate them on the fly.
Tip 14: Avoid Symbols
Avoid symbols in your URLs especially the “?” symbol. Search engines
tend to choke on it.
Tip 15: No Spamming
Search engine spamming normally centres around being top ranked for extremely
popular keywords. Spamming doesn’t always work with search engines. It
can also backfire. Search engines may detect your spamming attempt and penalise
or ban your page from their listings. Spamming techniques include: repeating
keywords in invisible text and tiny text.
Tip 16: Submitting your pages
Submit your top two or three pages that best summarise your site to each search
engine. Most search engines will index the other pages by following the links
on the submitted pages.
Tip 17: Don’t trust submission processes
Don’t trust the submission process to automated programs and services.
Some of them are excellent, but the major search engines are too important.
There aren’t that many, so submit manually, so that you can see if there
are any problems reported.
Tip 18: Verify and maintain your listing
Check on your pages and ensure they get listed. Once your pages are listed in
a search engine, monitor your listing every now and then. Occasionally strange
things happen and pages disappear from catalogs. If your pages disappear, resubmit
them.
Tip 19: Country listings
A number of the major search engines are now providing country-specific versions
of their directories. These mainly filter sites by domain. For example, for
the British edition only sites with a domain of .co.uk would be listed.
Tip 20: Bridge Page
Also known as entry page, jump page, portal page or gateway page. Submitting
a “bridge” page to search engines can improve traffic. These pages
are created to do well for particular phrases. They are designed primarily for
search engines. There are various ways to deliver bridge pages. The easiest
way is to create and submit a page for a particular phrase.
One problem with this is that these pages tend to be very generic. Another problem is that users don’t arrive at the goal page. To get surfers to the content page, webmasters usually propel visitors forward with a prominent “Click Here” link or with a fast meta refresh command.
Some search engines no longer accept pages using fast meta refresh, to curb abuses of bridge pages. To get around that, some webmasters submit a page, then swap it on the server with the “real” page once a position in the search engine has been achieved. This is “code-swapping”, which is also sometimes done to keep others from learning exactly how the page achieved good ranking. The downside is that if a search engine may revisit at any time, and if it indexes the “real” page, the position may drop.
Simply taking meta tags from a page (“meta jacking”) does not guarantee a page will do well. In fact sometimes resubmitting the exact page from another location does not gain the same position as the original page.
Tip 21: Link popularity
Get loads of HTML links pointing to your site from others. Excite uses link
popularity as part of its ranking method. It can tell which of the pages in
its index have a lot of links pointing at them. These pages are given a slight
boost during ranking, since a page with many links to it is probably well-regarded.
Tip 22: Web site reviews
Request a web site review. Some hybrid search engines which have associated
directories may give a boost to sites they have reviewed.
Conclusion
Search engines are a primary way people look for web sites but they are not
the only way. Whilst it is worth taking some time and trouble to make your site
search engine friendly, it is also worth negotiating reciprocal links with complementary
business sites as links are what the web was built on and remain one of the
best ways for people to find your site. Don’t obsess over your ranking.
People also find sites through word of mouth, traditional advertising, the traditional
media, a newsgroup posting, web directories and links from other sites.
Elkie Holland
Last Updated: 14 July 1999