A Little About Search Engines--Robots and Spiders
The World Wide Web without search engines would be one vast repository of files. Epic struggles would be waged with researching, well, pretty much anything. Search engines like Google and Yahoo! are the big industrial-strength workhorses of the Internet. They make sense of the huge file cabinet in cyberspace, making the task of finding persons, places or things a rather simple pasttime. If nothing else, these tools certainly narrow down the search.Search engines employ a few different strategies to organize their search findings. Some, like Google, use spiders-- a software that "crawls" the World Wide Web indexing new websites and updating the indices of any sites where the crawler detects updates. These are returned to the user in a ranked order, sometimes referred to as search engine page rankings. Up until a few months ago, I gave relatively little thought to the whys and hows a website ended up at the top of the search engine rankings heap. I thought, "Whatever.... just gimme my information and our relationship will be over." But now that I am wading about in the gummy waters of web content writing I've had the big "Ah-hah!" come over me. What are these little buggers, anyway and who the hell is the "Googlebot"?
What do search engines look for?
They don't all look for the same thing. Some search engines look at the site's text only. Others use a combination of site text, meta tags and page titles to index and catalog. And who are "they"? Google, AltaVista, HotBot, Lycos, MSN, Yahoo! (calls itself a "directory"), and lesser known search engines like MixCat and Surf Gopher. There are dozens of others.
The "Googlebot" is the affectionate name given to Google's website crawler. This software looks at web directories for newly added URLs and any updated websites already indexed. When you or I type in a search word or phrase into Google's "Search" box, we are actually querying the index or database of text, lots of text. The Googlebot returns those results it finds to be most relevant, current and popular based on the initial query. Returning a high Google page ranking is the hope of all websites, ("When I grow up, I wanna be..."), which is why Search Engine Optimization is big business today.
Where Am I?
So, where is my blogsite in relation to the search engines? Nail-biting time. Drum roll....
Nowhere, really. In Google, I type the site url and still find nothing. My other blogsite, Strong Coffee, at least gets found, so it has some sort of index. That's my agonizing admission for today. Time to get to work.
For detailed information on the Googlebot and how it works, take a look at Google Guide, http://www.googleguide.com/google_works.html
As far as meta tags go, this is one of the best articles I've found on the topic, from Search Engine Watch, http://searchenginewatch.com/webmasters/article.php/2167931
posted by jrotman @ 1:15 PM
<< Home