First off, let's describe what we are talking about. A "bot" is a piece of software from a search engine that is built to go through every page of your site, categorize it, and place it into a database.
Google has three well known bots: The Adsense bot, the Freshbot and the DeepCrawl.
The Adsense bot, as you could probably guess, is used for publishers who have Adsense on their sites. As soon as a new page is created, the JavaScript within the Adsense code sends a message to the Adsense bot, and it will come within 15 minutes to index the page so that it can serve up the most relevant ads.
But, for this conversation we are only concerned about the DeepCrawl and the Freshbot.
The Freshbot crawls the most popular pages on your website. It doesn't matter if that is one page or thousands. Sites like Amazon.com and CNN.com have pages that are crawled every ten minutes, since Google has learned that those pages have that amount of frequent changes. A typical site should expect to have a freshbot visit every 1 to 14 days, depending on how popular those pages are.
What happens to your site on a Freshbot visit is that it finds all of the deeper links in your site. It places those links into a database so that when the DeepCrawl occurs, it has a reference.
Once a month, the DeepCrawl bot visits your site and goes over all the links found by the Freshbot. This is the reason why it can take up to a month for your entire site to be indexed in Google - even with the addition of a Google Sitemap.
So, be patient and keep on adding content to your site, and work on getting valuable in-bound links to your site - Google will reward you for it.
-To your online success!
Paul Bliss www.SEOforGoogle.com
How Google Indexes Your Site - To learn more about this author, visit Paul Bliss's Website.
Like this article? Share it with your friends
 |
Related Articles |
|
 How Important are Sitemaps?
|
| |
A sitemap is a simple but often overlooked tool on many websites. Sitemaps act as a table of content for website visitors and as an index for search engine crawlers to help index your website’s pages.
|
|
Does a Domain Name Help with Higher Rankings?
|
| |
Based on the latest patent filing from Google the answer to this question is yes. Google states that how a site is hosted and the length of time that a domain name has been paid for does help in how a web site ranks...
|
|
SEO - How Important is Your Google Page Rank?
|
| |
The Google Page Rank is an indication of the number of websites that link to yours. It also gives information about the quality of those links. Your Google Page Rank may ranges from 0 to 10.
|
|
How Google Indexes Your Site
|
| |
I can't tell you how many times I've answered this question in forums, so I figured since so many are asking, it would make for a great article.
|
|
Using Local Search to Get Top 10 Rankings
|
| |
Making a small effort can pay off big, when it comes to local search!
|
|
|
Paul Bliss
(Visit Paul's Website)
My name is Paul Bliss and I've been
working with the Internet since 1996. From
graphic design to web programming to
database design, I have seen many changes
in the industry. I became fascinated with
search engines back when Alta Vista was
the top dog, and all you had to do in
order to get high rankings were to stuff
your meta tags with your keywords. I have
successfully placed over 80 clients into
top ranked positions for the past 5 years,
and along the way I have tried just about
everything, and I know what works and what
is just a waste of time.
|
|
|
Paul Bliss's
Complete
List Of
SEO
Articles
|
|
|
If you enjoyed this article, get Paul Bliss's Complete List of SEO Articles For FREE!
|
| |
|
|
|
![[Get Copyright Permissions]](http://license.icopyright.net/images/icopy-w.gif){kind=small}
