How Google Indexes Your Site
How Google Indexes Your Site
Google has three well known bots: The Adsense bot, the Freshbot and the DeepCrawl.
The Adsense bot, as you could probably guess, is used for publishers who have Adsense on their sites. As soon as a new page is created, the JavaScript within the Adsense code sends a message to the Adsense bot, and it will come within 15 minutes to index the page so that it can serve up the most relevant ads.
But, for this conversation we are only concerned about the DeepCrawl and the Freshbot.
The Freshbot crawls the most popular pages on your website. It doesn't matter if that is one page or thousands. Sites like Amazon.com and CNN.com have pages that are crawled every ten minutes, since Google has learned that those pages have that amount of frequent changes. A typical site should expect to have a freshbot visit every 1 to 14 days, depending on how popular those pages are.
What happens to your site on a Freshbot visit is that it finds all of the deeper links in your site. It places those links into a database so that when the DeepCrawl occurs, it has a reference.
Once a month, the DeepCrawl bot visits your site and goes over all the links found by the Freshbot. This is the reason why it can take up to a month for your entire site to be indexed in Google - even with the addition of a Google Sitemap.
So, be patient and keep on adding content to your site, and work on getting valuable in-bound links to your site - Google will reward you for it.
-To your online success!
Paul Bliss
www.SEOforGoogle.com
How Google Indexes Your Site - To learn more about this author, visit Paul Bliss's Website.
Like this article? Share it with your friends
First off, let's describe what we are talking about. A "bot" is a piece of software from a search engine that is built to go through every page of your site, categorize it, and place it into a database.
Google has three well known bots: The Adsense bot, the Freshbot and the DeepCrawl.
The Adsense bot, as you could probably guess, is used for publishers who have Adsense on their sites. As soon as a new page is created, the JavaScript within the Adsense code sends a message to the Adsense bot, and it will come within 15 minutes to index the page so that it can serve up the most relevant ads.
But, for this conversation we are only concerned about the DeepCrawl and the Freshbot.
The Freshbot crawls the most popular pages on your website. It doesn't matter if that is one page or thousands. Sites like Amazon.com and CNN.com have pages that are crawled every ten minutes, since Google has learned that those pages have that amount of frequent changes. A typical site should expect to have a freshbot visit every 1 to 14 days, depending on how popular those pages are.
What happens to your site on a Freshbot visit is that it finds all of the deeper links in your site. It places those links into a database so that when the DeepCrawl occurs, it has a reference.
Once a month, the DeepCrawl bot visits your site and goes over all the links found by the Freshbot. This is the reason why it can take up to a month for your entire site to be indexed in Google - even with the addition of a Google Sitemap.
So, be patient and keep on adding content to your site, and work on getting valuable in-bound links to your site - Google will reward you for it.
-To your online success!
Paul Bliss
www.SEOforGoogle.com
How Google Indexes Your Site - To learn more about this author, visit Paul Bliss's Website.
Like this article? Share it with your friends
 | |
 |
No article feedback found. |
| |
Leave Your Feedback |
 | |
| |||
|
I've created this section on my site to share some of the incredible tools that I've used to build my business. I hope you too can benefit from them and look forward to hearing your feedback on the reviews! - Visit Evan Carmichael's Website |
|||
Cheryl MatthynssensCheryl is a life skills coach, licensed Chemical Dependency Counselor and a 20 year entrepreneur. Cheryl's dedication to achieving a life of balance led to her expanding her teaching from the simple managing of life's daily challenges to adding financial well being as well. A direct marketer with DrinkACT, she is gaining ground in the online community with her concepts of making sure business owners, entreprenuers and employees have well rounded life styles. She opened up a small affiliate site - The Balance Guide- to help others find resources for mental and emotional well being. Visit Cheryl's blog to see more of the diversity beyond business she has began offering online at www.thebalanceguide.blogspot.com - Visit Cheryl Matthynssens's Website |
|||
John BrennanJohn Brennan Ed.D. Dr. Brennan is President of Interpersonal Development, LLC, a training and development firm. Interpersonal Development has provided sales training and coaching to more than 3,000 sales reps from over 100 companies. A native of Australia, Dr. Brennan received his doctorate from the University of Rochester. His dissertation researched the effectiveness of Behavioral Modeling Technology in training people in interpersonal skills. While he has spent most of his career designing or delivering training, he was also a Vice-President of Sales of a training and development franchise with operations in 25 markets. Dr. Brennan has designed and delivered sales training in North America, Asia, Europe, Australia and the Middle East. He has been a guest speaker at numerous national and regional professional conferences. When Microsoft wanted Best Practices articles on sales for their web site, they called Dr. Brennan. The results are at http://office.microsoft.com/en-us/FX011387391033.aspx His firm’s clients have included Volvo, The Prudential, Merrill Lynch, Eastman Kodak, Gannett, Equifax Europe, the Economist Group and countless small businesses. - Visit John Brennan's Website |
|||
|
To learn more about the Evan Elite Author Program please contact us. | |||
 | |
|  |
Subscribe to Paul's articles
 | |
|
|  |
 | |
|
| |
 | |||||||
|
 | ||
 |
||
 |
||
 |
||
|
| ||
 |
| Have you written articles that would be of value to entrepreneurs? Become an expert on our site by publishing them! Expose yourself to a wide audience, drive more traffic to your website and get more sales! Click Here for details. |
|
|
 |
| Modeling the Masters: Learn the true secrets behind Walt Disney's business success factors & grow your company! Video produced by Phanta Media |
|
|
 |
"Learn straight from Evan how you can Make a Full Time Income (And More) from a Website"
Click Here To Learn More |
|
|
|
|
Get advice & tips from famous business owners, new articles by entrepreneur experts, my latest website updates, & special sneak peaks at what's to come!
|
|
|
 Canadian Leadership Woman business success Vertical Market Ad Network SWOT Analysis Template Cash Flow Forecast Samples |
|
 | ||
 |
Choose A PR Topic
Press Release Builder | |
 |
Top 50 Productivity Blogs
Top Blogs To Watch In 2008 | |
| ||
 | ||||
| ||||
|
| ||||
| ||||
|
|
|
|
|
||||||||||||
|
Email us your ideas on how to make our website more valuable! Thank you Sharon from Toronto Salsa Lessons / Classes for your suggestions to make the newsletter look like the website and profile younger entrepreneurs like Jennifer Lopez and Sean Combs!
|
|
|
| ||||||||||||
| ||||||||||||
|
|
||||||||||||