8 Steps to Getting Your Web Site Crawled
8 Steps to Getting Your Web Site Crawled
Every SE has its own web crawler program and its own formula for ranking pages. However, while page factors might be ranked differently by different engines, the idea behind the web crawlers is basically the same. Getting your site crawled often is a good indicator that the SE finds your site important. And since important sites get better rankings, getting your site crawled frequently is definitely a good thing.
While there is no way you can make the SE crawl your site, there are a few things you can to do encourage the spiders to visit your site more often:
First and foremost, keep your content up to date. Add unique content often and on a regular basis. And be sure to ping Google each time you update your site.
Get as many backlinks as possible from other relevant sites that are crawled regularly. There are several tools available that will track and measure the crawl rate of other websites.
Use a unique title tag for each page. Also, while not as important as the title, each page should also have its own keywords and description tag. This will help the spiders "know" that each page is a unique and individual entity.
Keep your pages as small as possible. The spider will not spend an unlimited amount of time at your web site. If it's held up by your huge images, monstrous PDFs or even excessive text, it will simply abandon the page and move on.
Pay close attention to your internal links and avoid any duplicate content issues. If you have multiple pages with the same content (often used for testing and/or tracking purposes) be sure to use your robots text file to keep the SEs away.
Create a sitemap for the content you want crawled and make sure your content is compliant.
Monitor your web server. Obviously, your server needs to be up and running when the spiders come a-crawling. There are many commercial providers who will monitor your server and notify you when there is a problem. In addition, ensure that your server handles all of your error pages correctly. Refer to the Google Webmaster Tools report of unreached pages for an idea of what errors are being returned.
And while we're discussing Google Webmaster Tools, don't forget to use it to monitor your crawl rate, adjust it if necessary, test and track and see what works best for your site.
Time to implement: As with all SE optimization, working toward an optimal crawl rate is not a one-time job. You will need to add content approximately three times a week and update your site map as needed. Monitoring your web server is an ongoing task.
8 Steps to Getting Your Web Site Crawled - To learn more about this author, visit Karen Scharf's Website.
Like this article? Share it with your friends
In order to analyze the content on your web site and figure out which pages to display (and in which order) for certain search terms, the search engines (SEs) use a program called a web crawler. You might have also heard these referred to as spiders or web-bots.
Every SE has its own web crawler program and its own formula for ranking pages. However, while page factors might be ranked differently by different engines, the idea behind the web crawlers is basically the same. Getting your site crawled often is a good indicator that the SE finds your site important. And since important sites get better rankings, getting your site crawled frequently is definitely a good thing.
While there is no way you can make the SE crawl your site, there are a few things you can to do encourage the spiders to visit your site more often:
First and foremost, keep your content up to date. Add unique content often and on a regular basis. And be sure to ping Google each time you update your site.
Get as many backlinks as possible from other relevant sites that are crawled regularly. There are several tools available that will track and measure the crawl rate of other websites.
Use a unique title tag for each page. Also, while not as important as the title, each page should also have its own keywords and description tag. This will help the spiders "know" that each page is a unique and individual entity.
Keep your pages as small as possible. The spider will not spend an unlimited amount of time at your web site. If it's held up by your huge images, monstrous PDFs or even excessive text, it will simply abandon the page and move on.
Pay close attention to your internal links and avoid any duplicate content issues. If you have multiple pages with the same content (often used for testing and/or tracking purposes) be sure to use your robots text file to keep the SEs away.
Create a sitemap for the content you want crawled and make sure your content is compliant.
Monitor your web server. Obviously, your server needs to be up and running when the spiders come a-crawling. There are many commercial providers who will monitor your server and notify you when there is a problem. In addition, ensure that your server handles all of your error pages correctly. Refer to the Google Webmaster Tools report of unreached pages for an idea of what errors are being returned.
And while we're discussing Google Webmaster Tools, don't forget to use it to monitor your crawl rate, adjust it if necessary, test and track and see what works best for your site.
Time to implement: As with all SE optimization, working toward an optimal crawl rate is not a one-time job. You will need to add content approximately three times a week and update your site map as needed. Monitoring your web server is an ongoing task.
8 Steps to Getting Your Web Site Crawled - To learn more about this author, visit Karen Scharf's Website.
Like this article? Share it with your friends
 | |
 |
No article feedback found. |
| |
Leave Your Feedback |
 | |
| |||
|
I've created this section on my site to share some of the incredible tools that I've used to build my business. I hope you too can benefit from them and look forward to hearing your feedback on the reviews! - Visit Evan Carmichael's Website |
|||
Cheryl MatthynssensCheryl is a life skills coach, licensed Chemical Dependency Counselor and a 20 year entrepreneur. Cheryl's dedication to achieving a life of balance led to her expanding her teaching from the simple managing of life's daily challenges to adding financial well being as well. A direct marketer with DrinkACT, she is gaining ground in the online community with her concepts of making sure business owners, entreprenuers and employees have well rounded life styles. She opened up a small affiliate site - The Balance Guide- to help others find resources for mental and emotional well being. Visit Cheryl's blog to see more of the diversity beyond business she has began offering online at www.thebalanceguide.blogspot.com - Visit Cheryl Matthynssens's Website |
|||
John BrennanJohn Brennan Ed.D. Dr. Brennan is President of Interpersonal Development, LLC, a training and development firm. Interpersonal Development has provided sales training and coaching to more than 3,000 sales reps from over 100 companies. A native of Australia, Dr. Brennan received his doctorate from the University of Rochester. His dissertation researched the effectiveness of Behavioral Modeling Technology in training people in interpersonal skills. While he has spent most of his career designing or delivering training, he was also a Vice-President of Sales of a training and development franchise with operations in 25 markets. Dr. Brennan has designed and delivered sales training in North America, Asia, Europe, Australia and the Middle East. He has been a guest speaker at numerous national and regional professional conferences. When Microsoft wanted Best Practices articles on sales for their web site, they called Dr. Brennan. The results are at http://office.microsoft.com/en-us/FX011387391033.aspx His firm’s clients have included Volvo, The Prudential, Merrill Lynch, Eastman Kodak, Gannett, Equifax Europe, the Economist Group and countless small businesses. - Visit John Brennan's Website |
|||
|
To learn more about the Evan Elite Author Program please contact us. | |||
 | |
|  |
Subscribe to Karen's articles
 | |
|
|  |
 | |
|
| |
 | ||
 |
||
 |
||
 |
||
|
| ||
 |
| Have you written articles that would be of value to entrepreneurs? Become an expert on our site by publishing them! Expose yourself to a wide audience, drive more traffic to your website and get more sales! Click Here for details. |
|
|
 |
| Modeling the Masters: Learn the true secrets behind Walt Disney's business success factors & grow your company! Video produced by Phanta Media |
|
|
 |
"Learn straight from Evan how you can Make a Full Time Income (And More) from a Website"
Click Here To Learn More |
|
|
|
|
Get advice & tips from famous business owners, new articles by entrepreneur experts, my latest website updates, & special sneak peaks at what's to come!
|
|
 |
 The Art Of Networking Pump Up Your Calendar Franchise & Distribution The Leadership Edge Balanced Leadership |
|
 | ||
 |
Top 50 Marketing Blogs
Top Marketing Blogs of 2009 | |
 |
The Top 10 GTD Times Posts
Best Posts for Productivity | |
| ||
 | ||||
| ||||
|
| ||||
| ||||
|
|
|
|
|
||||||||||||
|
Email us your ideas on how to make our website more valuable! Thank you Sharon from Toronto Salsa Lessons / Classes for your suggestions to make the newsletter look like the website and profile younger entrepreneurs like Jennifer Lopez and Sean Combs!
|
|
|
| ||||||||||||
| ||||||||||||
|
|
||||||||||||