Crawlability Issues - How to Make Sure That Search Engines Can Spider Your Site
Search engine spiders aren't THAT smart. Spiders/bots are programmed to constantly crawl around search engines until they find content they can index. They aren't going to stop and wait around until you fix that broken link. That's not their job. It's YOUR job to make sure search engines can spider your site.
In order for your site's pages to be crawled, they need to take visitors to different locations. Visitors should be able to easily navigate within your site or be taken to an external link/site. If you have bad or broken links, this seriously reduces your site's crawlability and your site could be permanently filtered out of SERPs (search engine result pages).
Crawlability errors are what kill your SEO - don't make the following mistakes or say good bye to being found in search engines.
• Pay attention to robots.txt files - That's the easiest way to mess up your crawlability. Robots.txt files direct search engine spiders how to view your site. One mistake can bump you out of search engines permanently so be extra careful when working with these files.
• Don't use too many variables in your URLs - Google can crawl long links but it's still not a fan of long URLs. It's better to use short URLs - they get clicked on more often in SERPs
• Don't add Session IDs in URLS - If you do use Session IDs in your URLs, store them in cookies. Spiders won't crawl URLs with Session IDs - they clog up the SERPs
• Watch out for code bloat - This can be a HUGE problem. Spiders are good at separating code versus content, but don't make it difficult for them either. Don't use so much code that your content isn't easy to find.
• Spiders can't follow Flash links or execute JavaScript - Designs that use Flash and JavaScript will stop spiders in their tracks. They also can't read text or images inside Flash. Stick with HTML links (links should preferably be text or images)
How to Fix Crawlability Mistakes
• Do NOT change your URLs unless they are broken. Keep in mind the following tips mentioned above. Keep them short, don't use too many variables, leave out Session IDs, etc.
• When creating URLs, use keywords and hyphens to separate keywords. Spiders recognize hyphens as spaces - don't use underscores, etc. Don't make your URLs look spammy with keywords.
• Fix your domain canonicalization. Are you going to use both versions of your domain? (don't use both!) Which domain gets the most inbound links, etc? That's the URL you should use as your primary domain. If you use a secondary domain, make sure you include a 301 redirect back to your primary domain.
• Use SEO Browser to check how search engines view your site
The Benefits of Sitemaps
Sitemaps are XML files that list URLS along with each URL's metadata. This allows search engines to crawl sites efficiently. Sitemaps are an easy way for webmasters to inform search engines about pages on their sites that are available for crawling.
If you want to be found easily by search engines, build an efficient navigation system. A logical and easy navigation structure will make it easier for spiders to crawl around your site.
Crawlability Issues How to Make Sure That Search Engines Can Spider Your Site - To learn more about this author, visit Zeke Camusio's Website.
Like this article? Share it with your friends
 | |
 |
No article feedback found. |
| |
Leave Your Feedback |
 | |
| |||
John AlexanderJohn has taught keyword research and SEO skills to small groups of business owners and Webmasters from over 80 different countries world wide since 2002. John is also the Director of Search Engine Academy ; Co-director of Training at Search Engine Workshops offering live, SEO Workshops with his partner SEO educator Robin Nobles, author of the very first comprehensive online search engine marketing courses at SEO Training Online and the SEO Workshop Resource Center. I look forward to hearing from you! - Visit John Alexander's Website |
|||
|
I've created this section on my site to share some of the incredible tools that I've used to build my business. I hope you too can benefit from them and look forward to hearing your feedback on the reviews! - Visit Evan Carmichael's Website |
|||
Joe DagerJoe Dager is President of Business901, a progressive coaching company providing no-nonsense direction in areas such as Lean Six Sigma Marketing and organized referral marketing. What others say: In the past 20 years, Joe and I have collaborated on many difficult issues. Joe’s ability to combine his expertise with “out of the box” thinking is unsurpassed. He has always delivered quickly, cost effectively and with ingenuity. A brilliant mind that is always a pleasure to work with.” - James R. If you want to learn more about Business901, start a conversation with us. We can be found @ Web/Blog: Business901.com Web/Blog: FundingYourNonprofit.com LinkedIn Profile Follow me on Twitter - Visit Joe Dager's Website |
|||
Cheryl MatthynssensCheryl is a life skills coach, licensed Chemical Dependency Counselor and a 20 year entrepreneur. Cheryl's dedication to achieving a life of balance led to her expanding her teaching from the simple managing of life's daily challenges to adding financial well being as well. A direct marketer with DrinkACT, she is gaining ground in the online community with her concepts of making sure business owners, entreprenuers and employees have well rounded life styles. She opened up a small affiliate site - The Balance Guide- to help others find resources for mental and emotional well being. Visit Cheryl's blog to see more of the diversity beyond business she has began offering online at www.thebalanceguide.blogspot.com - Visit Cheryl Matthynssens's Website |
|||
John BrennanJohn Brennan Ed.D. Dr. Brennan is President of Interpersonal Development, LLC, a training and development firm. Interpersonal Development has provided sales training and coaching to more than 3,000 sales reps from over 100 companies. A native of Australia, Dr. Brennan received his doctorate from the University of Rochester. His dissertation researched the effectiveness of Behavioral Modeling Technology in training people in interpersonal skills. While he has spent most of his career designing or delivering training, he was also a Vice-President of Sales of a training and development franchise with operations in 25 markets. Dr. Brennan has designed and delivered sales training in North America, Asia, Europe, Australia and the Middle East. He has been a guest speaker at numerous national and regional professional conferences. When Microsoft wanted Best Practices articles on sales for their web site, they called Dr. Brennan. The results are at http://office.microsoft.com/en-us/FX011387391033.aspx His firm’s clients have included Volvo, The Prudential, Merrill Lynch, Eastman Kodak, Gannett, Equifax Europe, the Economist Group and countless small businesses. - Visit John Brennan's Website |
|||
Kim CastleWith nearly two decades in the advertising and design business, with clients like Domino's Pizza, General Motors, Direct TV, Pedigree, Wolfgang Puck, Higher Octave Music, Hollywood Celebrity Products, Disney, and Paramount, as well as thousands of entrepreneurs around the world define, structure, communicate, and position their business for greater profits, BrandU(R) co-creators Kim Castle and W. Vito Montone discovered that entrepreneurs could experience the same power that big brands command for a fraction of the cost with the world's only process-based results-drive Integral approach to business creation. BrandU(R) is helping entrepreneurs grow with the power of extreme clarity from idea...to brand...to market(TM) and helping one million entrepreneurs become successful and whole so that they can make a difference in the world. Are you one of them? If you want to experience clarity all the way to the bank(TM), get started now at http://www.brandu.com. - Visit Kim Castle's Website |
|||
Stephanie RobeyStephanie Robey is President and CoFounder of Pivot Positive, LLC - an Internet marketing business focused on helping people start work at home ventures. Previously, she was employed at The Search Agency with over 20 years experience in graphic design and 10 years experience in online marketing. She was responsible for launching the Conversion Path Optimization (CPO) unit where she and her team have conducted hundreds of optimization tests for online companies across multiple verticals. She is a successful entrepreneur having started and sold 2 companies and remains on the board of directors of the third, PhotoSpin.com Stephanie began her career in the direct marketing realm creating and producing direct mail for many of the major cable television companies and directly attributes her understanding of Internet marketing to those early offline experiences. Stephanie is a graduate of San Diego State University with a BFA in Graphic Arts and also holds an Executive MBA from the Graziadio School of Business and Management at Pepperdine University. Read Steph's Blog Meet Steph and Dave Sign up for our Free 7-Day BootCamp: Self Employed & Rich - Visit Stephanie Robey's Website |
|||
George LudwigGeorge Ludwig is a recognized authority on sales strategy and peak performance psychology. An international speaker, trainer, and corporate consultant, he helps clients like Johnson & Johnson, Abbott Laboratories, Northwestern Mutual, CIGNA, and numerous others improve sales force effectiveness and performance. Though it's George's strategies and processes that help corporations increase productivity and performance, it's his tremendous energy and dynamism that spark the transformation. Again and again, clients remark on his amazing ability to unleash human capacity and inspire men and women to break out of their comfort zones. The result is a whole new type of salesperson. His customized presentations teach achievers to make stunning advances in their lives. From helping salespeople realize cherished dreams to helping corporations exponentially accelerate revenue streams, George Ludwig leaves audiences and individuals empowered, emboldened, and clamoring for more. George is the best-selling author of Power Selling: Seven Strategies for Cracking the Sales Code and Wise Moves: 60 Quick Tips to Improve Your Position in Life & Business. - Visit George Ludwig's Website |
|||
|
To learn more about the Evan Elite Author Program please contact us. | |||
 | |
|  |
Subscribe to Zeke's articles
 | |
|
|  |
 | |
|
| |
 | |||||||
|
 | ||
 |
||
|
||
 |
||
|
| ||
 |
| Have you written articles that would be of value to entrepreneurs? Become an expert on our site by publishing them! Expose yourself to a wide audience, drive more traffic to your website and get more sales! Click Here for details. |
|
|
 |
| Modeling the Masters: Learn the true secrets behind Walt Disney's business success factors & grow your company! Video produced by Phanta Media |
|
|
 |
"Learn straight from Evan how you can Make a Full Time Income (And More) from a Website"
Click Here To Learn More |
|
|
|
|
Get advice & tips from famous business owners, new articles by entrepreneur experts, my latest website updates, & special sneak peaks at what's to come!
|
|
|
 Making Life Work Strategic Planning Guide Visual Pipeline How To Work Delegation Workbook |
|
 | ||
 |
Top 50 SEO Posts - 2007
Top SEO Posts of the Year | |
 |
Top 50 Niche Blogs 2009
Top 50 Niche Blogs 2009 | |
| ||
 | ||||
| ||||
|
| ||||
| ||||
|
|
|
|
|
||||||||||||
|
Email us your ideas on how to make our website more valuable! Thank you Sharon from Toronto Salsa Lessons / Classes for your suggestions to make the newsletter look like the website and profile younger entrepreneurs like Jennifer Lopez and Sean Combs!
|
|
|
