What Search Engines Don’t Like About Your Website

You may not know it, but you could be hindering or preventing your website from being crawled by search engine spiders. As spiders crawl the web, they rely on the architecture of hyperlinks to find new web pages and revisit those that may have changed. Complex links and deep site structures with little unique content may act as “speed bumps” in the process by slowing down the spiders. Even worse, data that cannot be accessed by web crawlers are really like “walls” in that they completely prevent your web pages from being ranked.

Beware of the Following “Speed Bumps”:

• URLs with 2+ dynamic parameters; i.e. http://www.url.com/page.php?id=4&CK=34rr&User=%Tom% (spiders may be reluctant to crawl complex URLs like this because they often result in errors with non-human visitors)
• Pages with more than 100 unique links to other pages on the site (spiders may not follow each one)
• Pages buried more than 3 clicks/links from the home page of a website (unless there are many other external links pointing to the site, spiders will often ignore deep pages)
• Pages requiring a “Session ID” or Cookie to enable navigation (spiders may not be able to retain these elements as a browser user can)
• Pages that are split into “frames” can hinder crawling and cause confusion about which pages to rank in the results.

Beware of the Following “Walls”:

• Pages accessible only via a select form and submit button
• Pages requiring a drop down menu (HTML attribute) to access them
• Documents accessible only via a search box
• Documents blocked purposefully (via a robot meta tag or robots.txt file)
• Pages requiring a login
• Pages that re-direct before showing content (search engines call this cloaking or bait-and-switch and may actually ban sites that use this tactic)

In order to avoid the above pitfalls and ensure that your website’s contents are fully crawlable, be sure to provide direct, HTML links to each page you want the search engine spiders to index. Remember to make every page of your site accessible from the home page, since the home page is usually the place spiders begin their crawl. It’s also a good idea to add a sitemap to your website in order to increase its navigation.

For More Information CLICK HERE

This article was written by David Montalvo, Senior SEO and Director of Business Development at Active Web Group.