Posted by
Shiri Berzack
In the past, Google has warned against blocking Googlebot from crawling specific aspects of your website, and now the warning has been extended to mobile sites. Google’s Zineb Ait Bahajji announced on Google Plus, that blocking Googlebot for Smartphones will have a negative effect on the snippet display for your website in the mobile SERPs.
The post mentions that Google has noticed mobile sites blocking their crawlers from seeing important information such as Javascript, CSS and image files. It is “for that reason, starting today, we’ll display a message in place of the description snippet for mobile search results blocking Googlebot for smartphones, just as we already do for search results blocking Googlebot.” The message that is displayed (as seen below) reads: A description for this result is not available because of this site’s robots.txt – learn more.
Google promotes transparency and allowing them to read your entire website, without blocking the googlebot from certain areas, is of vital importance. The Googlebot for Smartphones follows robots.txt and other guidelines similar to the desktop version. In order to be sure that you’re adhering to the correct Google guidelines, we recommend checking your mobile site with the Fetch as Google feature, with the “Mobile: smartphone” option selected.
For further information, or to ask questions, you are able to visit Google’s Webmaster forum.
Is your mobile site blocking the Googlebot?
About The Author
