Google’s Gary Illyes answered a question about the non-search features that the GoogleOther crawler supports, then added a caution about the consequences of blocking GoogleOther. What Is GoogleOther? GoogleOther is a generic crawler created by Google for the various purposes that fall outside of those of bots that specialize for Search, Ads, Video, Images, News, … Read more
Google has launched GoogleOther, a new web crawler. GoogleOther will be used internally by various Google teams to crawl the public web, said Google’s Gary Illyes on LinkedIn. It respects the same directives and protocols of the main Googlebot crawler and will free up some resources for the main Googlebot crawlers. GoogleOther. Google added GoogleOther … Read more
Google has added a new crawler to its list of Google Crawlers and user agents, this one is named GoogleOther. It is described as a “generic crawler that may be used by various product teams for fetching publicly accessible content from sites.” For example, it may be used for one-off crawls for internal research and … Read more
421
229
714
826
543