Google On When Robots.txt Is Unreachable, Other Pages Reachability Matter
There is this interesting conversation on LinkedIn around a robots.txt serves a 503 for two months and the rest of the site is available. Gary Illyes from Google said that when other pages on the site are reachable and available, that makes a big difference, but when those other pages are not, then “you’re out … Read more
