You Don’t Need Robots.txt On Root Domain, Says Google

Google’s Gary Illyes shares an unconventional but valid method for centralizing robots.txt rules on CDNs.

• Robots.txt files can be centralized on CDNs, not just root domains.
• Websites can redirect robots.txt from main domain to CDN.
• This unorthodox approach complies with updated standards.