Google limits robots.txt support to four fields, clarifying its stance on unsupported directives.
- Google only supports four specific robots.txt fields.
- Unsupported directives in robots.txt will be ignored.
- Consider auditing your robots.txt files in light of this update.
Source link : Searchenginejournal.com
