On 30 October 2026, Google announced support for a shared robots.txt rule on the Google Search Central Blog. The rule lets you tell multiple AI providers at once whether they can use your content to train their AI models.
Previously, you had to block each AI developer separately. This effort comes from the IETF AIPREF working group. If you use the shared rule to opt out of training, Google will treat it as an opt-out for Google-Extended, unless you configure Google-Extended differently yourself. Google introduced its own setting in 2023, and it will keep working alongside the new rule. Websites that allow Google-Extended remain eligible for citations in Gemini Apps.
Practical implications for business owners
Managing settings for AI crawlers will get simpler. You will no longer need to add a new rule for every bot. A single rule lets you state your choices to participating providers.
If you run an installation company in The Hague and want to stop AI models using your text for training, this agreement will soon let you manage that centrally. You will not need to maintain individual rules for every AI developer.
The setting has limits. It is a proposed open standard being rolled out in phases, so not all AI companies will process the rule straight away. It also only affects model training, not whether links to your site appear in answers.
Steps you can take
You do not need to update your robots.txt file straight away, as support is coming out in stages.
- Check which AI bots currently have access through your robots.txt file.
- Decide whether to block your content from training or opt for visibility through services.
- Track updates as different AI providers confirm full support.
Source: Google Search Central
