One-sentence summary
This guide walks web admins through configuring the common AI-training opt-out signal announced October 30, 2026 in robots.txt, while keeping existing rules like Google-Extended during the transition.
Steps
Step 1: Locate robots.txt Log into the server and find robots.txt in the site root (e.g. example.com/robots.txt).
Step 2: Back up the file Copy robots.txt before editing so you can roll back.
Step 3: Keep existing controls Keep existing rules such as Google-Extended to exclude Gemini training during the transition.
Step 4: Follow the unified label spec The common policy label is still discussed at the IETF AI-preferences working group; add the rule once OpenAI, Google and Microsoft fully support it.
Step 5: Validate and deploy Use search-engine robots testers to confirm only AI training is opted out and regular indexing is unaffected, then deploy.
Step 6: Review regularly Re-check rules as vendors roll out support, to avoid accidentally blocking legitimate search crawlers.