KKnowFlowHub
DatasetAuthorKnowFlowHub

robots.txt Common AI Opt-Out Vendor Adoption Timeline Dataset (October 2026)

Summarizes adoption status and timelines of OpenAI, Google, Microsoft, Anthropic and Meta for the common robots.txt AI-training opt-out signal announced October 30, 2026.

robots.txtAI训练厂商采纳数据

One-sentence summary

This dataset summarizes vendor adoption of the common robots.txt AI-training opt-out signal announced October 30, 2026: Anthropic already adopted, Google and Microsoft plan 2027, OpenAI in coming months, and Meta supports but has not committed.

Notes ​

  • The signal is a common policy label in robots.txt, discussed in the IETF AI-preferences working group.
  • Announced October 30, 2026; timing per reports including The Verge.
  • Each vendor's existing controls remain.

Core Data ​

modelAdoption statusExpectedNote
AnthropicAdoptedLiveFirst to ship
GooglePlans to support2027Keeps Google-Extended
MicrosoftPlans to support2027Adds training/anchoring opt-outs
OpenAIVoluntary joinComing monthsNo firm date
MetaSupports the ideaNot committedPush commercial terms to Nov 20

Field Notes ​

  • Google-Extended: Google's crawler control since 2023; allowed content may appear in Gemini answers.
  • Microsoft granularity: Sites can opt out of training and anchoring separately without affecting Bing indexing.
  • IETF process: Remaining AI-preference vocabulary is still under discussion.

Sources ​

AuthorKnowFlowHub
Published
KEEP READING

Keep reading

Article

From Per-Vendor Rules to One-Step Opt-Out: What the Common robots.txt AI Signal Means

Compare

Common robots.txt AI Opt-Out vs Google-Extended vs Per-Vendor Crawler Blocking

FAQ

robots.txt Common AI-Training Opt-Out Signal FAQ

COPYRIGHT

Copyright & Infringement Notice

Content on this site is compiled from public sources for reference and learning. If you believe any content on this site infringes your rights (including but not limited to text, data, images and tables), please contact us and we will handle it promptly after verification:

  • Email us at: liu@chenra.com
  • In the email, please specify: the exact page URL of the infringing content, your proof of rights (e.g. copyright certificate, authorization document), and your contact information
  • We will verify and process within a reasonable period after receiving a valid notice, including removal or source attribution

All content on this site is attributed to its data sources. If any source is inaccurate or you believe a citation exceeds a reasonable scope, please let us know.