OpenAI, Google, Microsoft and Anthropic pledged on Thursday to give news publishers and other websites a single, standard way to stop their content being scraped to train artificial intelligence models.
Publishers currently have to set out separate blocking instructions for every bot that crawls the web collecting data. The proposed arrangement would let them enter one standardised instruction, which any AI company that adopts the protocol would recognise and honour. Agence France-Presse reported the announcement, in copy carried by The Straits Times.
It comes as publishers face a surge in bots gathering their articles to feed AI models. Some have gone to court. In December 2023 the New York Times sued OpenAI and Microsoft, alleging that millions of its articles had been used without authorisation or payment to train ChatGPT. The two companies are invoking the copyright exception known as fair use.
The change will not take effect quickly. Microsoft will implement the standard from 1 February 2027 and Google at some point during 2027. OpenAI said it would roll it out "in the coming months", while Anthropic said it had "already started".
Meta is not among the four but signalled support. "We believe creating a common signal that conveys such preferences to multiple providers at once is an important step," the company's spokesman Andy Stone wrote on X. "And we look forward to working with other AI companies on the standards and their implementation across the industry."
Other mechanisms already exist. The internet infrastructure provider Cloudflare has blocked AI bots by default on new customer sites since July 2025, and a standard called Really Simple Licensing allows publishers to charge bots that visit their pages.
The new setting has limits, and the companies have stated them plainly. It applies only to the future training of models and does not prevent bots from crawling a site otherwise. Content scraping will be allowed by default until a publisher changes the setting, Microsoft said in a blog post.
Microsoft and Google also said, in separate posts, that opting out of AI training could reduce the chances of a publisher's content surfacing in AI apps and experiences.
Taken together, those terms describe a permission system of unusual construction. Consent is granted unless withdrawn. Withdrawal is available only to publishers who go and find the switch. And the switch carries a stated cost to anyone who uses it.
The protocol is also self-selecting in a way that deserves quoting in full: the single instruction will be recognised and honoured by any AI company that adopts the protocol. A publisher's refusal therefore binds precisely those companies that have already agreed to be bound by refusals.
This newspaper regards it as a clear advance on the previous position, under which a publisher had to write the same refusal out many times over to achieve the same result.
Our analysts note that the refusal does not stop the crawling. The bots will continue to arrive and read the page. They will simply decline to learn from it afterwards, at a stage that takes place once the visit is over.
There is a final elegance to it. Because opting out may reduce a publisher's visibility in AI products, the industry now has a single, interoperable way to express a preference, and a single, interoperable reason not to express it.
Microsoft begins honouring the instruction on 1 February 2027. Until then, and by default afterwards, the answer is yes.