The Wikimedia Foundation, the non-profit that hosts Wikipedia and other open knowledge projects, filed an amicus brief on 30 October with the Southern District Court of New York in In re: OpenAI, Inc. Copyright Infringement Litigation, twelve combined lawsuits of which the best known is The New York Times Co. v Microsoft and OpenAI.
The court will rule on whether the use of copyrighted content by artificial intelligence companies to develop large language models constitutes fair use. The Foundation said the ruling will be highly influential, given the significance of the jurisdiction, the parties involved, and the small number of cases that have addressed the question so far.
The underlying suit, originally filed by The New York Times in 2023, alleges that OpenAI’s and Microsoft’s use of Times-owned content for any purpose, including AI model training, is a direct violation of copyright law. The two companies assert that training and deploying models based on publicly available data constitutes technological development and falls within fair use.
The Foundation declines both positions. Its brief argues that a ruling for a too restrictive version of fair use could inhibit technological innovation and limit the availability of tools that benefit open source projects such as Wikipedia, while a ruling in favour of blanket fair use could lead to a more closed web, in which online knowledge is weakened by a lack of available secondary source materials.
Its alternative applies fair use to two specific cases. For model training, the Foundation argues that using copyrighted content should be considered fair if it is being used to train models that are something new and different from the original sources, with no requirement to meet any licensing terms.
For retrieval-augmented generation, or RAG, and newer techniques that serve a similar grounding purpose, it argues the reverse. RAG finds and copies news content or other works of human knowledge from the web and then delivers that same content back to users through a chatbot interface. Here, the Foundation says, the use should not be considered fair, and companies offering the feature should respect each platform’s or publisher’s licensing requirements. Wikipedia’s licence is Creative Commons CC BY-SA 4.0, which requires credit to the original creators and that adaptations remain under the same licence.
“We are taking a position in this case to defend the free and open human knowledge that Wikipedia depends on,” the Foundation’s chief executive, Bernadette Meehan, said. “Expanding access to knowledge through AI is a powerful goal, but it must respect the people, contributors, and communities that make reliable knowledge possible.”
“Any conversation around fair use and copyright in the age of AI cannot be limited to for-profit organizations: it must include stakeholders across the entire online information ecosystem,” the Foundation’s general counsel, Stephen LaPorte, said. He added that “a ruling that does not apply fair use principles carefully could threaten the sustainability of free knowledge, and stifle the development of open source technologies”.
Wikipedia carries more than 67 million articles across 300 languages, built by nearly 250,000 volunteers a month working from reliable sources, and is among the highest-quality datasets used to train large language models. The Foundation also said that scraping of its projects must be done sustainably and within its public API rate limits, or through its Enterprise offering.
The brief’s test is the most useful thing filed in this litigation so far, and it is worth restating in its own terms. Copying is fair when the result is something new and different from the source. Copying is not fair when the result is the source.
Put the two halves together and a rule emerges that nobody intended. The lawfulness of a copy depends on how far it has succeeded in ceasing to resemble the original. Accuracy becomes the aggravating factor.
Our analysts have applied this to the ordinary case of a reader. A machine that reads Wikipedia and produces a confident approximation of it is innovating. A machine that reads Wikipedia and tells you what it actually says is infringing. The safest way to use an encyclopedia, legally speaking, is to half remember it.
The escalation is in the licence itself. Creative Commons CC BY-SA 4.0 asks for credit, and credit requires knowing what you took. Under the Foundation’s split, the system that cannot attribute anything is the system with nothing left to attribute, because it no longer holds anything recognisable. Compliance is cheapest for whoever has forgotten the most.
The Department of Unnecessary Research has been through the brief twice and can find no error in it, which is precisely the difficulty. The distinction between learning something and copying it is real, and obvious to every human being who has ever sat an examination. It is now being written down for the first time, by people who must say exactly where it falls.
Wikipedia is the most quoted thing ever written and asks only to be credited. The court is being invited to agree that this is a reasonable request, provided the quotation is inaccurate.