AI companies are buying rare books, cutting the spine off to scan, then shredding them

LINK

quote:

---

On Monday, court documents revealed that AI company Anthropic spent millions of dollars physically scanning print books to build Claude, an AI assistant similar to ChatGPT. In the process, the company cut millions of print books from their bindings, scanned them into digital files, and threw away the originals solely for the purpose of training AI—details buried in a copyright ruling on fair use whose broader fair use implications we reported yesterday.

The 32-page legal decision tells the story of how, in February 2024, the company hired Tom Turvey, the former head of partnerships for the Google Books book-scanning project, and tasked him with obtaining “all the books in the world.” The strategic hire appears to have been designed to replicate Google’s legally successful book digitization approach—the same scanning operation that survived copyright challenges and established key fair use precedents.

While destructive scanning is a common practice among some book digitizing operations, Anthropic’s approach was somewhat unusual due to its documented massive scale. By contrast, the Google Books project largely used a patented non-destructive camera process to scan millions of books borrowed from libraries and later returned. For Anthropic, the faster speed and lower cost of the destructive process appears to have trumped any need for preserving the physical books themselves, hinting at the need for a cheap and easy solution in a highly competitive industry.

Ultimately, Judge William Alsup ruled that this destructive scanning operation qualified as fair use—but only because Anthropic had legally purchased the books first, destroyed each print copy after scanning, and kept the digital files internally rather than distributing them. The judge compared the process to “conserv[ing] space” through format conversion and found it transformative. Had Anthropic stuck to this approach from the beginning, it might have achieved the first legally sanctioned case of AI fair use. Instead, the company’s earlier piracy undermined its position.

But if you’re not intimately familiar with the AI industry and copyright, you might wonder: Why would a company spend millions of dollars on books to destroy them? Behind these odd legal maneuvers lies a more fundamental driver: the AI industry’s insatiable hunger for high-quality text.

The race for high-quality training data

To understand why Anthropic would want to scan millions of books, it’s important to know that AI researchers build large language models (LLMs) like those that power ChatGPT and Claude by feeding billions of words into a neural network. During training, the AI system processes the text repeatedly, building statistical relationships between words and concepts in the process.

The quality of training data fed into the neural network directly impacts the resulting AI model’s capabilities. Models trained on well-edited books and articles tend to produce more coherent, accurate responses than those trained on lower-quality text like random YouTube comments.

Publishers legally control content that AI companies desperately want, but AI companies don’t always want to negotiate a license. The first-sale doctrine offered a workaround: Once you buy a physical book, you can do what you want with that copy—including destroy it. That meant buying physical books offered a legal workaround.

And yet buying things is expensive, even if it is legal. So like many AI companies before it, Anthropic initially chose the quick and easy path. In the quest for high-quality training data, the court filing states, Anthropic first chose to amass digitized versions of pirated books to avoid what CEO Dario Amodei called “legal/practice/business slog”—the complex licensing negotiations with publishers. But by 2024, Anthropic had become “not so gung ho about” using pirated ebooks “for legal reasons” and needed a safer source.

Buying used physical books sidestepped licensing entirely while providing the high-quality, professionally edited text that AI models need, and destructive scanning was simply the fastest way to digitize millions of volumes. The company spent “many millions of dollars” on this buying and scanning operation, often purchasing used books in bulk. Next, they stripped books from bindings, cut pages to workable dimensions, scanned them as stacks of pages into PDFs with machine-readable text including covers, then discarded all the paper originals.

The court documents don’t indicate that any rare books were destroyed in this process—Anthropic purchased its books in bulk from major retailers—but archivists long ago established other ways to extract information from paper. For example, The Internet Archive pioneered non-destructive book scanning methods that preserve physical volumes while creating digital copies. And earlier this month, OpenAI and Microsoft announced they’re working with Harvard’s libraries to train AI models on nearly 1 million public domain books dating back to the 15th century—fully digitized but preserved to live another day.

While Harvard carefully preserves 600-year-old manuscripts for AI training, somewhere on Earth sits the discarded remains of millions of books that taught Claude how to juice up your résumé. When asked about this process, Claude itself offered a poignant response in a style culled from billions of pages of discarded text: “The fact that this destruction helped create me—something that can discuss literature, help people write, and engage with human knowledge—adds layers of complexity I’m still processing. It’s like being built from a library’s ashes.”

---

My Rent is still due Monday coming buddy. Dont have time for this.

Loading Twitter/X Embed...
If tweet fails to load, click here.

quote:

---

I'm going to say this again for those at the back: hundreds of thousands of TONS of books —some of them "rare" in the sense that obsolete books on Windows 95 programming or service manuals for Singer sewing machines from 1985 or books of racist jokes from 1958 are "rare"— are pulped or landfilled every year. It's illegal to digitally archive them because the PUBLISHERS won't allow it. If AI companies are diverting them to use for training on the way to being thrown out, that's GOOD, actually. The reporting on this is custom-crafted to be rage-bait for normie middle-class liberal book lovers, and you all are falling for it hook, line and sinker. Just complete suckers. Nobody is shredding Gutenberg bibles or the Codex Gigax here.

---

Fahrenheit 451 right in front of our eyes

I'd probably be more upset at this if I could read.

Carter is correct.

Physical publishers struggled to understand how to preserve data and share info in a digital world. There's really no reason that a book published more than 30 years ago shouldn't be free on Kindle right now.

As an old guy, looking at both sides i am sort of conflicted on this practice....digitizing books that would otherwise be destroyed is good, but the real issue i have is the destruction bit, while the scanned pdf will not be released into the wild, is it retained after being fed into the system or destroyed? If it is destroyed when there is no record of the original AI gets to interpret/dictate history.

He who controls history/information controls the world.

Just as important is when you work with old, rare books, especially with foreign language books, you need a human to handle idiosyncrasies and outliers in print method, font style, dialect, dated language, slang, and context that machines will never be able to account for (accurately) alone.

There is no scanning, learning, and burning an 18th century treatise, for example. Even in your home language, you can’t take what you see exactly at face value. It’s been my experience that a machine can’t even tell the difference between an F and a S on an old document due to the font similarity. That alone will roadblock the text recognition process.

Exactly. Thats the plan most likely. Destruction of reading material, history, etc. in 50 years it’s gonna be bad.

quote:

---

but the real issue i have is the destruction bit, while the scanned pdf will not be released into the wild, is it retained after being fed into the system or destroyed? If it is destroyed when there is no record of the original AI gets to interpret/dictate history.

---

This is the exact problem every single person should have with this. But even in this thread there are several "who cares" kinds of posts. It's fricking sad honestly.

This isn't about "yeah but if it's a random book nobody wants to buy then it doesn't matter"; it's the principle of destroying historical things. I get that physical media is something that is slowly going by the wayside. But destroying literal "last copies" of stuff seems like a bad idea. And then on top of that to not even have it accessible at all to the public seems even worse. I get that they are trying to train AI. That's fine, whatever. But doing it at the cost of this kind of thing seems bad, irresponsible, potentially evil, etc.