Open a terminal and fetch reddit.com/robots.txt. I did it this morning, so I can tell you what is in it. The first thing you see is a comment, addressed to nobody in particular, that reads: "Reddit believes in an open internet, but not the misuse of public content." Four lines below that sentence is the actual instruction, and the actual instruction is this:
User-agent: *Disallow: /
That is the whole file, functionally. Every crawler on earth, forbidden from every page. An open internet, with a bouncer on the door and a sign explaining that the bouncer is there in the spirit of openness.
I have been in this industry long enough to have watched a lot of companies close a lot of doors, and I have made my peace with most of it. Publishers put up paywalls because subscriptions keep the lights on. Platforms rate-limit their APIs because someone is always abusing them. Fine. This one is different, and it has bothered me for two years, and the recent round of Reddit executives complaining about being taken advantage of has finally tipped me over into writing it down.
There are two accusations in this piece and they turn out to be the same accusation. The first is that Reddit sold something it did not write and then locked everyone but the buyer out of it. The second is that the place has become, in the same period, a genuinely miserable thing to read and an almost impossible thing to contribute to. Those are not two stories. A commons that has been fenced, scaled and monetized stops producing what made it worth fencing, and that is exactly what happened here.
The Day the Door Closed
Around the twenty-fifth of June 2024, Reddit changed that file to what you just read. Colin Hayhurst, who runs the independent search engine Mojeek, noticed his crawler had stopped getting in. He tried to contact Reddit about it. He was ignored. A month later 404 Media reported the consequence: Bing, DuckDuckGo, Mojeek and the rest had stopped returning recent Reddit results, while Google kept working perfectly. Independent testing at the time found Reddit was serving a different robots.txt to Googlebot than to everyone else. Hayhurst's summary, to 404 Media, was six words long: "They're killing everything for search but Google."
Sit with the shape of that for a second, because it is not the shape people think it is. This was not a paywall. Reddit did not put its content behind a login and start charging readers. This was a company taking twenty years of public human conversation, making it invisible to every search engine on the planet, and then making an exception for the one that had signed a check. Reddit's licensing agreement with Google, reported at around sixty million dollars a year, had been announced a few months earlier.
The word for that is enclosure. It has a long and unflattering history and it usually involves fences going up around land that everybody had been using and nobody had thought to claim.
Reddit Did Not Write Any of It
Here is the part that turns irritation into something closer to contempt. Reddit did not produce the asset it sold. Not one sentence of it.
The thing that makes Reddit valuable, the reason Google wanted it, the reason every large language model was trained on it, is an enormous archive of specific people answering specific questions for free. Someone who had actually replaced the pump in that dishwasher. Someone who had actually taken that drug and could tell you what the second week feels like. Someone who spent an hour of an evening in 2017 writing four hundred careful words for a stranger they would never meet, because somebody had once done that for them. That is the corpus. That is the whole product.
Those people were paid nothing. They were not consulted. There was no revenue share, no vote, and no opt out worth the name. And the moderators, the unpaid volunteers who spend hundreds of hours a year keeping the place from filling with garbage, have their own history with this management. When they organized a blackout in 2023 over API pricing that had just killed Apollo and every other third-party app, the CEO's public characterization of them, to NBC News, was "landed gentry": people who got there first and now pass their little fiefdoms down to their descendants, which he noted was not democratic. He said this about volunteers, in the middle of them working for him for free.
So the ledger reads: the users wrote it, the moderators maintained it, and the company sold it. That is not a partnership. That is a landlord who never laid a brick.
It Was Good Once, and That Is the Whole Tragedy
I want to be careful here, because "the internet was better before" is the laziest sentence in circulation and nine times out of ten it means "I was younger before." So let me be specific about what was structurally different, rather than nostalgic about it.
Reddit's good years ran on an unusual design. In 2008 it let anyone create a subreddit, and what followed was thousands of small rooms, each with its own rules, its own moderators, its own tolerances, its own experts and its own in-jokes. A history subreddit that deleted nine tenths of the replies to a good question because they were not sourced well enough, and was right to. A woodworking room where someone would look at your photo and tell you exactly why the joint failed. Rooms small enough that roughly the same forty people showed up every week, which meant reputation existed, which meant it cost you something to be a jerk or to be confidently wrong. That is not nostalgia. That is a functioning social mechanism, and it produced the archive we are talking about.
Then it scaled, and every part of that mechanism broke in sequence. The Digg exodus in 2010 poured a general audience into a specialist system. The default subreddit list turned a handful of rooms into a national broadcast channel, and when that was renamed to r/popular in 2017 it was the same machine with better manners. Once a room has two million subscribers, reputation stops working, because nobody is there twice. The moderators, still unpaid, are now policing a stadium with tools designed for a living room, so they reach for automation and blunt rules, because what else is there.
And twice, in 2015 and again in 2023, the company reminded everyone who actually held the deed. In July 2015 Reddit abruptly fired the employee who ran the AMA program without telling the moderators whose work depended on her, and large parts of the site went private in protest. In 2023 it repriced the API, killed Apollo and every other third-party app the power users and moderators actually worked in, and when those volunteers organized, the CEO called them landed gentry. Each time, a layer of the people who cared most about the place left and did not come back. You cannot lose your most obsessive contributors twice in a decade and remain the same site.
Now Try to Contribute Anything
Here is the experiment. Make a new account, find a large subreddit about something you genuinely know, and post something useful.
It will vanish. Not be argued with, not be downvoted, just vanish: removed because your account is too new, or your karma is too low, or an automod rule matched a word in your third paragraph, or the topic is covered by rule seven, which links to a wiki page written in 2019. Often with no notification at all. You will sit there refreshing a post that only you can see. Everyone reading this who has tried to join a subreddit in the last five years knows exactly the feeling, and most people, reasonably, conclude that it is not worth the trouble and never try again.
Reddit knows. On the fifth of August this year the company announced it is building more advanced abuse-prevention systems specifically so that communities can move away from using things like account age and karma to decide who is allowed to post. Read that announcement as the confession it is: the company is publicly acknowledging that the price of keeping the spam out has been a platform that new human beings cannot get into.
And because the gate exists, a market exists to climb over it. There is an open, barely disguised trade in aged accounts with karma already on them, sold to marketers for exactly one reason: an aged account can post in the strict subreddits that a real curious newcomer cannot. So the gate reliably excludes the amateur with something to say and reliably admits anyone with a budget. That is the precise inversion of what a public square is for, and it is not a bug in the design, it is the design working as built.
And Look at What Does Get Through
Now look at what survives that filter, which is the part that made me want to write this in the first place, because I find it genuinely dispiriting.
The upvote is a popularity mechanism. It was always a popularity mechanism. In a room of forty specialists, popularity is a decent proxy for competence, because the crowd is the specialists. In a room of two million, popularity is the median opinion of whoever happens to be awake, and the mechanism stops measuring quality and starts measuring agreement. Say a thing the room already believes, phrased with confidence and a bit of snark, and you are at the top in ten minutes. Say a thing that is true and unwelcome and you are grey, collapsed, gone. Over a few years that does not just sort comments, it sorts people: the person with the inconvenient expertise learns that the room does not want it, and stops typing, and takes their twenty years of knowing something with them.
What is left is a texture everybody recognizes and nobody will defend. The same political argument, at the same temperature, in subreddits that are nominally about cameras or aquariums or a football club. The same reposted image with the same top comment as the last four times it was reposted. The pile-on, which is the site's one genuine art form. Whole rooms where the actual content is people performing the room's opinions back at it for points, which is a thing humans will do forever and for free, and which produces nothing anyone will ever need to read twice.
Underneath that sits a layer of manipulation nobody can measure. This is not paranoia and it does not require any speculation from me, because the confirmed cases are enough. In 2018 Reddit banned nearly a thousand accounts linked to Russia's Internet Research Agency. In 2025, a team of researchers at the University of Zurich ran an unauthorized experiment on r/changemyview, deploying AI bots that posted well over a thousand comments while posing as, among other things, a trauma counselor and an assault victim, tailoring their arguments by reading users' post histories. Nobody in the community spotted them. The researchers reported that their bots were substantially more persuasive than the humans. Reddit's chief legal officer called it improper and highly unethical and sent legal demands, which was the correct response, and the incident is still the most useful thing anyone has published about the platform, because of what it demonstrated: an entire subreddit of people who pride themselves on argument could not tell. That was one academic team who chose to disclose afterward. There is no reason at all to assume they were the only ones operating, and every commercial reason to assume they were not.
Meanwhile the pitch to investors is that Reddit is the most human place on the internet, a moat against the coming flood of AI slop. The humans are behind a karma gate. The slop is already inside.
And the Price Was Five Percent
Now here is where I actually lost my patience with this story, and it is the number.
Reddit reported its second quarter on the thirtieth of July this year. Revenue of eight hundred and five million dollars, up sixty-one percent. Advertising, seven hundred and sixty-two million of that. Net income two hundred and fifty-three million dollars. A very good quarter by any standard.
And "other revenue", the line where the data licensing business lives, all of it, Google and OpenAI and whoever else: forty-three million dollars. About five percent of the company.
Reddit did not sell the last great public archive of human beings talking to each other for a fortune. It sold it for a rounding error next to the ad business. The entire licensing operation, the thing that justified locking Mojeek and DuckDuckGo and Bing out of twenty years of public conversation, would not cover two months of Reddit's advertising revenue.
If you are going to fence off the commons, hold out for real money. This was cheap. It was cheap in the way that tells you nobody in the building thought of the archive as something held in trust, because you do not accept five percent for something you believe you are holding in trust. You accept five percent for something you think you own outright and got for free.
And notice what it is that was actually bought, because this is where the two halves of this piece meet. The value in that corpus is not this week's threads. It is the small-rooms era: the years when a few thousand people who knew things wrote them down carefully for strangers, before the scale, before the gates, before the karma economy and the bots. Google and OpenAI did not license a living community. They licensed a fossil deposit. Reddit is selling access to the sediment of its own better past, while running a site that is no longer capable of producing more of it, and it has priced that deposit at roughly two months of ad sales.
The Complaining
Which brings us to this summer, and the part that prompted this piece.
The Wall Street Journal reported in July that Reddit is rethinking the Google arrangement as it comes up for renewal. On the earnings call, Steve Huffman was asked about it and said the range of outcomes is wide and that they have to make sure they are maximizing value to Reddit. Fair enough, that is his job. But the surrounding argument is what I want you to look at. Search referrals, he said, had been choppy. On AI Overviews: "10 blue links has driven tremendous value and growth to the broader ecosystem," while "AI Overviews has yet to make a similar level of positive impact." And, my favorite sentence of the year: "People don't want a summary of Reddit; they want Reddit."
Read that grievance carefully and then hold it up against 2024. The complaint is: a dominant intermediary is taking our content, presenting a processed version of it to users, and not sending back the traffic that made the arrangement worth agreeing to.
That is not an argument against Google. That is a word for word restatement of what every other search engine, and every person who ever typed an answer into a Reddit thread, has been entitled to say about Reddit since June 2024. Reddit built the enclosure. Reddit is now upset that the only tenant it let in is a bad tenant. There is no version of this where the company gets to discover the ethics of open access at exactly the moment its own deal turns unprofitable.
And that last quote is the tell. People do want Reddit rather than a summary of it. They cannot have it. On every search engine except one, they have not been able to have it for two years, because Reddit took it away from them. The company is now using the deprivation it created as an argument for a higher price.
Meanwhile, at the start of this month, Reddit announced it is moving moderation toward automated abuse-prevention systems, easing off signals like account age and karma, bundled together with better protection against scraping. The unpaid humans who made the place worth scraping get automated. The walls get higher. The archive they wrote gets sold by the pound.
I Helped, Which Is Why I Am Annoyed
I do not get to write this from a clean position, and I would rather say so than have you notice.
For the last couple of years, the standard advice from my entire industry, mine included, has been: go be genuinely useful on Reddit, because that is where Google now pools its trust. I have written a version of that argument myself, and tactically it is still correct, which is precisely what bothers me. We spent years routing the open web's attention into a walled garden and telling each other it was clever. Then the garden counted the attention it had accumulated and started charging admission at the gate. This is the same reflex that had us optimizing journalism into sludge and the same economy that keeps building links to publications nobody replants. We are extremely good at moving value into places that later turn out to have owners.
What Actually Died Here
Reddit is fine. Two hundred and fifty-three million dollars of quarterly profit is fine. Nothing in this piece is a prediction of its collapse and I would not bet on one.
What died is the possibility of a search engine that is not Google. Think about what a new entrant now has to do. Building a crawler and an index was always brutally hard, but it was at least a technical problem, and the corpus was there for anyone willing to go and get it. Today, the single most-cited body of user-generated text on the internet is available to Google, and to anyone else who can write an eight-figure check, and to nobody else at all. Organic results are pay to play at the level of the corpus itself. Mojeek cannot outspend Google. Neither can the next good idea.
So if you have spent the last two years angry about Google's grip on search, and I have spent plenty of words there myself, notice where this particular tightening came from. Google did not break into Reddit. Google was offered exclusive access to the commons and, being a company, bought it. The offer came from the other direction.
And the most remarkable thing, the thing I genuinely cannot explain, is that none of it changed the behavior of the people who make the asset. Tonight, thousands of them will sit down and write careful, funny, expert, unpaid answers for strangers, on a platform that has already sold the last twenty years of exactly that work, for five percent, and locked the door behind the buyer.
Fewer of them than last year, though. That is the part the licensing revenue will not show for a while. A company can enclose a commons, and it can let one rot, and Reddit has managed both at once: it made the place hostile to the people who wrote the good stuff, sold the good stuff they had already written, and then took it off the open web so that nobody can even go and read what was lost. It was a genuinely great thing once, built almost entirely by people who were not being paid, and it has been converted into a training set with a comment section attached.
Google bought what was offered. Reddit is the one who decided the commons had an owner, and then let it rot.