Substack Asked for Volume, Then Shipped a Scanner
The company built a feed that pays for frequency. The bill for inspecting what frequency produced goes to the reader and the writer.
For three years Substack told writers that the way to grow was to post notes, and to post them often. On July 21 it gave readers a button that scans those posts for machine authorship. The scan runs when a reader asks for it, and the result attaches to the writer, which leaves exactly one party in the arrangement carrying nothing.
The announcement is called Against Claudefishing, published by CEO Chris Best. It opens with a quote from Freddie deBoer about wanting other humans behind the art he consumes, and about being fooled in that process amounting to a con. Two days before that post went up, deBoer published a teardown of Pangram, the detector Substack had just licensed. The announcement quotes him on the principle and does not link the teardown.
What shipped
The conditions come from Substack’s own announcement. Text longer than 100 words, published from the launch day on, so everything written before it stays unexamined. Posts, notes, comments and replies.
Best describes the reader’s side in one clause: the analysis appears only for those who request it. A reader opens a menu and asks, and Pangram returns a percentage.
The writer’s side has four moves. Write a statement in settings describing how you make your work. Run your drafts through the detector before publishing. Disable detection on a post, after which readers see a notice that detection is unavailable, which is its own kind of verdict. Report a scan you believe is wrong.
Read that list again as a description of labor. Verification became something the reader performs, one text at a time. Reputation management became something the writer performs, permanently. The feed that distributes both of them was not part of the release.
I traced the same move last month through age verification and chat logs.
What was promised in 2023
Notes launched on April 5, 2023, and Substack’s Introducing Notes states the purpose plainly: a growth channel, designed to drive discovery.
The post opens by restating why the company was built. Substack set out to make a system that fosters deep connections and quality over shallow engagement and dopamine hacks, in the founders’ own words, and the sentence sits there as the reason this new feed would be different from the others. In legacy social networks, the post says, people get rewarded for content that goes viral regardless of whether anyone values it. Then the promise about Notes specifically: it will not feel like the social media we know today.
Hold those sentences. Every argument that follows depends on them, because a company that never made the promise cannot be measured against it.
What was advised in 2024 and 2025
Fifteen months later Substack published growth instructions. Writers who post three or more thoughtful notes during their launch week gain 50% more total subscribers than those who don’t, because the app and network drive half of all new free subscriptions and 30% of new paid ones.
That is a cadence, a window and a payout, published by the platform in the platform’s own voice.
In October 2025 came Demystifying the feed, still the most detailed public account of how distribution works there. Publish on a regular basis so your work stays visible. Reply, restack, endorse. Substack’s own phrasing is that signal-boosting others isn’t just good community behavior, it’s one of the most effective ways to grow. In one three-month window the app drove nearly half a million paid subscriptions and more than 32 million free ones, most of them originating in the feed.
None of this is unusual advice. LinkedIn says it, X says it, every platform with a ranked feed says it. The difference is that the others never opened by promising quality over dopamine hacks.
What grew
Pangram scanned just over a million social posts and published the results on July 9, twelve days before Substack licensed the company. On Substack, 21.9% of posts flagged as AI-generated or AI-assisted, a combined figure rather than a rate of pure generation, and the best number in the study. LinkedIn ran past 40% on longform, and Best cites that figure in his announcement, saying the company would rather not wait until the Substack app turns into it.
The more uncomfortable measurement came from inside. Karen Spinner analyzed roughly 16,800 posts across 31 categories, tracking one construction that models overproduce. Its frequency is about five times what it was before ChatGPT, and newsletters launched after late 2022 use it more than twice as often as older ones.
Nobody can look at a sentence and know who wrote it, and Spinner says so herself. She draws no line from her data to any platform’s advice, and neither can I. What her distribution shows is where the pressure landed, and it landed hardest on the smallest and newest publications. What those have in common is that growth instructions are written for them.
What the announcement does not say
Against Claudefishing runs long. It names the problem as a mismatch between what a reader expects and what a reader gets. It concedes that not everything made with AI is slop and not all slop is made with AI. It warns that platforms rewarding fakeness create a race to the bottom.
It does not contain the word Notes. It does not mention the feed, ranking, recommendations or discovery. It does not reference any advice the company published about posting cadence. The only place ranking appears at all is in a list of things Substack is considering for later.
A post that treated distribution as part of the problem would have said so somewhere. This one, as published, puts the problem entirely with the people posting.
The instrument, and who pays for its errors
On July 20, thirty-six hours before the feature went live, Pangram founder Max Spero posted a response to deBoer. Pangram 3.3 works on segments of roughly 150 to 350 words and judges each segment as a unit. Put in 200 words containing 50 written by a model, and the system sees major signs of AI writing and calls the whole segment AI. His own assessment: not ideal, with architectural changes coming.
Substack’s own materials do not carry that caveat. The number a reader sees is not a proportion of the text, it is an aggregate of verdicts on chunks. A reader looking at 54% will read it as half, and half is not what it means.
Four days before the launch the company also published an explainer. It describes what the detector looks for: assistant models are trained to produce helpful, clear writing, and those preferences are ingrained deeply enough that they surface anyway. It also describes what it compares against. The corpus of known human text is drawn exclusively from 2021 and earlier, before AI reached the open internet. Pangram states this plainly and adds that language drifts, that adjustment will be needed, and that the problem sits among its research priorities.
So the question the instrument asks is narrower than the one readers think they are asking. It is whether this reads like a person writing before 2021, or like a helpful, clear assistant. Someone producing clean organized prose in 2026 is measured against a reference class that closed before these tools entered ordinary writing practice.
A writer is not misclassified for sharing a device with ChatGPT, and visible tells such as the em dash are one signal among hundreds of thousands. The writers currently pulling dashes out of their drafts and second-guessing phrasing they have used for a decade are doing it against nothing the system counts.
deBoer had already shown what the segmentation does in practice. A 300-word section of one of his old posts came back as 100% AI with high confidence, while the 5,000-word essay containing that section came back as 100% human with high confidence. He then produced a false positive on writing he did himself, on purpose, in about fifteen minutes.
The morning the feature went live, a writer named Ida-Emilia Kaukonen posted her own test, without naming the detector she used. Her own writing scored 79% AI. She then added text that was entirely machine-written, and the score fell to 54%. Transparency is fine, she wrote, but she is uncomfortable with the idea that she would have to destroy her voice further in order to not look like a machine. She signed it as a former mistress of em dashes, a habit she gave up ahead of a verdict that was never watching for it.
Spero’s defense of induced false positives is that any classifier can be tricked, and he reaches for airport scanners and stop-sign detection. The analogy breaks at the point that matters. A scanner alerts an operator, and a person opens the bag before anything happens to the traveler. Between Pangram’s verdict and its consequence on Substack there is nobody. The number goes to the reader and the writer wears it.
His second argument is that adversarial cases should be assessed separately from population-level false positive rates. As engineering that is correct. As experience it is worthless, because no writer ever encounters a population rate. They encounter one verdict, on one piece, once.
That verdict does not land evenly. A 2023 study in Patterns ran essays by non-native English writers through seven detectors and produced an average false positive rate of 61%, while essays by native speakers came back almost clean. Pangram was not among the seven. A peer-reviewed study from Vrije Universiteit Brussel did test it in June, on forty ESL-written academic papers of four thousand words and up, and found no false positives. That is the friendliest condition a detector gets: long documents, plenty of signal. Nobody has run the same test on feed posts of a hundred and fifty words, which is where the question is now being answered in practice.
Writers on this platform reached the paper before I did, among them JHong at Natural Intelligence and Sam Illingworth. A March 2026 preprint by Nathan Garland works out the bound for any text-only, one-shot detector and finds that a useful rate of detection forces a rate of false accusation, set by how much human writing and machine output overlap across a diverse population. Garland’s point is that this constraint is independent of model quality and cannot be engineered away, and that it falls hardest on identifiable groups, which gives the 2023 measurement a theoretical floor rather than a vendor to blame. The paper is a preprint by a single author arguing about university assessment, so treat it as an argument rather than a settled result. Its central assumption transfers here with room to spare: a teacher has at least seen the student’s earlier work, and a reader pressing scan on a writer they have never read before has nothing.
Garland’s own recommendation is that a detection score should not be the sole evidence in a misconduct proceeding. On Substack the score is the only evidence there is, and there is no proceeding.
What the score does after that has been measured, though not on detectors. A pre-registered experiment by Haoran Chu and colleagues put fabricated college applications in front of 644 US participants and recorded who they took for a machine. Two findings carry over. International students were more likely to be read as AI users, especially when their writing carried features that sound machine-made. And once a reader attributed the text to AI, their assessment of the applicant fell on competence, sociability, morality and future success.
That study measured people rather than software. Substack’s tool does not deliver a judgment. It delivers a number to a person, who then makes one.
A couple of years back we built text generation for a client. Every review came back the same: slop, machine garbage, do better. At some point we sent them one of their own articles, written by their own people before any of this existed. Same review. Slop.
One client, one article, no control group. It stayed with me anyway.
The same shape, somewhere else
This is a rhyme rather than a proof, and I will mark it as one.
ICLR submissions have roughly tripled in three years, and the review pool was filled largely out of the authors themselves. Pangram scanned this cycle’s reviews and found about 21% fully machine-generated. The conference responded by running detectors and desk-rejecting offenders. I can find no announcement that the reviewer load itself was changed.
One is a commercial platform monetizing attention and the other is volunteer academic labor under career pressure, so the cases are not equivalent. What they share is the sequence. Volume grows because the structure rewards it, synthetic output appears where the volume outruns the humans, and the institution buys detection from the same vendor rather than touching the structure. In neither case has anyone published evidence that detection reduced the synthetic output at all.
A flag is not a judgment, and the people who can tell one from the other are the expense every institution in this pattern is trying to avoid. Security tooling reached that wall first.
What is fair to say
Substack is the cleanest large feed in Pangram’s data, and I would rather write here than on the alternatives. Spero answered his critics in public and in detail, which is more than most vendors manage under pressure. deBoer, whose findings sit at the center of this piece, calls Pangram a useful tool and says he is not telling anyone to stop using it.
The claim that survives all of that is narrow and holds anyway. A platform promised in 2023 that its feed would reward quality over engagement. It then spent two years publishing instructions on cadence and reciprocity, with its own numbers attached. The output arrived on schedule. When it arrived, the company shipped an instrument that inspects writers and priced the labor to readers. It never once named the machine that set the terms.
Where I stand
I owe you to share where I stand in this story.
I use AI. There are sighs in the hall, women pass out.
I never was against AI as a tool. What I am against is the delegation of thinking and judgment, the lack of output validation, hype, and CEOs’ bullshit. AI can write good text, and humans can write bad text. The tool does not affect either side, it is the point.
I use models for research purposes. Without a tool that vacuums up 500 sources and proves or disproves my idea, I would not be able to write because I cannot handle such a volume of info. Validating each number, each quote, takes so much time that the only choice I would have is between silence and opinion.
Another shocking piece of information: I’m using Grammarly. English is my third language, and I’m very strict with myself about publishing anything without editing. The alternative on offer is that people working in a second language should write less, or not at all. I’ve been using Grammarly since December 2021, almost a year before ChatGPT appeared. The product has added the gen AI functionality over the years, but I do the same as I did five years ago: fix my grammar and sentence structure.
I did not change my approach, but classification changed. The thing that was text editing in 2021 is flagged as AI-assisted in 2026, and a person who does the same five years in a row accidentally finds himself in a new category without any choice.
That human oversight is the main difference between slop and something meaningful, and I know it not from commentary battles. I see this every day in my codebase. The model is capable of producing as much sense as a human could squish out of it, and the result depends on someone who made the judgment.
My team in Ukraine writes in English with this smooth, accurate register, which the whole industry treats as suspicious right now. Not because anyone is cheating, but because a person who writes not in his native language does it more neatly. And in our culture it is normal to be direct, without hedging and word fillers, a register that now reads as machine-made.
Someone can say, aha, little bastard, you are using AI, so you are against the AI checker, tiny slop maker! Read the article one more time; I did not say anything about what side I took. I do not take part in this fight. I did not mention that the AI detector is bad; I just said that the platform did everything to bring the slop here.
I do not know which score the AI detector will show for this text, and I don’t care. I know that the bill will be bound to my name.
And if you think that it is my real voice, you are wrong. The Tech Trenches voice finished one section above.