A Reddit user spent four hours writing a biopsychology paper, ran it through ZeroGPT, and got an unpleasant surprise: the detector labeled it 100% AI-generated.
Curious about what else might fail the test, they submitted the U.S. Constitution. ZeroGPT reportedly scored it 98.72% AI-generated.
Either the Constitutional Convention had excellent Wi-Fi, or that number needs a closer look. Other historical documents have produced similar results.
In a separate test, sections of the Constitution triggered AI warnings in more than one detector. Another reported test gave the Declaration of Independence a 98.51% AI-generated score.
Those are two different documents and two different demonstrations, but both make the same question hard to ignore: What does an AI detector’s percentage actually tell us?
I enjoy a tool failure that seems to accuse the eighteenth century of opening an OpenAI account. I’m less amused when a similar score appears beside a student’s essay or an employee’s work.
A wrong result on the Constitution does not prove every detector fails on every task. It does give us an excellent reason to stop treating a confident score as proof of authorship.
In this article
Putting the Founding Fathers Through the AI Police Lineup
Let’s keep the tests straight, because the details matter. In the Reddit post, the writer said ZeroGPT labeled their biopsychology paper 100% AI-generated and the Constitution 98.72% AI-generated. A separate examination of part of the Constitution found that GPTZero called it likely entirely AI-written.
ZeroGPT also labeled a passage AI-generated. For the Declaration of Independence, a reported test produced the distinct figure of 98.51%.
One Reddit commenter tried the test in reverse. They said they asked ChatGPT to generate text resembling the Constitution, submitted that imitation to ZeroGPT, and received a human-written result.
If those reported outputs are taken at face value, the detector made both kinds of error.
A false positive labels human writing as AI-generated. A false negative lets AI-generated writing pass as human.
Here, the real Constitution was flagged, while a commenter’s ChatGPT imitation reportedly got through. We have reached the unusual point where a machine suspects James Madison but is prepared to vouch for ChatGPT.
Still, a few demonstrations cannot tell us how often a particular detector gets things wrong. Input length, the passage selected, product updates, and settings may change the result.
I also did not run those historical tests myself. What these reports show is a failure mode, not a measured error rate for every detector on the market.
Why the Constitution Looks Suspiciously Like a Large Language Model
AI detectors do not have access to a document’s writing process. They analyze the text in front of them and estimate how closely its patterns match writing associated with AI systems.
Different products use different methods, but two terms help explain the Constitution problem: perplexity and burstiness.
Perplexity is roughly a measure of how surprising a sequence of words looks to a language model. “I’d like a cup of coffee” is easy to predict. “I’d like a cup of spiders” takes a sharper turn.
A system that uses perplexity may treat highly predictable wording as one signal associated with machine-generated text.
Now consider the Constitution. Its language has been copied, quoted, studied, and published countless times. GPTZero’s creator has explained that it appears repeatedly in language-model training data.
A model familiar with those words may find them unusually predictable. Formal legal writing can also follow consistent patterns without any help at all. Burstiness describes variation across a passage, including changes in sentence length and structure.
Some detection methods associate more variation with human writing and more uniformity with AI writing. Neither pattern belongs exclusively to one kind of author. People write orderly sentences. AI systems can produce varied ones.
No detector is discovering a hidden file named constitution_final_FINAL_chatgpt.docx. It is making an estimate based on textual features.
That distinction is the heart of the problem. Text that resembles model output has not necessarily been produced by a model.
Similarity tells us something about the words. Authorship asks what happened while those words were written.
A Precise Percentage Can Still Be a Weak Claim
98.72% looks impressively exact. Two decimal places suggest someone has calibrated the device and checked the batteries. It is tempting to read the number as “There is a 98.72% chance ChatGPT wrote this.”
A detector’s display does not necessarily support that interpretation. Its result depends on the product’s model, training examples, classification rules, and the text supplied.
Length, style, editing, and paraphrasing can affect the output, too. Paraphrasing creates trouble at the other end of the test.
University of Maryland researchers stated that, after AI-generated text was paraphrased, even their best-performing detector in one test fell to roughly coin-flip accuracy.
A tool that flags an old human document and misses edited AI prose has a harder task than its tidy result screen suggests.
Unless a provider clearly explains and validates what its percentage means, readers should avoid translating it into a probability of authorship.
Research discussed alongside the Constitution tests found that paraphrasing can make AI-generated text harder for some detection approaches to identify.
Another study raised concerns about false positives for writers whose first language is not English. Those findings deserve attention, although they do not establish that every current product has the same error rate.
Historical examples make the scoring problem particularly visible. Reports have described an old McDonald’s hot coffee lawsuit being labeled AI-written, along with passages of the Bible receiving high AI scores.
Nobody needs a statistical model to establish that those texts predate ChatGPT. I don’t object to a tool producing a score. I object to treating the score as if it had witnessed the writing happen.
A result can give someone a reason to look closer. By itself, it is a poor basis for declaring that an author cheated.
It Stops Being Funny When a Student Gets the Same Result
Reddit commenters quickly moved past the Founding Fathers joke and started talking about schoolwork.
Some described accusations involving their own writing. Those comments are personal accounts, not a survey of how schools use detectors, but the concern is easy to see.
In one reported case, a student accused of using AI had to defend his work before an academic honor board. His Google Docs history helped show how he developed the paper.
Even when a student has that record, answering an accusation takes time and can be frightening. A false positive has uneven consequences. A detector can print a new number in seconds.
A teacher then has to decide what it means. A student may have to reconstruct days of reading, notes, and revisions to defend work they wrote themselves.
Concerns about false positives for some non-native English writers make that decision more serious.
If a writing style affects a detector’s output, the people who write in that style could face added suspicion for reasons unrelated to their conduct. One study I came across shows several concerns.
Researchers tested seven detectors on TOEFL essays written by non-native English speakers. On average, the tools mislabeled 61.3% of those human-written essays as AI-generated.
When a concern has real consequences, I would look for evidence tied more closely to the writing process:
- Drafts, notes, and revision history can show how an argument developed.
- Sources and citations can be checked for accuracy and existence.
- A conversation with the writer can reveal how well they can explain their choices and claims.
- Earlier work may provide context for a sudden change in style, though style alone is not proof.
- School or workplace rules can establish what kinds of AI assistance were allowed.
None of those checks is a magic authorship test either. Together, they give a person a fairer basis for a decision than a percentage on a screen.
No, Thomas Jefferson Wasn’t Prompt Engineering
Reported detector tests labeled the Constitution as AI-generated and gave the Declaration of Independence a 98.51% AI score.
Those results show that human writing can trigger AI detectors, especially when text is formal, predictable, or familiar to language models.
They do not show that every detector returns the same result, that every warning is mistaken, or how often a particular product makes false accusations today.
I would use a detector result as a reason to examine a piece of writing more closely. I would not use its percentage, on its own, to accuse someone of cheating.
James Madison is probably safe. A student whose essay gets the same score deserves a little more than “the computer says 97%.”
