Pangram vs Turnitin: Why the Two AI Scores Don't Line Up

A Pangram result and a Turnitin result are not two readings of the same quantity, so a gap between them is expected rather than evidence that one is broken. Turnitin's August 2024 whitepaper states that the company does not use "accuracy" as a metric. Pangram's homepage, checked on 15 September 2026, carries a "99.9%+ Accuracy" badge. Underneath the headline the two diverge again: Turnitin's FAQ describes sentences scored inside overlapping segments and a percentage taken over qualifying prose, while Pangram's model card describes token-level prediction with three labels and document fractions weighted by characters. The number that carries consequences is the one produced by the tool your institution uses to make its decision.

Turnitin's whitepaper, Turnitin's AI Writing Detection Model Architecture and Testing Protocol, dated August 2024, says this:

"Turnitin does not use 'accuracy' as a metric as it is too easily manipulated and too dependent on the specific dataset upon which it is computed."

The next sentence gives the reason, and you can check the arithmetic yourself:

"For example, consider a dataset with 100 pieces of writing, 99 of which are human written. A simple, naive algorithm that identifies all pieces of writing as 'human-written' would achieve 99% accuracy on this dataset, despite having no value as an AI writing detection system."

That example is the useful thing to take away. A detector that never flags anything gets 99 of those 100 right, so the figure moves with the mix of the test set rather than with the quality of the detector. The whitepaper says Turnitin "uses two main metrics to test AIW-2: recall and FPR", and defines both in the same section with worked examples.

Pangram publishes an accuracy figure prominently. The homepage badge reads "99.9%+ Accuracy" (15 September 2026). The higher-education page carries both at once: "99.98% accuracy" in the opening sentence and a "99.9%+ Accuracy" badge 52 characters later, in the same banner, with only "Detects AI Assistance Free Credits" in between. Pangram's model card for Pangram 4, dated 29 July 2026, does not headline accuracy at all: it reports false positive and false negative rates broken out by corpus, language and domain.

So the two headline figures are not a high number and a low number for the same property. One vendor has published a written objection to the metric the other vendor leads with. We are not saying either choice is wrong, and neither is a secret: both statements are on public pages today.

Both systems return a percentage. The percentages are built from different raw material.

The denominator row is the one that bites. Turnitin's FAQ says the displayed percentage covers "the amount of qualifying text within the submission", and that "If text within the submission is not considered long-form prose text, it will not be included." Pangram's fractions cover the analyzed text and add to 1.0. Feed both systems a paper with a long reference list, a table of contents and a couple of tables, and they are not working on the same body of words before either model runs. How to read a Turnitin AI writing report works through what falls in and out of the Turnitin denominator.

These are the two figures people put side by side, and dividing one into the other produces a ratio that means nothing. Both are called a false positive rate. Neither is measured the way the other is.

Turnitin's FAQ states a target:

"We strive to maximize the effectiveness of our detector while keeping our false positive rate - incorrectly identifying fully human-written text as AI-generated - under 1% for documents with over 20% of AI writing."

That trailing condition is load-bearing, and the whitepaper explains where it comes from. A document is labelled AI-generated when more than 20% of its sentence-level scores clear a sentence threshold, and the whitepaper says that "the 20% document proportion cutoff as well as the predetermined model threshold were chosen to keep document-level FPR below 0.01 (1%)." The cutoff and the rate are parts of one mechanism, not an independent claim about it. What a 1% false positive rate means when a university submits 75,000 papers does the multiplication.

Pangram's homepage FAQ states a rate:

"Pangram's false positive rate (the rate at which human documents are incorrectly flagged as AI) is currently 1 in 10,000. This number is calculated on an aggregate of public datasets containing tens of millions of written documents."

The model card gives a narrower figure with a named corpus: "At the production operating point, Pangram 4 achieves a 0.0041% false positive rate (roughly 1 in 24,000) on 1,000,000 human-written English FineWeb evaluation examples." A second table on the card breaks the rate out by domain, and its academic-writing row reads 0.019% over 62,971 items, higher than the overall English figure above it.

Different labelling rules, different operating points, and on the Turnitin side a rate that only applies above a 20% cutoff. So we are not going to hand you an arithmetic comparison between the two numbers. What you can do instead is ask what each was measured on, and both vendors answer that in writing. Turnitin's FAQ says it validates its rate against "over 700,000 additional academic papers that were written before the release of ChatGPT". Pangram's homepage attributes its rate to "an aggregate of public datasets" and its model card attributes the narrower one to FineWeb examples, with academic writing appearing separately as one domain row among eleven. Those are answers to different questions, which is the whole problem.

Two reports can show "99%" and mean different things, and both vendors say so in their own documentation.

Pangram's model card treats the document numbers as shares: the three fractions are character-weighted portions of the text that sum to 1.0. It is also explicit that its other numbers are not what they look like. ai_assistance_score is "a continuous AI-involvement score, not the probability of the displayed discrete label", and the segment confidence value "measures the peakedness of the unconstrained CRF posterior, it is not a calibrated probability estimate." Read plainly, a segment marked High is one where the model's internal scores bunched tightly around a single label, and the vendor is telling you not to convert that into odds.

Pangram's guidance page for teachers reads a percentage the other way round:

"If a segment of text gets a 99% AI score, that doesn't mean we necessarily think the entire text was AI-generated. Rather, we are 99% confident that AI was used to generate some portion of the text."

Turnitin's FAQ warns against the share reading too: "Unlike our Similarity Report, the AI writing percentage does not necessarily correlate to the amount of text in the submission."

Before you compare two percentages, work out what each one is a percentage of. If one is a share of characters and the other is a confidence level, they were never going to agree, and neither was lying to you. Why the same text scores differently on every detector collects the other reasons.

A false positive rate only means something together with the set of documents it was computed over. Both vendors publish several numbers under one label, and one pair is worth working through in full, because the obvious explanation for the gap turns out to be the wrong one.

Pangram first, all checked on 15 September 2026. Read the right-hand column as carefully as the left: part of the spread is which page you landed on, and part of it is what each figure was measured against.

Turnitin produces the mildest version of this inside one file: the August 2024 whitepaper gives the AIW-2 document-level false positive rate as 0.5% in its prose and as 0.51% in Table 1, on the same set of 719,877 pre-2019 student papers.

The pair that matters more is the two numbers Turnitin publishes that are both called a sentence-level false positive rate. In June 2023 its Chief Product Officer wrote:

"Our sentence-level false positive rate is around 4%. This means that there is a 4% likelihood that a specific sentence highlighted as AI-written might be human-written."

The August 2024 whitepaper gives 0.33%, from a stress test on papers submitted before 2019 of which it says "All papers in this dataset are human written."

The tempting reading is that a model changed in between. It did, and it does not explain the gap. The whitepaper says AIW-1 launched in April 2023 and that "In December 2023, Turnitin launched AIW-2, an updated and improved model to replace the AIW-1 model", so the model running in June 2023 was AIW-1. Then look up AIW-1 in the whitepaper's own Table 2: on that same 719,877-paper set its sentence-level false positive rate is 0.42%, not 4%. Same model, same label, two numbers an order of magnitude apart. The version story collapses.

What differs is the set each percentage is taken over. The 4% is conditional on a sentence that has already been highlighted: of the sentences the detector flagged, that share may turn out to be human. The 0.42% and the 0.33% run over writing that is entirely human to begin with: of all those sentences, that share got flagged. Those answer two different questions, and there is no reason for them to land near each other. You can read the 4% definition in Turnitin's June 2023 explanation.

None of this is a vendor caught out, and none of it is a number going stale. It is what happens when one phrase gets attached to several different measurements. So when a rate is quoted at you in a meeting, the question that gets you somewhere is what set it was taken over. All submitted documents? Only the sentences already highlighted? Only the qualifying prose? A percentage with no set attached cannot be checked, and it cannot be argued with either.

One peer-reviewed, open-access study has tested both tools head to head. Who wrote this? Evaluating the reliability of AI detection tools in higher education, published in the International Journal for Educational Integrity on 29 June 2026, built a synthetic set of 160 English academic papers, forty in each of four categories, each at least 4,000 words.

On the human-written set, the result covers all four tools it tested: "all four tools classified 100% of human texts correctly as 'True Negatives' (receiving a score between 0 and 20%). This indicates that none of the detectors are prone to incorrectly flagging human writing as AI-generated for this collection of papers." On the fully AI-generated set the two tools parted company. The paper reports that "Turnitin classified 100% of the Fully AI generated papers as False Negatives (scores between 0 and 20%)", and that "Pangram was the only tool that performed well, with a strict accuracy of 65% and an inclusive accuracy of 97.5%, leaving only one case misclassified."

Before anyone quotes that at you in either direction, the authors set their own limits: "the scope of our research was limited to 160 papers, four AI detection tools, and one GenAI model (GPT-4o Deep Research) for the fully AI-generated category", and the results are "valid only for a specific snapshot in time". Their recommendation to institutions is the sentence worth carrying into a meeting. Detection tools "should not be used as sole evidence in high-stakes decision-making but should be implemented in a broader evaluation strategy."

One wording note, because the edited version travels further than the original. The paper's concluding sentence reads: "From the four AI detection tools studied here, at this moment only one produced satisfactory results." Pangram's own summary of the study renders it as "only [Pangram] produced satisfactory results", with the substitution in brackets.

Two other pieces of work get cited in this comparison, and both need a caveat attached. A University of Chicago Becker Friedman Institute research brief dated 6 October 2025 says that "Among commercial options, Pangram achieves essentially zero false positive rates and false negative rates on medium-length to long passages". It did not test Turnitin: the four tools were Pangram, Originality.ai, GPTZero and a RoBERTa baseline. Its corpus was 1,992 passages across six everyday genres including news, blogs, consumer reviews and résumés, not student coursework. There is also a connection worth putting on the table, with both halves of it: one of the two authors, Alex Imas, appears in a signed testimonial on Pangram's homepage, and the working paper carries a line on its opening page saying "All authors declare that they have no financial or personal conflicts of interest related to this study." The paper is a free download from the institute, two clicks from the brief. Weigh that however you think it deserves; we would rather you saw both facts than neither.

The second is a TechCrunch reporter's hands-on test in July 2026, which produced two numbers from one article: "Pangram gave it a 13% AI assisted score", and "when I gave Pangram that same article in its entirety, as I had written it, it got a 100% human score." One journalist's session is not a benchmark. It is a first-hand record of how a text is submitted changing the number that comes back, which is the position you are in when two institutions run two different files.

Then there is the case where the two cannot be compared at all, because only one of them returns a number. The same peer-reviewed paper notes under its main figure that "Turnitin regards papers with an AI score between 0% and 20% as uncertain and marks them with an asterisk instead of a numerical score", and that as a result "19 Fully AI-generated papers, 7 Hybrid, 6 Humanised, and 3 Fully Human papers were coded as missing in the figure." That is 35 of 160 papers with no Turnitin figure to set against anything. If your Turnitin result is an asterisk, you are not defending a low score, you have no score, and what the asterisk (*%) means on a Turnitin AI score explains the display rule behind it.

Start by getting hold of the reports, because on both sides you are dependent on whoever ran them. Turnitin's FAQ says "The AI writing detection indicator and report are not visible to students. However, with the PDF download feature, instructors can download and share the AI report with students." Pangram's help centre says a result can be shared as a link, and that recipients "will see the scan overview and segment details without having to create a Pangram account." Ask for both, and on the Pangram side ask specifically for the share link rather than a screenshot, because the segment view shows which passages produced the figure and a screenshot of a percentage shows you nothing you can work with.

Do not plan on correcting the record with Pangram directly. Its feedback control is attached to the account that ran the scan: the help page says to "Open the result you want to give feedback on, either after scanning it or from the History / All Checks page in your account", with a thumbs-up and thumbs-down under the result. A scan your professor ran is not in your account.

Nor did we find a published appeals route for the person on the receiving end of a result. We read the body text of all 256 URLs listed in Pangram's sitemap on 15 September 2026, 2,117,382 characters in total, and searched every page. `appeal` appears on one, in a blog post about advertising, in the sense of something being appealing. `dispute` appears on two: the arbitration clause in the terms of service, and a partnership post about authorship disputes. `grievance`, `contest`, `redress`, `ombuds` and `request a review` appear on none of the 256. The same search found `false positive` on 162 of those pages and `student` on 182, which is how you know the counter was matching rather than quietly returning zero for every query.

One limit on that count, since it is the sort of claim you should be able to break if it is wrong: a sitemap is not the whole site. The page this article quotes most, Pangram's Pangram 4 model card, returns 200 and is not listed in that sitemap. So read the result as 256 listed pages with no such route on them, not as a statement about every page Pangram hosts. And read it in both directions: it does not mean Pangram refuses corrections, and it certainly does not mean your institution has no process. It means the vendor is not the address for one.

Two organisations that deploy Pangram publish a route of their own, and those are the ones worth knowing about. Substack tells writers to "select Report detection error" on an AI detection report, and documents a "Disable AI detection" control on drafts. Wiki Education, which runs its participants' Wikipedia edits through Pangram, tells instructors what has been behind most of the errors it has confirmed: "If a student saves an empty outline (for example, with just short bullet points or empty section headers), or includes a significant amount of non-prose text (like sentence fragments or bibliographic entries) this might trigger the AI detector. Non-prose text is a common factor in most of the confirmed false positives we've seen." It also writes that it does "not use Pangram as definitive proof that the text was AI generated", but as "a way to flag which text needs additional human scrutiny for verifiability."

That non-prose point is checkable against your own file, and Pangram's model card backs it from the vendor side. The model is intended for writing "of at least 50 words, written primarily in complete sentences", and the card lists what sits outside that scope, including "tables of contents, reference sections, templated or automated writing, instructions and technical manuals, and text dominated by mathematical notation". It adds that "human-written headers, footers, instructions, and other extraneous formatting should be removed before checking a document", and that "Raw text and .docx files are recommended over PDFs when available because PDF parsing can introduce unintended artifacts." So two concrete questions to put to whoever ran the scan: was it a PDF, and did the reference list and front matter go through with it.

A few lines are worth quoting in a meeting precisely because they come from the vendors rather than from us. Pangram's guidance to teachers says "we believe that AI detection is a great way to flag assignments, but detection warrants futher investigation before any punitive measures" (the spelling is theirs), and adds that "A nonzero false positive rate means that any positive detection could be real, or it could be the statistically anomalous one-in-ten-thousand situation where Pangram gets it wrong." Turnitin's FAQ says "Turnitin does not make a determination of misconduct; rather, it provides data for the educators to make an informed decision based on their academic and institutional policies."

Pangram's teacher page also points instructors at writing-process evidence, naming Google Docs: "select File -> Version history -> See version history to see a full history of their writing process." If you have that record, it moves the conversation off the question of whose percentage is right, which is the one question the published numbers cannot settle. How to keep version history in Word, Google Docs and Overleaf covers where each tool stores it, and falsely flagged for AI: how to appeal, and what evidence your university will accept covers the rest of the process.

• The two vendors do not report the same metric. Turnitin's August 2024 whitepaper says it does not use accuracy as a metric; Pangram leads with an accuracy figure on its homepage and its higher-education page.
• The percentages count different things. Turnitin's covers qualifying prose only; Pangram's fractions are character-weighted over the analyzed text and sum to 1.0. Turnitin's under-1% target is also conditioned on documents with over 20% AI writing, so there is no sum you can do that turns it into Pangram's 1 in 10,000.
• Ask what set a percentage was taken over, not which version produced it. Turnitin publishes two sentence-level rates, around 4% (June 2023) and 0.33% (August 2024 whitepaper). A model change did happen in between, in December 2023, and it does not account for the gap: the whitepaper puts the older model at 0.42% on its test corpus, not at 4%. What separates them is the set. The 4% counts only sentences already highlighted; the 0.42% and the 0.33% run over writing that is entirely human.
• Work out whether your number is a share or a confidence before comparing it to anything.
• An asterisk is not a low score. In the peer-reviewed study, 35 of 160 papers had no Turnitin figure at all.
• Route your challenge through whoever ran the scan. Pangram's feedback control lives in the scanning account, and none of the 256 pages listed in its sitemap carries an appeals route for the person who was flagged. Ask for the report itself, and for the Pangram share link with its segment view.

If you are holding a Turnitin or iThenticate report, you can import the report and work on the passages it flagged. Eligible passages can be re-run at no charge.