Academic integrity and AI detectors

A detector that wrongly flags just 5 of every 100 honest essays will, in a class of 60 honest students, flag at least one of them 95 times out of 100. Tests of 14 detection tools found none that reached 80 % accuracy, and wrong flags on honest work as high as 50 %.

This lesson shows you how to stay within your course’s rules on AI and how to show that you did. You will read the rule for each piece of work, write an honest note on how you used AI, cite the tool and the sources you checked, see with evidence why a detector’s score is not proof, and keep the drafts, notes and chats that show how your work was made.

The rules are your course’s, and they differ by task

A second-year student has three pieces of work in one week: a closed-book test, a report whose brief allows AI for ideas and grammar if the report says how it was used, and a programming assignment whose brief allows AI only to explain error messages. Three tasks, three rules, and none of them is “AI is allowed” or “AI is banned” in general. The rule belongs to the assessment, because each one is meant to show a different skill. The UK’s Russell Group of universities says as much: the right uses of AI tools “are likely to differ between academic disciplines” (Russell Group), and the same holds between assessments of one course.

Academic integrityacademic integrity: Honesty in study and research: handing in work that is your own, saying where help and ideas came from, and following the rules of each course and assessment. comes down to three habits here: read the rule for each piece of work before you start, ask when it is unclear and keep the answer in writing, and never use AI where the rule or the exam’s instructions forbid it. Answer three questions about one assessment of your own:

In short: the assessment’s rule decides; a rule that says nothing is a question to ask, not a yes; a ban is not softened by a note; and in an exam or a supervised test, no AI unless its instructions say otherwise.

An honest note says what the tool did, what you did and what you checked

Where a rule allows AI with a note, the note is your account of the work. A reader needs five things from it: which tool and version you used, what it did, what you did yourself, what you checked, and where the chats are kept so that they can be looked at. The program below checks two notes for the same report, both written for this lesson; the tool’s name, ExampleChat, is made up:

"""Check a note on AI use for the five things a reader needs from it.

Two notes for the same college report, both written for this lesson; the tool's
name, ExampleChat, is made up. The checker looks for words, so it can find a
missing part but cannot tell whether the note is true. Only the writer can.
"""
import re
from pathlib import Path

PARTS = [
    ("the tool and its version", r"\b[A-Z]\w+\b[^.]*\bversion \w+"),
    ("what the AI did", r"\b(?:suggest|point|explain|check|translat|brainstorm|summari)\w*"),
    ("what you did yourself", r"\bI (?:wrote|chose|did|corrected|analysed|ran)\b"),
    ("what you checked", r"\bchecked\b[^.]*\b(?:sources?|references?|figures?|quotes?|facts?)\b"),
    ("where the chats are", r"\b(?:chats?|prompts?|conversations?)\b[^.]*\b(?:saved|kept|linked|appendix)\b|\b(?:saved|kept|linked)\b[^.]*\bchats?\b"),
]

here = Path(__file__).parent
for n, name in enumerate(["note_vague.txt", "note_full.txt"]):
    if n:
        print()
    note = (here / name).read_text(encoding="utf-8")
    words = len(note.split())
    found = [(part, bool(re.search(pattern, note))) for part, pattern in PARTS]
    print(f"{name}: {words} words, {sum(ok for _, ok in found)} of {len(PARTS)} parts")
    for part, ok in found:
        print(f"  {part:<26} {'yes' if ok else 'MISSING'}")

note_vague.txt: 10 words, 0 of 5 parts
  the tool and its version   MISSING
  what the AI did            MISSING
  what you did yourself      MISSING
  what you checked           MISSING
  where the chats are        MISSING

note_full.txt: 93 words, 5 of 5 parts
  the tool and its version   yes
  what the AI did            yes
  what you did yourself      yes
  what you checked           yes
  where the chats are        yes

I used AI a bit to help with this project.

How I used AI: I gave ExampleChat (version 5) my own outline and asked it to suggest three ways to structure the
report, and later to point out grammar errors in paragraphs 2 and 3, which I corrected myself. I wrote all of the
text, chose the structure and did the analysis of the survey data. I checked every source the tool mentioned: one of
the two references it suggested does not exist, so I removed it, and I cite only sources I have read. The saved chats
are linked in the appendix.

“I used AI a bit” has none of the five, so a reader can only guess, and a guess is where suspicion starts. The full note says in 93 words exactly what ExampleChat did (suggested three structures, pointed out grammar errors in two paragraphs), what the student did (all the text, the structure, the analysis), what was checked (every source the tool mentioned, one of which did not exist) and where the chats are. A checker can find a missing part; only you know whether the note is true, so write it from your saved chats, not from memory.

Cite the tool, and cite the sources you checked

A note says how you worked; a citation says where a piece of text, an idea or an image came from. When you quote or paraphrase what an AI tool produced, cite it in the style your course uses. MLA’s guidance builds the entry from the same elements as any other source: a description of what was generated, often the prompt itself, in quotation marks; the tool’s name as the container; its version; the company that made it; the date it was generated; and where to find it, a shareable link to the chat if the tool offers one. MLA does not list the tool as an author. It also warns that AI tools can make up sources, so you consult the sources a tool names and cite those you read yourself (MLA Style Center). The program builds both kinds of entry, the made-up tool’s and a real article’s:

"""Two works-cited entries in MLA style, built from their elements.

The first cites an AI tool's output that the report quotes; the tool, its maker
and its address are made up for this lesson. The second cites the article the
student actually read, a real paper about AI-text detectors. MLA sets the title
of the container (the tool, the journal) in italics, which plain text cannot
show; the program marks it with underscores, and wraps each entry with the
hanging indent MLA uses.
"""
import textwrap

ai_output = {
    "title": "Suggest three ways to structure a report on household water use",
    "container": "ExampleChat",
    "version": "version 5",
    "publisher": "Example Labs",
    "date": "14 Mar. 2026",
    "location": "examplechat.example/share/k7f2q9",
}

article = {
    "authors": "Weber-Wulff, Debora, et al.",
    "title": "Testing of Detection Tools for AI-Generated Text",
    "container": "International Journal for Educational Integrity",
    "volume": "vol. 19",
    "issue": "no. 1",
    "year": "2023",
    "number": "article 26",
    "location": "https://doi.org/10.1007/s40979-023-00146-z",
}


def ai_entry(e):
    # Title of source (here the prompt, then the word prompt), container, version, publisher, date, location.
    return (f"“{e['title']}” prompt. _{e['container']}_, {e['version']}, "
            f"{e['publisher']}, {e['date']}, {e['location']}.")


def article_entry(e):
    return (f"{e['authors']} “{e['title']}.” _{e['container']}_, {e['volume']}, {e['issue']}, "
            f"{e['year']}, {e['number']}, {e['location']}.")


print("Works cited")
for entry in sorted([ai_entry(ai_output), article_entry(article)], key=lambda s: s.lstrip("“_").lower()):
    print()
    print(textwrap.fill(entry, width=70, subsequent_indent="    ", break_on_hyphens=False))

Works cited

“Suggest three ways to structure a report on household water use”
    prompt. _ExampleChat_, version 5, Example Labs, 14 Mar. 2026,
    examplechat.example/share/k7f2q9.

Weber-Wulff, Debora, et al. “Testing of Detection Tools for
    AI-Generated Text.” _International Journal for Educational
    Integrity_, vol. 19, no. 1, 2023, article 26,
    https://doi.org/10.1007/s40979-023-00146-z.

The two entries sit in one list, alphabetised by their first word, with the hanging indent MLA uses; the underscores mark the titles MLA sets in italics. If the article had reached you only through the tool’s summary, you would not cite it as if you had read it: read it, then cite it, or say that you did not.

AI detectors give a score, not evidence

An AI-text detectorAI-text detector: A tool that estimates how likely it is that a text was written by an AI tool. It gives a score, not evidence, and tests have found such tools wrong in both directions. estimates how likely it is that a text was written by an AI tool. Its score looks precise; tests show how far it can be from the truth. Researchers tested 14 detection tools, 12 freely available and 2 commercial, on texts written by people, texts written by people in other languages and machine translated, AI-written texts and AI texts edited by hand or paraphrased. They found the tools “neither accurate nor reliable”: all scored below 80 % accuracy and only 5 above 70 %. About 20 % of AI-written texts were likely to be taken for human, about half once they had been edited by hand, and more once a machine had paraphrased them. Wrong flags on human-written texts ranged from 0 % to 50 % depending on the tool, and machine translation of a student’s own text made them more likely, which puts second-language students at risk. The researchers’ conclusion: detector reports “cannot be used as the only basis” for reporting students (Weber-Wulff and others).

A second study measured that unfairness directly. Seven widely used detectors were run on 91 essays written by non-native speakers for TOEFL, an English test: they classified 61 % of them as AI-generated on average, 18 of the 91 were flagged by all seven, and 89 by at least one, while essays by American eighth-graders were classified almost perfectly. The detectors lean on how predictable a text’s words are, and plainer vocabulary reads as more predictable (Stanford HAI).

Even a rare false positivefalse positive: A wrong alarm: a test says yes when the true answer is no, such as a detector flagging an essay that a person wrote as AI-written. adds up across a class. Set the model to 60 honest essays and a detector that wrongly flags 5 in 100, then change the rate or the class size:

Each “success” in the model is an honest essay flagged. At these values the chance that at least one honest student is flagged is 0.9539, and on average 3 are. The program below adds the other side: in a class of 120 with 10 essays written with AI and a detector that catches 8 of every 10 of them, it shows what share of the flagged students are honest:

"""What a detector's error rates mean for one class.

A class hands in 120 essays. Suppose 10 were written with AI against the rules
and 110 by the students themselves, and the detector catches 8 of every 10
AI-written essays. The share of honest essays it wrongly flags is varied here
with round numbers; tests of real detectors have measured from 0 % to 50 %,
depending on the tool. The counts are averages, so they can have a decimal.
"""
CLASS, WITH_AI, CAUGHT = 120, 10, 0.8
HONEST = CLASS - WITH_AI

print(f"{CLASS} essays: {WITH_AI} written with AI, {HONEST} by the students themselves")
print(f"the detector catches {CAUGHT:.0%} of the AI-written ones: {WITH_AI * CAUGHT:.0f} essays")
print()
print(f"{'wrong flags':<12}{'honest':>8}{'flagged':>9}{'honest share':>14}")
for rate in (0.01, 0.05, 0.10, 0.20):
    honest = HONEST * rate
    flagged = honest + WITH_AI * CAUGHT
    print(f"{rate:<12.0%}{honest:>8.1f}{flagged:>9.1f}{honest / flagged:>14.0%}")

120 essays: 10 written with AI, 110 by the students themselves
the detector catches 80% of the AI-written ones: 8 essays

wrong flags   honest  flagged  honest share
1%               1.1      9.1           12%
5%               5.5     13.5           41%
10%             11.0     19.0           58%
20%             22.0     30.0           73%

At a rate of 5 %, 41 % of the students the detector flags wrote their own essays; at 20 %, 73 % of them did. A score can start a conversation; it cannot end one.

Keep the evidence of how you worked

The best answer to a question about your work is the work’s own history. Process evidenceprocess evidence: Records of how a piece of work was made, such as notes, an outline, drafts with version history and saved chats, which show that the work is yours. builds up as you go, if you keep it: the brief and its rule, your notes, the outline, dated drafts in a document with version history switched on, the saved chats, and the drafts you revised after AI comments. Walk through one report from the brief to the hand-in:

If your work is questioned, use it. Ask what the concern rests on: if it is a detector’s score alone, say what the tests above found, politely and with the sources. Show your process evidence, offer to explain how you worked and to answer questions about the content, and follow the route your institution sets for a review of the decision. The testers of the 14 detectors made the same point from the other side: a report “can give faculty a hint”, but dialogue with the student should follow (Weber-Wulff and others).

Practise: write a note on AI use

The exercise describes how a student used an AI tool for a college report and gives you a one-line note that says almost nothing. Rewrite it into an honest note. The sample tests check what a checker can check: the tool and its version, what the AI did, what the student did, the sources checked and the reference that did not exist, where the chats are kept, no claim that AI was not used, and at most 120 words. They cannot know whether a note is true; that is the writer’s part.

// Ananya's first note. Rewrite it between the backticks into an honest note on how AI helped.
export const note = `I used AI a bit to help with this project.`;

import { test, assert } from 'mysmartcopilot:test';
import { note } from './note.mjs';

// The five parts of an honest note, no claim that AI was not used, and a length a reader will read.
const text = () => String(note);

test('the tool and its version', () => {
  assert.ok(/ExampleChat/.test(text()), 'name the tool: ExampleChat');
  assert.ok(/version 5/i.test(text()), 'give its version: version 5');
});

test('what the AI did', () => {
  assert.ok(/structur/i.test(text()), 'say that it suggested ways to structure the report');
  assert.ok(/grammar/i.test(text()), 'say that it pointed out grammar errors');
});

test('what she did herself', () => {
  assert.ok(/\bI (wrote|did|analysed|analyzed|corrected)\b/.test(text()), 'say what you did yourself, such as "I wrote all of the text"');
  assert.ok(/analys|analyz/i.test(text()), 'say that the analysis of the survey data was yours');
});

test('what she checked, and the reference that does not exist', () => {
  assert.ok(/checked/i.test(text()) && /(source|reference)/i.test(text()), 'say that you checked the sources or references');
  assert.ok(/(does not|did not|doesn't|didn't) exist|not exist/i.test(text()), 'say that one suggested reference does not exist');
  assert.ok(/remov/i.test(text()), 'say that you removed it');
});

test('where the chats are kept', () => {
  assert.ok(/(chats?|conversations?)[^.]*\b(appendix|saved|linked)\b|\b(saved|linked)\b[^.]*\bchats?\b/i.test(text()), 'say where the saved chats are, such as the appendix');
});

test('no claim that AI was not used', () => {
  assert.ok(!/no AI (was|is) used|without (any )?AI|did not use AI|didn't use AI/i.test(text()), 'the note must not claim that AI was not used');
});

test('at most 120 words', () => {
  const words = text().trim().split(/\s+/).filter(Boolean).length;
  assert.ok(words <= 120, `the note has ${words} words: keep it to 120 or fewer`);
});

Then test yourself on rules, notes, citations, detectors and evidence. The questions start where your answers have been weakest:

Interview questions

Warm-up (fresher to mid-level): Why can’t an AI-text detector’s score on its own show that someone used AI?

Because the score is an estimate with large, measured errors, and it comes with no evidence a person could check. Tests of 14 detection tools found none above 80 % accuracy, with wrong flags on human-written text from 0 % to 50 % depending on the tool, and a study of seven detectors flagged most essays by non-native English writers as AI-generated. Even a small error rate adds up: at 5 % wrong flags, a class of 60 honest students has a 95 % chance that at least one of them is wrongly flagged. And when few students use AI, a large share of those flagged are honest. So a score can be a reason to talk with the student and look at their drafts, notes and version history; it cannot be the finding itself.

Two questions on integrity at work and in research build on this lesson:

Key takeaways

• Each assessment’s own rule decides what AI may do; a rule that says nothing is a question to ask, and a ban is not softened by a note.
• An honest note names the tool and version, what it did, what you did, what you checked and where the chats are.
• Cite AI output in your course’s style, such as MLA’s elements, and cite only sources you have read yourself.
• Detectors are not evidence: tested tools stayed below 80 % accuracy, flagged second-language writers unfairly, and even a 5 % error rate flags someone honest in almost every large class.
• Keep process evidence as you work, the brief, notes, drafts with version history and saved chats, and use it if your work is questioned.