How to Actually Tell What an AI Rewrite Changed

You ask ChatGPT to tighten a paragraph. It comes back with something that's clearly different — but is it *better*? Reading both versions top to bottom and trying to hold the first one in your head while you read the second is a bad way to answer that question. Your eyes are good at spotting differences placed side by side. They're bad at spotting differences held in memory across two separate blocks of text.

The problem with "just read both"

This gets worse the more you regenerate. Ask for three variations of a product description and you're now holding four versions in your head at once, trying to remember which one dropped the pricing line and which one added the CTA. Most people give up and just pick whichever one they read most recently — which is a recency bias, not a decision.

A text diff solves this the same way it has for programmers reviewing code changes for decades: it computes the actual, minimal set of insertions and deletions between two versions and shows you only that — additions highlighted, removals struck through, everything unchanged left alone. No memory required.

Word-level vs. line-level — pick the right lens

Prose and code need different diffing granularity, and using the wrong one makes the result harder to read, not easier. For a rewritten paragraph, you want word-level: it shows you exactly which phrases changed inside sentences that are otherwise the same. For code or a config file, you want line-level: the meaningful unit there is the whole line, and word-level diffing on code just produces visual noise from every shifted bracket and indent.

• Rewritten email, blog paragraph, product copy → word-level
• Regenerated function, config file, JSON blob → line-level
• A whole document reorganized into new paragraphs → line-level, since the "line" unit tracks the reordering better than word-level would

A "was this worth it" gut check

One useful side effect of diffing: you get a rough percentage of how much actually changed. If you asked for "a small tweak to the tone" and the diff shows 80% changed, the model did a lot more than you asked for — worth knowing before you ship it without a full re-read. If you asked for a full rewrite and it comes back 15% changed, you probably need to push the prompt harder.

Photo by Mick Haupt on Unsplash

That number isn't an AI judging quality — it's a plain count of words or lines added and removed, the same category of measurement `git diff --stat` gives a programmer. It's not smart, and that's exactly why you can trust it: it's not interpreting your text, just counting the actual edits.

Beyond AI rewrites

This isn't only for AI output. It works for any two versions of anything text-based: a contract redline, two drafts of an email a colleague sent back, the same README before and after someone else edited it. The moment you catch yourself scrolling back and forth between two tabs trying to spot what's different, that's the signal to diff it instead of eyeballing it.