Why your spell-checker misses the mistakes that matter

Your spell-checker misses the mistakes that matter because it was never looking for them. It compares each word against a dictionary and marks the ones that are not in it, which means a sentence like “I could of sent it to there team” passes without a single underline. Seven words, seven dictionary entries, two errors, no complaint.

That is not a defect. Dictionary spell-checking is one of the few genuinely solved problems in software, and it solved the problem it was given. The trouble is that the problem it was given is not the one most people have. Typing teh is a motor error and you catch it yourself half the time. Writing there when you meant their, or leaving a verb agreeing with the wrong noun, is a language error, and those are the ones that survive your own proofreading and land in front of a client.

What a spell-checker actually does #

A classic spell-checker splits your text into tokens, looks each one up in a word list, and for every token it cannot find, proposes the dictionary entries that are one or two character edits away. That is the whole mechanism. It is fast enough to run on every keystroke, it works offline, it is deterministic, and on an actual typo it is almost never wrong.

Some checkers bolt a rule layer on top: a few hundred hand-written patterns that catch “a apple”, a doubled word, or a missing space after a full stop. Rules are cheap and precise, and they do not scale. Each one has to be written by a person, for one language, for one construction, and the constructions that go wrong in real writing outnumber any rule set anyone will maintain. That is why the grammar layer in a word processor has always felt like it checks five things thoroughly and everything else not at all.

The structural limit is simpler than the engineering: a dictionary lookup takes one word as input. Whether a word is correct almost never depends on that word alone.

The mistakes it structurally cannot see #

Six classes cover most of what actually goes wrong in work writing, and a dictionary checker is blind to all six for the same reason: every word involved is a real word.

Subject-verb agreement across a clause

The verb agrees with the subject, but the subject can be six words back with a different noun sitting next to the verb. Written: “The list of items that were approved last week are attached.” Correct: “The list of items that were approved last week is attached.” The verb belongs to list, not to items. To see that, you have to hold the sentence's structure, which is exactly what a per-word lookup does not do.

Tense that drifts mid-sentence

Long sentences start in one tense and end in another, usually because you edited the front half after writing the back half. Written: “I opened the file and see that the totals in column D are wrong.” Correct: “I opened the file and saw that the totals in column D were wrong.” Both see and saw are in the dictionary, so only the relationship between them is the error, and a relationship is not a token.

Articles, when your first language has none

Russian, Polish, Czech, Ukrainian, Mandarin, Japanese and Korean carry definiteness through word order and context rather than through an article, so there is no habit to transfer into English. Written: “I attached report you asked about, and I also sent invoice.” Correct: “I attached the report you asked about, and I also sent the invoice.” Nothing here is misspelled. The mistake is a missing word, and a checker that examines words that are present cannot flag a word that is absent.

Homophones and near-homophones

Their, there and they're. Its and it's. Affect and effect. Complement and compliment. Written: “Its almost finished, and the new version will effect the reporting job.” Correct: “It's almost finished, and the new version will affect the reporting job.” This is the purest case of the whole problem: the wrong word is a perfectly good word.

Comma splices

A comma cannot join two sentences that could each stand alone. Written: “The build passed on staging, we can ship on Friday.” Correct: “The build passed on staging, so we can ship on Friday.” A semicolon or two sentences would do as well. Every character in the wrong version is legal; only the joint is wrong, and the joint is punctuation between two clauses, which is the definition of something a token-level tool cannot inspect.

False friends

Words that exist in both languages and mean different things. Written by a Spanish speaker: “I will assist to the meeting to discuss the actual version of the contract.” Correct: “I will attend the meeting to discuss the current version of the contract.” Asistir means attend and actual means current, and both English words are spelled impeccably. This class is invisible not only to the dictionary but often to the writer, because the sentence feels right in the language they are thinking in.

What a model-based checker does differently #

A model-based checker reads the whole sentence at once, and often the sentences around it, then compares what you wrote against what a fluent writer would most plausibly have written to mean the same thing. It is not consulting a rule for comma splices. It is scoring your sentence against an enormous prior over how sentences go, and the places where your version is far less probable than an obvious neighbour are the places it proposes a change.

That single difference is why it catches all six classes above without anyone writing six rules, and why it transfers across languages: the same mechanism that notices a missing English article notices a wrong German case or a Spanish subjunctive, because in each language the corrected form is the far more probable one. It is also why it can fix several unrelated things in one pass. Take a sentence FixMeBot is routinely given:

“I has been working on this report since three days and its almost finish.”

“I have been working on this report for three days and it is almost finished.”

Four separate problems: agreement (has for have), a preposition calqued from another language (since for for), a homophone (its for it's) and a participle (finish for finished). Exactly none of them is a misspelling, so a dictionary checker returns a clean sentence. FixMeBot's mistakes correction handles the four together because it is reading the sentence, not the words.

The prior is also what lets it disambiguate: “to there team” and “to their team” differ by one letter, and only one of them is a sentence English speakers produce. No lookup can rank those; a model does it trivially.

Side by side #

The last row is the honest one. It is the case where the dictionary checker is more useful, and the case a model gets wrong in a way that is hard to notice.

Where each one still fails #

Both tools fail, and they fail in opposite directions, which is the most useful thing to know about them.

A dictionary checker fails by crying wolf. Product names, colleagues' surnames, acronyms, library names and anything in a second language get underlined, and after the fortieth red squiggle under a word you spelled correctly you stop reading the squiggles. A checker you have trained yourself to ignore is worse than no checker, because you believe you are being checked.

A model fails by being confident and plausible. Three specific ways, worth knowing before you trust one:

• It regularises rare terms. A term of art it has seen a thousand times loses to a common word it has seen a billion times. Internal product names, transliterated names, ticker symbols, chemical shorthand, a variable called usr. The correction is fluent, which is exactly what makes it easy to skim past.
• It flattens a deliberate voice. Sentence fragments become sentences. A one-line answer grows a preamble. A line you wrote short because you wanted it short comes back padded and polite. If the bluntness was the point, the “corrected” version has lost the point while gaining a comma.
• It is not deterministic. Run the same paragraph twice and you can get two slightly different rewrites. For correction that rarely matters; when you start asking for rewrites rather than fixes, it starts to matter. A fluent rewrite can change what a sentence commits you to without changing how right it sounds. That is a different failure with its own checklist, in how to rewrite a sentence without changing what it says.

How to use both #

1. Leave the built-in spell-checker on. It is free, instant, offline and right about typos. There is no version of this argument where you turn it off.
2. Run a model pass on anything you cannot un-send. A client email, a pull-request description, a report, a message written in a language that is not your first. Not on a message to a friend.
3. Read the diff, not the output. Any checker worth using shows you what changed. Accepting a clean-looking paragraph without seeing the changes is how RUM becomes RAM.
4. Correct, do not rewrite, when the voice matters. Asking for corrections gets you your sentence with the errors gone. Asking for improvements gets you somebody else's sentence.
5. Check the terms you own. Before you send, scan for your product names, acronyms and anyone's surname. That is a five-second pass that catches the one failure mode you cannot delegate.

The two tools are answering different questions. A spell-checker answers “is this a word?” and answers it perfectly. A model-based checker answers “is this the sentence you meant?” and answers it well, most of the time, with a bias toward the ordinary. Knowing which question you have is most of the work.