The Solvability Problem

Somewhere in the code for the mahjong game on this site, there is a function with a strange kind of stubbornness. The obvious way to deal a game of mahjong solitaire is to shuffle the tiles and drop them on the board, the way you'd shuffle a deck of cards. This function refuses. It builds the board backwards instead, starting from an empty, already-won game and working backwards to the pile you are actually handed. Going to that much trouble just to guarantee one small thing turns out to be the most useful idea I know for deciding whether to trust anything a machine hands you.

Stick with the tiles for a few minutes. I promise this isn't really about tiles. It just starts there, the way the good arguments usually do.

---

01A board that lies to your face

Mahjong solitaire is the one where tiles are stacked in a heap and you clear them by matching free pairs. A tile is free when nothing sits on top of it and one side is open. Simple enough to explain in a sentence. Here is the part that is not obvious: you can lay out a board that is completely legal and completely impossible.

Legal has a precise meaning in the code. There is a function whose whole job is to check that a board is structurally sane: no tile floating in mid-air, no two tiles overlapping in the same spot, an even number of tiles so the pairs can work out. The comment above it calls it exactly what it is, a structural sanity check. A board that passes it looks, in every visible way, like a proper game.

Solvable is a completely different question, and that sanity check has nothing to say about it. Deal the tiles at random onto a legal board and you get a legal board. Sometimes it happens to be winnable. Sometimes it is a dead end that cannot be finished no matter how well you play, because two tiles that had to come off early are trapped under a tile that can only come off late. Nothing you can see tells you which kind you are holding. It looks finished either way.

That is the quiet cruelty of the naive version. Nobody hands you a sign that says this one cannot be won. You just play on, matching tiles happily, and somewhere around tile forty the board goes silent: no legal pairs left, game unfinished, and no obvious villain except yourself. You restart, swearing you'll spot the two tiles you missed this time. There were no two tiles. The game was rigged against you before you touched a single tile, and it let you find that out the hard way, alone, at tile forty.

---

02Building it backwards

So the game refuses to deal at random. Instead it does something that felt like a small magic trick the first time I understood it. It plays the game in reverse.

Start from the finished state: an empty board, every tile already cleared. Now run the game backwards. Look at the positions that would be the last to open up, and drop a matched pair onto them. Then another pair, onto positions that are legal now, and another, working all the way back until the board is full. Because every single pair was placed onto a move that was legal at the time, the sequence you just recorded, played forwards, clears the whole board. You did not deal a board and hope. You built a board out of a winning game.

The function that does this carries a one-line comment above it that is the entire philosophy in a sentence: the board is solvable by construction. It never checks whether the board can be won. It makes it impossible for the board to be anything else. There is even a second function kept around as a safety net that replays the recorded solution to confirm it is legal, but that is a seatbelt, not the engine. The engine is that an unwinnable board simply cannot be built this way in the first place.

Worth admitting how inelegant this actually is under the hood, because the honesty is half the fun. The code does not nail this in one graceful pass. It tries, and if a particular attempt paints itself into a corner with too few legal moves left, it shrugs, throws the whole attempt away, and tries again, up to two hundred times, until one run finally makes it start to finish clean. The random board shapes get the same treatment one level up: up to sixty attempts at a pleasing procedural layout before the code gives up on being clever and falls back to a plain pyramid it already knows works. There is no magic in here. Just a machine that fails cheerfully, over and over, in milliseconds, until the guarantee holds. Which, honestly, is closer to how most real guarantees get made than the tidy one-paragraph version of this story ever lets on.

It never checks whether the board can be won. It makes it impossible for the board to be anything else.

---

03Now look at what the machine hands you

I told you the tiles were not really the point. Here is the point.

Almost everything a language model gives you is the shuffled board. It is structurally perfect. The sentences parse, the code compiles, the numbers sum, the format is immaculate, the tone is calm and sure of itself. It passes every check you can run just by looking at it. And exactly like the mahjong board, structurally perfect tells you nothing about whether it can actually be won: whether the summary is faithful to the document, whether the citation is a real case, whether the forecast holds, whether the clause means what you think it means.

In an earlier piece I called these models fluent guessers, and I keep coming back to the phrase because it names the exact danger. The model is not usually wrong in a way that looks wrong. It is wrong the way a dead mahjong board is wrong: everything in view is in order, and the failure is sitting somewhere you cannot see from where you are standing. A crash you can spot. A confident, well-formatted, plausible mistake is the one that reaches the client.

---

04The two honest ways to trust an answer

There are really only two ways to trust something, as opposed to feeling like you trust it. You can check it, or you can build it so it cannot be wrong. The mahjong game quietly demonstrates both, because it leans hard on one of them and pointedly refuses the other.

Checking is what most of us do, on a good day. You take the finished thing and you test it against reality: you replay the solution, you re-run the figures, you call the client to confirm, you read the clause a second time. Checking works, and it has a price, and the price is that you have to actually do it, every time, on the finished output, and you have to know what a real check even looks like. Most looks-right output never gets one.

Building it so it cannot be wrong is the rarer and better move, and it is what the mahjong dealer does. You do not verify the board afterwards. You arrange the method so the bad outcome cannot be produced at all. At a desk this is the difference between proofreading a total by hand and using a formula that can only ever add up to the right total. It is the difference between reminding people to follow the process and designing a process that has no wrong path to take. When you can get it, it is worth far more than any amount of checking, because it moves the guarantee from your attention, which is finite and tired, into the structure, which never blinks.

Now the uncomfortable middle. Most of how people actually use AI today is neither of these. It is not checked and it is not built to be correct. It is glanced at. It reads well, so it goes out. That is the shuffled board handed straight to a player, with the modern twist that the player is a client, or a board of directors, or in a few well-publicised cases, a court.

And this gets sharper by the month, because the machine is more and more trusted to act on its own answer rather than merely suggest it. When a person was always going to read every line before it mattered, looks-right at least bought some time. When the answer flows straight into an action that nobody reviews, looks-right is the only check that ever happened.

---

05Some things you prove, some you can only probe

There is one more turn in this little file, and it is my favourite, because it keeps the whole idea honest.

Guaranteeing that a board can be won is not the same as making it a good game. A board can be winnable and trivially dull, or winnable and genuinely hard. Solvable is a floor, not a difficulty. So the game carries a second engine right next to the first, and it works in a completely different way. To judge how hard a board is, it does not reason and it does not prove anything. It plays the board out, hundreds of times, making random legal moves, and counts how often that mindless strategy stumbles into a win. A board that random play almost always beats is easy. One it almost always loses is hard. The difficulty rating is just that win rate.

Look at what that means. Two claims about the same board, arrived at in two completely different ways. Whether the board is solvable is proven, in advance, by construction, and it is exactly true: every board can be won, no exceptions. How hard the board is gets estimated, after the fact, by sampling, and it is only roughly true: play it enough times and the average tells you something real, but no single number is the final word. One is a proof. The other is a measurement with error bars.

That distinction is, I have come to think, the real skill for this era, more than any clever way of phrasing a prompt. Not simply is the answer right, but what kind of claim is this, and how would its rightness be established. Some things can be made true by construction and you can put your full weight on them. Some can only be estimated by sampling, and you should hold them the way you hold a weather forecast: useful, real, and never to be mistaken for a guarantee. The trouble starts the moment you treat a measurement like a proof, when you take a confident-sounding estimate and stack a load-bearing decision on top of it as though it were a winning line you had recorded yourself.

---

06The question that replaces "does this look right?"

So here is the whole thing, the part worth keeping after you close the tab.

When a machine hands you something finished, the instinct is to ask, does this look right? It is a bad question now, because the entire problem is that looking right and being right have quietly come apart. The mahjong board that cannot be won looks exactly like the one that can. The confident wrong answer looks exactly like the confident correct one. Your eyes were trained in a world where producing something that looked finished was hard, and usually meant a competent person had done the work behind it. That world is gone. Finished-looking is now free, and available in unlimited quantity, on demand.

Two better questions take its place. The first: how would I actually know? Can I check this, and do I know what a real check looks like? If the answer is yes, and the thing matters, then do the check. Do not just admire the output because it is well dressed. The second question is the one that changes how you work: can I build this so it does not have to be checked? Can I set it up, the way that deal can only ever produce a winnable board, so that the failure I am worried about simply cannot occur?

You do not need to write a line of code to use either one. They are for the report, the deck, the analysis, the forecast, the email you are about to trust because it reads well. That small function in the mahjong game goes to real, deliberate trouble so that it will never, not once, hand a player a game they cannot win. It would rather do more work up front than let someone discover, forty tiles deep, that they were doomed from the deal. That is not a bad standard to hold the rest of it to, now that the machine will hand you anything you ask for, instantly, and make every bit of it look finished.

Finished-looking is now free. Being right is the part that still costs something, and knowing the difference is the job.