Verifiability is not the only limit

Limits to intelligence

A common view is that AI systems can only get as good as we can verify¹. Much of AI's recent progress comes from the generate, measure, improve loop. It works because, for many problems, checking a solution is far easier than producing one². But it can only make progress where a measure can tell better candidates from worse ones. Where we can't verify, the loop has nothing to be guided by, and progress stalls. This is the reason for the jaggedness we observe in AI models.

But verifiability surely can't be the whole story. Predicting a fair coin flip is perfectly verifiable: either you guessed right or you didn't. Yet nobody gets better at it, because there is nothing to learn beyond 50%. So before asking whether we can score a candidate, we have to ask whether there is anything to learn at all. The two questions are independent:

	Learnable	Not learnable
Verifiable	Playing Go/chess	Predicting coin flips
Not verifiable	Essay quality	Astrology

Since this property belongs to the task and not the method used, whether by learning, search or something else, I'll call this property achievability rather than learnability: how good can a candidate possibly get?

Inner and outer limits

For a single task, the two properties act as limits, one nested inside the other. Achievability is the outer limit, set by the task: the best that is possible at all. Verifiability is the inner limit, set by the verifier: how much of that we can actually reach by optimising against our score.

The two limits differ in one important way. The outer limit is given: no amount of cleverness makes coin flips predictable. The inner limit is built: it depends on the verifier we construct. That's why a lot of AI's recent progress has come from better verifiers, not just better generators. Self-play let Go be practised endlessly, LEAN made mathematical proofs machine-checkable, and SWE-bench turned real GitHub issues into tasks that can be checked by running tests. In each case, the task stayed the same, but the inner limit moved outward.

So building a good verifier is about pushing the inner limit as close to the outer one as possible. It's an engineering practice, just as feature engineering and prompt engineering: verifier engineering.

Footnotes

1. See for example Andrej Karpathy, Jason Wei and Alperen Keles. ↩

2. This asymmetry is the intuition behind the P vs NP question: can every problem whose solutions are quick to check also be solved quickly? Most believe not, though it remains unproven. The version that matters here is more practical. P vs NP is about exact yes-or-no checks and how cost grows for the hardest problems, while we only need checking to be cheaper than producing on the problems we actually care about, even when the check is a noisy score. ↩