Every investment prospectus in America carries the same sentence: past performance does not guarantee future results. It's such a familiar piece of legal boilerplate that most people skip past it without a second thought. But it's not just a liability disclaimer. It's a precise claim about the structure of a certain kind of problem, and understanding why it's true turns out to explain something important and mostly unexamined about how today's AI systems actually work, and where their limits really are.

Why the disclaimer is true

The idea behind it has a name: the random walk hypothesis, popularized by economist Burton Malkiel in his 1973 book A Random Walk Down Wall Street. The claim is that a stock's price already reflects all currently available information, so future price movements are driven by information that doesn't exist yet, an earnings surprise, a war, a regulatory shock, something nobody could have read off yesterday's chart. Because that information hasn't happened yet, no amount of clever pattern-fitting on historical prices can reliably predict it. Technical analysis, the practice of finding patterns in past price charts to forecast future ones, runs directly into this problem: whatever pattern it finds is, by definition, made of history, and the future isn't required to repeat it.

The theory isn't uncontested. Researchers Andrew Lo and Craig MacKinlay wrote an entire book, pointedly titled A Non-Random Walk Down Wall Street, documenting real, if modest, predictable patterns like momentum and mean reversion. But even those predictable patterns are notorious for decaying: the moment enough traders find and exploit a pattern, the market adapts around it and the edge disappears. Either way, the deeper point holds. Fitting a model to history works only as well as history resembles what comes next, and it fails exactly at the moment something genuinely new happens.

Keep that idea in mind. It's about to explain something about ChatGPT.

What's actually happening inside a "reasoning" model

Large language models are built in stages, and it's worth walking through them plainly, because the public conversation about AI tends to blur them together.

Stage one is pretraining: the model reads enormous amounts of text and learns to predict the next word in a sequence. This gives it broad, general pattern-completion ability, a feel for language, facts, and structure, but not focused skill at solving hard problems step by step.

Stage two, used for years now, is reinforcement learning from human feedback, where the model is nudged toward responses that human raters prefer. This is how a raw pattern-predictor becomes something that behaves like a helpful assistant.

Stage three is newer, and it's the specific technique behind the recent leap in math and coding performance, from OpenAI's o-series to DeepSeek's R1 and beyond. It's called Reinforcement Learning with Verifiable Rewards, RLVR for short. The idea is simple to state: have the model attempt a problem, then automatically check whether the answer is correct, does the code pass its test suite, does the mathematical proof actually check out, and reinforce whatever reasoning steps led to a checkable, correct answer. Do this across millions of attempts, and the model gets dramatically better at the kinds of problems where correctness can be verified.

This is genuinely powerful. It's also, when you look closely, doing something structurally identical to fitting a trading model to historical prices.

The bottleneck nobody's hiding, but few are naming

RLVR only works where a verifier exists. Math has one: a proof is right or it isn't. Code has one: it passes the tests or it fails them. But most of the problems that actually matter in life don't come with an automatic checker. Multiple papers from 2025 and 2026 name this directly, in their own words, not as a criticism from outside the field but as an acknowledged limitation researchers are actively trying to work around. One study states plainly that RLVR "fundamentally relies on verifiers that presuppose the existence of standard answers" and "cannot be directly applied to open-ended tasks." Another calls it a "bottleneck of verifiability": in domains like strategic planning, medical judgment, or genuinely novel scientific questions, where the ground truth is ambiguous or simply doesn't exist yet, "reliable reward signals are largely unavailable."

Researchers have proposed workarounds, using a second AI model as a subjective judge, or building detailed scoring rubrics by hand. These are real, active efforts, and they come with their own documented problems: judge models can be gamed, rubrics are expensive to build and don't generalize well. The field isn't ignoring this gap. It just hasn't closed it.

A benchmark that keeps proving the point

In 2019, AI researcher François Chollet built a test called ARC-AGI to probe this exact gap: visual puzzles designed to be easy for humans and resistant to memorization or brute-force pattern-matching. What happened next is instructive.

ARC-AGI's first version fell fast, OpenAI's o3 was scoring 75 to 87% on it by late 2024. So Chollet's team built a harder version, ARC-AGI-2. That one fell too, from 53% in December 2025 to over 95% within months, largely through hybrid approaches combining language models with evolutionary code search or multi-agent planning.

Notice what those puzzles actually are, though: each comes packaged with example input-output pairs, and the final answer is checkable. That makes them verifiable-domain problems in disguise, exactly what RLVR-style optimization is built to conquer. So Chollet's team built a third version, ARC-AGI-3, testing interactive, agentic reasoning in genuinely unscripted environments rather than static, checkable puzzles. As of March 2026, the leading frontier models score below 1% of human efficiency on it. Humans solve it at 100%.

This is the random walk problem playing out in real time. Every version that could be converted into a checkable answer got solved, fast. The version that couldn't be converted that way remains almost entirely out of reach.

Getting good at the hill you're already on

There's a second finding, more surprising than the first, that gets at exactly why this happens. A 2025 study, published at both ICML and NeurIPS, compared "base" language models, before RLVR training, against their RLVR-trained counterparts, using a metric called pass@k: does at least one of k attempts get the right answer?

At k equal to one, a single best guess, the RLVR-trained models win clearly. They're faster and more reliable at finding the answer they're already inclined toward. But at large k, given many attempts, the untrained base model actually finds more distinct correct solutions overall, sometimes solving problems its RLVR-trained descendant never solves no matter how many tries it's given. The paper's own language describes the model's "reasoning capability boundary" narrowing as RLVR training progresses. RLVR, in their words, doesn't create fundamentally new reasoning patterns, the correct answers it reliably produces were already present in the base model's broader, messier range of possibilities. Training doesn't discover new terrain. It reallocates confidence toward the hill the model was already standing on.

This is worth being honest about: the finding is contested. A separate paper argues the apparent narrowing is partly an artifact of only checking final answers rather than the reasoning that produced them, and finds RLVR does improve performance once you account for that. Other researchers report mixed results depending on the domain. This is a live, unresolved dispute, not settled science. But even the contested version of the finding points at something real: there is a genuine, measurable tension between getting very good at exploiting a known-good path and staying able to explore toward a different, possibly better one. That's not a new problem. It's the exploration-exploitation trade-off, one of the oldest and most fundamental problems in optimization, the same reason a hill-climbing algorithm can walk confidently to the top of a modest hill and have no way of knowing there's a taller one across the valley, because getting there requires a step that looks, in the short term, like getting worse.

The distinction that explains why this isn't a bug to be patched

There's a hundred-year-old piece of economic theory that names exactly what's going on here, and it's worth introducing properly, because it turns this from a list of AI quirks into a single, coherent argument.

In 1921, economist Frank Knight published Risk, Uncertainty, and Profit, and drew a sharp distinction that economics has never fully let go of since. Risk, in Knight's terms, is randomness that can be measured, because it comes from repeated, comparable events, a dice roll, an actuarial table, the kind of thing an insurer can price with confidence. True uncertainty is different in kind, not degree: situations so unique that the underlying odds can't be known at all, because there's no class of similar prior instances to draw a probability from. In Knight's own words, real business decisions "deal with situations which are far too unique, generally speaking, for any sort of statistical" treatment. And crucially, Knight argued that genuine profit, as distinct from a wage for managing calculable risk, arises specifically from bearing that second kind, uncertainty that cannot be insured away because it cannot be measured in advance.

RLVR thrives only where Knightian uncertainty can be collapsed into Knightian risk, and most real-world judgment resists that collapse. Gather enough repeated attempts with a checkable answer, and what looked unique becomes statistically tractable, exactly what happened to ARC-AGI-1 and ARC-AGI-2. But the domains that matter most in real life, a doctor's judgment about a specific patient, an entrepreneur's irreversible bet, a scientist chasing a genuinely open question, a judge's ruling, don't offer that luxury. There is no automatic verifier standing by to check the answer, because if there were, the problem probably wouldn't be hard, or new.

No hindsight, no reset

Here's the sharpest version of the point, and it's the one that matters most in practice: an RLVR training run gets to fail privately, in bulk, before anything real is at stake. Millions of attempts, a wrong answer discarded, weights nudged, try again. That's the exact luxury Knight says true uncertainty denies you. A doctor doesn't get to run the diagnosis a thousand times in parallel and keep the version that worked. An entrepreneur committing capital to a genuinely novel bet doesn't get a do-over once the decision is made and the world has moved. An investor can't rewind a trade once it's placed and the news has broken. The consequence is real, singular, and irreversible, the opposite condition from the one that built the model's competence in the first place.

This is worth naming plainly rather than letting the field's own cautious language get lost in translation on the way to a headline. The researchers who build RLVR systems describe their own results carefully: "current training does not elicit fundamentally new reasoning patterns," a narrowing "capability boundary," a "bottleneck of verifiability" they're still trying to solve. The public conversation, driven in no small part by the financial incentive to describe a breakthrough as bigger than the paper underneath it claims, calls the same systems "reasoning models" and lets the word do work the research hasn't earned yet.

That gap between what's demonstrated and what's implied isn't a minor communication problem. It's exactly the kind of complacency that keeps attention pointed at the wrong target. The frontier was never reasoning under risk. It's judgment under uncertainty, and for now, that's still a human monopoly.

Sources

• On the random walk hypothesis and Burton Malkiel's A Random Walk Down Wall Street: en.wikipedia.org/wiki/Stock_market_prediction, smartasset.com/investing/random-walk-theory
• On Andrew Lo and Craig MacKinlay's counter-argument, momentum and mean-reversion anomalies: chartschool.stockcharts.com
• On RLVR's verifiability bottleneck in open-ended domains: arxiv.org/pdf/2601.14700 (DARL), openreview.net/forum?id=uZxyvmN72d (VMR-RLVR), arxiv.org/pdf/2508.12790 (Rubric Anchors)
• On ARC-AGI's history and results across all three versions: arcprize.org/arc-agi/1, www.bracai.eu/post/arc-agi-2-benchmark, www.emergentmind.com/papers/2603.24621 (ARC-AGI-3 results)
• Yue, Y., et al., "Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?," ICML/NeurIPS 2025: openreview.net/forum?id=4OsgYD7em5
• On the contested rebuttal using CoT-Pass@K: arxiv.org/pdf/2506.14245
• Knight, F., Risk, Uncertainty, and Profit, 1921, and modern summaries of the risk/uncertainty distinction: oll.libertyfund.org/titles/knight-risk-uncertainty-and-profit, news.mit.edu/2010/explained-knightian-0602, fee.org/articles/frank-knight-s-risk-uncertainty-and-profit-100-years-later