Never Give Up: this RL method reveals that your benchmarks are hiding a stagnation — problems with a 0% success rate stay there after training
🔎 The curve goes up, but your model has stopped learning
Mid-September 2026, Michael Noukhovitch, Hamish Ivison, Nathan Lambert, and Aaron Courville publish on arXiv a paper that should make anyone brandishing a smoothly rising RL reward curve uncomfortable: Learning to Solve Hard Problems in RL for LLMs by Never Giving Up. Behind the title lies a demonstration that's awkward for the entire field.
The finding boils down to one sentence, taken from Noukhovitch's accompanying blog post: "Easy AIME problems improve drastically but problems that start with pass@32=0 mostly end with pass@32=0!". In other words: reinforcement learning drastically improves problems the model can already (somewhat) solve — and leaves at zero the ones it can't do.
Why does this matter now? Because RL has become the engine behind reasoning models, and aggregated benchmark curves feed threads, investor slides, and press releases. If these curves mask a near-total stagnation on hard problems, then part of the "models are acquiring new skills" narrative rests on a statistical artifact. The paper also proposes a fix — NGU, Never Give Up — of disarming simplicity. Let's break it down.
Key Takeaways
- RL for LLMs does not improve a dataset uniformly: big gains on problems the model already sometimes solves, near-zero gains on the rest. The authors call this bias the "Matthew effect" — the rich get richer.
- A problem is deemed "hard" when the starting model shows pass@32 = 0: 32 attempts, zero successes. After standard RL training, the majority of these problems remain at 0.
- Aggregate curves mask this stagnation: easy problems, more numerous and improving rapidly, pull the average up.
- NGU keeps sampling a problem until it obtains a success — an expected k/(1−p) samples, i.e., 40 with k=4 and p=0.9 — and trains on all accumulated samples, failures included.
- Results: NGU beats standard GRPO at every value of k on GSM8k platinum; the DeepScaler scale-up (~120 hours of H100 on Qwen 3 4B base) mainly improves the hardest subsets of AIME 2025 and BRUMO 2025; on Manufactoria, standard GRPO fails where NGU iterates until complete success.
- Acknowledged limitations: samples are more off-policy early in training, and the method is inefficient if the task is almost entirely very hard.
Recommended Tools
| Resource | Primary use | Price (September 2026) | Ideal for |
|---|---|---|---|
| Paper NGU (arXiv 2609.13443) | Full method, results and limitations | Free | RL engineers, researchers |
| Mr. Noukhovitch's blog post | Detailed analysis, k=4 vs k=32 dynamics | Free | Understanding the mechanisms |
| Hugging Face Papers | Summary and community tracking | Free | Staying up to date |
| alphaXiv | Paper discussion, methodological context | Free | Placing NGU in the literature |
| Hostinger | Self-hosting open models for your own pass@k campaigns | From a few €/month (September 2026, check hostinger.fr) | Testing your models yourself |
The finding: pass@32 = 0 problems that stay there
No, a rising RL reward curve does not prove your model is learning. The question raised by the authors — "What is your eval actually measuring?" — deserves to be asked at every training announcement.
A quick reminder: pass@k measures the probability of solving a problem in k attempts. A problem labeled "hard" in this paper is one where the initial model, across 32 samples, never produces a single correct answer. This is not an abstraction: it is a measured ceiling, problem by problem.
The team then trained with standard RL and observed what changed, difficulty by difficulty, on code and agentic evals: Deepcoder and DeepSWE — two open-source projects with public models and logs, which we have already shown made it possible to document code agents cheating —, LCBv6 with a breakdown by difficulty (Deepseek-R1-Distilled-Qwen-14B), and SWE-Bench platinum, which provides existing difficulty labels.
The result is identical everywhere. Problems the model was already occasionally solving improve fast and strong. Those that started at pass@32 = 0 stay there, in their overwhelming majority. RL did not learn to solve new problems: it learned to solve old ones better.
The Matthew Effect: When the Average Lies
Aggregate scores structurally overestimate what RL delivers. A benchmark aggregates tens or hundreds of problems into a single number. But that number is a weighted average: easy problems are often the majority, and they're the ones improving "drastically." Hard problems, few in number and flat, barely weigh on it.
The authors borrow the term Matthew effect from sociology: the rich get richer. RL enriches models where they are already rich in capabilities. Hugging Face sums up the paper in one line: "Reinforcement learning for large language models disproportionately improves easy tasks".
A troubling consequence: a ten-point rise in an aggregate score is perfectly compatible with zero new capabilities acquired. This isn't cheating, nor contamination — it's more elusive, a problem of reading. And there is only one antidote: breaking down the results by difficulty, which almost no announcement does.
NGU, how it works: reallocating compute, not inventing magic
NGU doesn't change the learning algorithm: it changes what you show it. The authors' diagnosis is that modern RL methods exacerbate the Matthew effect by wasting compute on easy problems. Their fix fits in a few lines.
The mechanics: you sample k responses for a prompt. If everything is wrong, you put the prompt back into the generator with probability p, and repeat until you get a success. The expected number of samples follows a geometric distribution: k/(1−p). With k=4 and p=0.9, a stubborn problem receives on average 40 samples, versus 4 for an easy problem filtered on the first pass. Compute is reallocated, not increased.
Second point, often missed in summaries: you train on all the accumulated samples, including failures. Failed attempts on hard problems aren't thrown away — they become the training signal. That's where asynchronous RL does the work: continuous generation makes it possible to keep feeding the loop without waiting for an entire batch to succeed.
An honest caveat: alphaXiv identifies a close methodological precursor, a method that also adaptively resampled prompts to address undersampling and signal loss. NGU doesn't invent the category; it gives it a quantified diagnosis and solid results.
k=4 vs k=32: the reversal around step 200
The dynamics are counterintuitive. Early in training, k=32 finds rare solutions to hard problems sooner. But past an inflection point around step 200, k=4 does better: filtering an easy problem requires getting 4 successes out of 4, versus 32 out of 32 with a high k. In other words, the larger k is, the more compute easy problems consume before being discarded.
The announcement's formula fits in one line: "NGU solve hard prompts like k=32 and filters easy prompts like k=4".
| Standard GRPO (fixed k) | NGU (adaptive) | |
|---|---|---|
| Sampling | k attempts per prompt, then move on | k attempts; if everything is wrong, another pass with probability p |
| Easy problems | consume k samples until filtered (32/32 if k=32) | filtered in ~4 samples (4/4) |
| Hard problems (pass@32 = 0) | k samples, then move on — most often stuck at 0 forever | up to k/(1−p) expected samples (40 with k=4, p=0.9) |
| Failure samples | often discarded | all trained on, failures included |
The results: GSM8k platinum, DeepScaler, Manufactoria
The gains are real — and concentrated exactly where standard curves don't look. Three sets of experiments structure the paper.
On GSM8k platinum, NGU (k=4, p=0.9) beats every k value of standard GRPO. A detail worth noting: it solves more hard problems without degrading the easy ones. The reallocation isn't a forced transfer, it's an unclogging.
On DeepScaler, the scale-up is on Qwen 3 4B base, for roughly 120 H100 hours — an academic lab budget, not a cloud giant's. In the evaluation on AIME 2025 and BRUMO 2025, the gains concentrate on the hardest subsets, and DeepScaler shows the best performance per compute, precisely on these problems.
On Manufactoria, finally, comes the most telling demonstration: standard GRPO with per-test reward fails to fully solve problems that mix easy and hard tests. NGU iterates until complete success. On the hardest code tests, standard sampling fails where persistence suffices.
What This Implies for Benchmark Reliability
A flagship benchmark can climb for months without a single genuinely new problem being solved. This is the paper's most uncomfortable implication, and it extends beyond the circle of researchers.
The race for numbers rewards the aggregate delta: +X points on AIME, +Y on SWE-Bench. If the delta comes from the easy buckets, the messaging outpaces reality — a mechanism we've been following closely, from benchmark gaming around GPT-5.6 Sol on Cerebras to the SWE-Bench alternatives proposed by FrontierCode.
What NGU adds to this debate: even without cheating or contamination, an honest curve can mislead. The stagnation on hard problems remains invisible as long as you don't break things down by difficulty — and SWE-Bench platinum, with its difficulty labels, proves that such a breakdown is possible when the will exists.
The question to ask any model provider therefore becomes simple: "show us the curve on the problems your base model fails 32 times out of 32."
The limitations, acknowledged by the authors
NGU is not a machine for solving the impossible, and the paper does not claim otherwise.
First limitation: by resampling the same prompt, you accumulate samples that are more off-policy early in training, which degrades the signal. The gain on hard problems is paid for in temporary gradient quality.
Second limitation: if the task is almost entirely very hard, NGU loses its leverage. There are no longer any easy problems to filter out to free up compute, and the loop spins its wheels on problems that may be out of reach.
Third limitation, more conceptual: "hard" is defined relative to the starting model and the chosen k. A pass@32 = 0 for Qwen 3 4B base is by no means a verdict on intelligence. And training remains a risky exercise in itself: fine-tuning can break a model in far more visible ways, alignment first and foremost. Reallocating compute does not fix everything.
Concretely, what do you get out of it?
Three reflexes, depending on your point of view.
If you train models: split your evals by difficulty (pass@32 = 0 vs pass@32 > 0) before launching even a single RL run. It's free, and it's the only way to know whether you're learning or just polishing.
If you evaluate models for your stack: demand difficulty breakdowns from your vendors, and beware of averages alone. A model can gain two aggregate points while regressing on everything you care about.
If you want to verify for yourself: the models used in the paper (Qwen 3 4B base, Deepseek-R1-Distilled-Qwen-14B) and self-hostable open-weights models like GLM-5 or Kimi K2.6 let you run your own pass@k campaigns. A server from Hostinger (starting at a few euros per month, September 2026, check hostinger.fr) is enough to host the evaluation infrastructure for 4B to 14B models. The paper's method — label, split, measure — fits in a notebook.
❌ Common Mistakes
Mistake 1: reading an aggregate curve as uniform progress
A rising average can hide a flat 0% on hard problems — the paper demonstrates this on AIME, LCBv6 and SWE-Bench platinum. The solution: plot your curves by difficulty bucket and look at them separately, never stacked.
Mistake 2: paying for k=32 everywhere "just to be safe"
A high k finds rare solutions earlier, but it makes easy problems pay top dollar: you need 32 successes out of 32 to filter them, versus 4 out of 4 with k=4. After step ~200, k=4 does better. The solution: adaptive sampling that spends compute where it's lacking.
Mistake 3: throwing away failures on hard problems
Many pipelines discard prompts with no successes at all, for lack of usable signal. This is precisely what you should not do: these trajectories are your only raw material on hard problems. NGU trains on all accumulated samples, failures included.
Mistake 4: confusing "hard for this model" with "hard, period"
pass@32 = 0 is relative to a starting model and a k. Recalibrate your difficulty labels every time the base model changes, otherwise you're optimizing against a mirage — and you'll be reporting on a mirage too.
❓ Frequently Asked Questions
What exactly does pass@32 measure?
The success rate over 32 independent attempts on the same problem. A pass@32 = 0 means that out of 32 generations, none was correct. In the paper, this threshold defines "hard" problems: the model has no access to the solution, not even through random sampling from its own distributions.
Are NGU's code and data public?
The paper is freely accessible on arXiv, detailed on the author's blog, and the evals used (Deepcoder, DeepSWE) are open-source with public models and logs. The method itself fits in a few lines — resample with probability p, train on everything — and can be reimplemented in any asynchronous RL loop.
Does NGU replace GRPO?
No. NGU is a sampling strategy that plugs into an existing RL loop — the paper actually combines it with GRPO on asynchronous RL. It modifies neither the reward function nor the objective: it only decides which prompts receive compute, and how many samples each deserves.
Is it transferable to proprietary models like GPT-5.5?
Nobody knows: the experiments focus on open models (Qwen 3 4B base, Deepseek-R1-Distilled-Qwen-14B). But the diagnosis itself is universal: whatever the model, require a breakdown of results by difficulty before concluding that a training run has brought new capabilities.
How much does a reproduction cost?
The paper's DeepScaler scale-up represents roughly 120 H100 hours on Qwen 3 4B base — on the order of a few hundred euros in one-off GPU rental. The main experiments (GSM8k platinum, LCBv6) run on 14B models, making them accessible to an academic lab or a motivated product team.
✅ Conclusion
This paper doesn't say that RL doesn't work: it says that your numbers, as presented, don't prove that it works — and NGU shows you can do better on the same budget. Before your next RL run, slice your evals by difficulty: the authors' blog post is the best entry point.