Your model can code, so what?

1. Your model can code

It is becoming increasing clear that one shot programming a moderately complex file or function is trivial for most large language models (both frontier, and now including open-source). Furthermore, an increase in general codebase understanding, through both intelligence/reasoning skill increase of underlying models, as well as harness engineering and tool engineering, is further allowing rapid increase in productivity for debugging and codebase creation pipelines.

Benchmark	Launch	Initial	Current	Time to collapse
HumanEval (pass@1)	2021	28.8%	95%+ (solved)	~3 years
SWE-bench Verified	2023	~4%	80%+	~2.5 years (ongoing)

The above table shows this, with the one-shot HumanEval benchmark (as well as MMLU) long solved, demonstrating the intelligence/reasoning skill increase (and now saturated). Whilst the SWE-Bench Benchmark shows a huge increase in performance from frontier models, although there are huge saturation worries currently (before being well solved).

Despite this, other benchmarks that are more closed are noting a huge increase in codebase debugging, tool usage, and understanding, in time periods that continue to outpace the rate at which we produce benchmarks, and this can be readily observed by anyone using a modern agentic harness (for example Hermes).

So whats the problem?

Lets get the obvious part out of the way. There is no current guarantee that coding and programming capability definitely implies a increased proximity to an AGI or Super-Intelligence. It has long been posited that as soon as you can teach a model to code as well as the people who created it, it will improve itself to a singularity.

This presumes what is essentially a linear scaling to complexity. Furthermore it indicates what is essentially a sanctification of the concept of coding. Coding is what younger devs do, because it can be easily taught and structured. Senior Developers have always moved more to the side of management and delegation, and this has always been understood in industry to be a far more difficult task that relies on quite a fair bit of intuition over many years of experience on how to create long-term stable codebases and delegate people to produce and maintain it very precisely.

Alongside this, this process as a simple single Senior Developer and set of Junior Developers, lends itself to a direct goal -> production pipeline, whereby an iterative improving pipeline can not be easily bolted on, as each goal is pretty much blind to the next iterative improvement. Continuing this (albeit a bit disjointed) analogy, this is why most new technology ideas are tied to entirely new companies or papers: long term visions of improved technology seem to lend themselves to starting anew each time (obviously pre-existing general frameworks designed to be extensible and reusable can be utilized, you needn’t start entirely from scratch).

Despite these qualms, if we want a mostly self contained AI that aims to reach a Super-Intelligence, we are stuck somewhat to a regime of iterative self improvement, perhaps with the option for the model to delegate to start from scratch with what it has learnt.

So What?

Your model can code, so what? What does programming and extensibility mean in the context of transformers bolted into harnesses?

Well, assuming a very basic one shot step by step file by file agent that doesn’t debug itself, due to the inherent probabilistic nature of transformers we are lower bounded by the following long term horizon success rates:

Per-step accuracy	10 steps	50 steps	100 steps
99%	90.4%	60.5%	36.6%
97%	73.7%	21.8%	4.8%
95%	59.9%	7.7%	0.6%
90%	34.9%	0.5%	0.003%

This does presume a few things that do need to be noted. Firstly, this assumes that we produce a file, add it to our codebase, then move on to another, without debugging. Debugging is essentially what benchmarks like SWE-Bench measure, and its clear we are getting better at it, but we can’t ensure perfection every time.

The second thing is that this presumption of a constant error rate drastically underestimates the problem. As noted earlier, complexity and the errors that come from it do not scale linearly, in fact they always seems to explode (this is why naive debugging every step is futile). There are three pieces of recently published research that show why complexity explodes:

1. Prior errors have been shown to contaminate the context, making future errors increase over time. [arxiv:2509.09677]

2. Early Decisions and constraints scroll out of context windows, especially on already complex tasks, compounding errors. [arXiv:2604.11978]

3. There is Formal Blindness of a first pass metric. This paper proved that single shot benchmarks are unable to distinguish between models with identical 1-step accuracy but diverging 10-step reliability. [arXiv:2603.29231]

Boiling these all down, we are currently bottlenecked by error accumulation in Long-Term agentic harnesses.

So how do we fix this?

We are still going to need two more components:

1. Raw next-step intelligence that also takes into account LONG horizons. It seems to be the opinion of most currently that the models are officially smart enough, and this is an easy trap to fall in to. Plenty of benchmarks are pretty much solved, but the over-reliance of single step metrics, and what I would call pseudo multi-step metrics - Most agentic coding benchmarks where you only need to accomplish a single or few “task/s” through delegation, and one where the harness can be over-relied on - means that there is simply not good enough evidence to suggest we have the right model to oversee it’s own improvement to an ever-increasing degree.
2. Harness engineering. Scaffolding, verification loops, memory that actually helps, and recovery protocols that in my opinion should let the model start anew when necessary.

That second step currently seems like all the rage right now, but fundamentally all it can do is move the complexity explosion further to the right. Is it possible its enough for the singularity? Sure, but a truly super intelligent model will be made one day, and I doubt we will get there through asking it to verify its response again and again.

Thanks for reading!

7. Sources

1. arXiv:2603.29231 — Beyond pass@1: A Reliability Science Framework for Long-Horizon LLM Agents (2026).
2. arXiv:2509.09677 — Measuring Long Horizon Execution in LLMs (2025).
3. arXiv:2604.11978 — The Long-Horizon Task Mirage? (2026).
4. arXiv:2511.09030 — Solving a Million-Step LLM Task with Zero Errors (2025). MAKER.
5. Tian, P. (2026). The Compound Accuracy Problem. tianpan.co.
6. Chen, M., et al. (2021). Evaluating LLMs Trained on Code. arXiv:2107.03374.
7. Jimenez, C. E., et al. (2024). SWE-bench. arXiv:2310.06770.
8. arXiv:2506.17208 — Dissecting SWE-bench Leaderboards (2025).
9. Stanford HAI (2025). AI Index Report.
10. Morris, M. R., et al. (2024). Levels of AGI. arXiv:2311.02462.
11. Chollet, F. (2019). On the Measure of Intelligence. arXiv:1911.01547.
12. Hendrycks, D., et al. (2025). A Definition of AGI. arXiv:2510.18212.
13. Goertzel, B. (2014). AGI: Concept, State of the Art, and Future Prospects.
14. Liu, N. F., et al. (2024). Lost in the Middle. TACL.
15. Attainment Labs (2026). Frontier AI Model Report — Feb 2026.