~4m1:40:55
The Students

Jonathan Blow — If AI Solved Coding, Why Is My Software Still the Same?

Nov 13, 2026

Read: ~4m · You save: 97 min

Jonathan Blow: If AI Really Solved Programming, Why Is My Software Still the Same?

Game developer Jonathan Blow dives into why software keeps growing more complex and less reliable, and the real limits of AI programming amid bubble debates.

Background

Jonathan Blow is a programmer and video game developer, known for works including Braid, The Witness, and Order of the Singing Star. In interviews, he has expressed his views on the current state of software engineering and his assessment of artificial intelligence's programming capabilities. In 2019, he gave a talk on software complexity and reliability that sparked widespread discussion.

软件工程缺乏职业责任体系

布洛认为,软件工程的核心问题在于缺乏与传统工程相当的责任机制。

  • 传统工程的约束: 工程师必须理解现实约束,并设计出达到预期效果的系统。桥梁工程师需要对结构安全负责,历史上工程师甚至需要在桥下生活,若桥梁坍塌则面临法律后果。这种责任形成了行业的谦逊与审慎文化。
  • 软件行业的现状: 数据泄露和性能问题频繁出现,用户已习以为常。商业软件长期处于“无法无天”的状态。航空航天领域的软件曾建立相应规范,但这些规范没有延伸到商业软件。
  • 原因分析: 布洛表示难以确定单一原因。他认为这是多因素造成的,包括行业缺乏法律后果与监管体系,以及早期从业者的经验问题。

The Gap Between Complexity Growth and Accumulated Experience

Bullow points out that early software was far less complex than today's, so practitioners were able to master the entire system.

  • Early software: In the 1970s, system structures were simple, and hardware was relatively simple as well. New employees usually needed only one or two months to become familiar with the existing software. Flowcharts were once regarded as a sound tool for program design.
  • Today's software: The complexity of high-end software has increased significantly. Bullow considers video games to be the most complex type of software he knows. He also argues that the more fun a game is, the more likely it is to conflict with the established conventions of software engineering.
  • The experience structure of practitioners: The software industry has grown rapidly over a long period, and the number of newcomers keeps increasing. Bullow illustrates this with a doubling example: if the number of new entrants doubles each year, then half of practitioners have zero experience, a quarter have one year, an eighth have two years, and the average experience is about one year.
  • Time needed to master a system: Bullow studied computer science at the University of California, Berkeley. He considered himself a good student, but after graduating he found that he was not familiar with most complex technologies. He estimates that it takes about five to six years of experience to truly understand the whole picture of a system.
  • Loss of knowledge: Within a company, only a small number of employees who have worked for many years understand why the system was designed the way it was. Many practitioners lack historical context, so they repeat the same mistakes or argue about technical directions without a sound basis.

The History and Technical Debt of Operating Systems

Bu Luo uses Unix as an example to explain how software systems gradually become entrenched over time.

  • Original design intent: Unix was designed to be simple, fast, and easy to use. One of its core principles is to treat system resources as files that can be read and written.
  • Time-sharing mechanism: Early computers had only one CPU, and multiple programs shared it through time-sharing. Modern operating systems still use this mechanism, only at a much finer time granularity.
  • Security issues: Bu Luo cites an early case in which the password verification of a government system was cracked. The attacker exploited the timing difference between memory page boundaries and character-by-character comparison to infer the password. Afterward, password comparison algorithms were changed so they no longer return early on a character-by-character basis, and passwords were stored in encrypted form.
  • Challenges of parallel architecture: Today's computers are generally based on multi-core parallel architectures, and shared resources such as CPU caches have given rise to new security problems, such as the Heartbleed vulnerability and cache conflict issues. Bu Luo argues that fundamentally solving these problems would require redesigning the kernel architecture, for example by running the operating system on a dedicated CPU. However, such a change would be extremely costly, and the industry lacks the impetus to drive it.
  • Layer stacking: Once a design is adopted, the cost of changing it becomes very high. New improvements are often implemented by adding new layers, eventually forming a complex structure of stacked layers. Bu Luo compares this to a "landfill": new waste keeps covering old waste, and digging down to the bottom becomes very difficult.
  • Increasing difficulty of modification: The larger the system, the more dependencies it has, and the more likely that modifying any one part will affect other parts. Software engineering theory advocates decoupling, but in large systems this is difficult to achieve fully.

Evolution of File Systems and Foundational Software

  • File systems: Early file systems were prone to data loss during improper shutdowns. Modern file systems have improved significantly, and after a power failure, most data can usually be recovered. However, the startup recovery process may take about 15 minutes.
  • Changes in Linux: Linux was originally a small and flexible operating system, so modifying its underlying design was relatively easy. As it has grown in scale, making fundamental changes to it is now nearly impossible.

Platform Strategies in the Industry

Bu Luo argues that large technology companies commonly adopt platform strategies, designing their APIs to differ from those of competitors in order to raise developers' migration costs. He cites iOS and Android as an example, noting that developers must invest considerable effort in adapting to each specific platform. He also mentions that Linux follows a similar pattern in developing new features, such as the /proc filesystem, whose design differs from that of other systems and reduces portability.

Bu Luo uses systemd to illustrate the growth in system complexity. Early Unix startup scripts typically had only five to ten lines, whereas systemd's codebase has exceeded 1.5 million lines and takes on multiple functions, including login and message passing.

Duplicated Effort in the Open Source Ecosystem

Bruno points out that the history of Linux distribution forks contains a great deal of duplicated work. Many distributions died out within a short time, producing a large amount of redundant development. He admits that forking can, in theory, lead to valuable innovation, but in practice it wastes considerable resources.

He argues that a major problem in the open source community is the lack of preservation of existing knowledge. He suggests establishing a role similar to a "monk," responsible for recording the reasons behind system design, so that later developers can understand historical decisions.

Technical Debt and Code Quality

Bro believes that technical debt is acceptable to a certain extent. In the early stages of a project, rapidly delivering features helps gather feedback and make better decisions. But when technical debt accumulates to a certain point, the team needs to stop and pay it down. Bro says he often does this in his own projects, but performs this kind of work less frequently within teams.

He believes that one of the essential abilities of excellent programmers is keeping systems clear, easy to understand, and maintainable. He has observed that even though schools emphasize this, graduates often still add large amounts of code after starting work.

Bro argues that minor superficial changes are often easier to complete than work that truly improves technology, so the industry tends to pour a great deal of effort into them.

Assessing AI Programming Ability

AI's Performance in Writing Code

Bruno believes that current large language models perform poorly at writing high-quality code.

  • Code quality: He observes that code output by AI is usually rather messy. If it produced the same code at the speed and cost of a normal human, it would in most cases not be hired. AI's advantages are speed and low cost.
  • Suitable scenarios: Bruno gives an example in which he used AI to generate about 30 lines of Visual Basic code for formatting spreadsheet cells. He believes that tasks like this, which are well-defined, small in scale, and will be thrown away afterward, are suitable for AI. He estimates that completing the task from scratch would take about 5 hours, while with AI it took only about 1 hour.
  • Value for non-professional users: For users unfamiliar with programming, AI can significantly lower the barrier to entry, which is a genuine improvement in capability.

AI's Reasoning Ability

Bruno believes that AI's reasoning is merely reasoning on the surface.

  • Judgment and trade-offs: AI can produce reasoning processes that seem plausible, but its output shows that the technical trade-offs it makes are often inappropriate.
  • Prediction example: Bruno once asked AI to predict the sales of a game. The AI produced a well-structured line of reasoning, but when he pointed out a factor it had missed, the model immediately changed its conclusion, showing that its reasoning lacks stability.
  • Mathematical proofs: AI-generated mathematical proofs can be long and lack insight, yet many of them are correct. Bruno believes that models perform better on problems that are well-defined and self-verifiable. But even so, their efficiency remains lower than that of human mathematicians.
  • Large-scale systems: As programs grow in size, programmers need to understand the constraints and influences among their components and build a mental map of them. Bruno believes that current AI lacks the reliable reasoning needed for this task and can only work by relying on approximate "intuition."

Productivity and Software Progress

Bruno believes there is no visible evidence of productivity gains from AI programming.

  • Evidence of output: If productivity had increased tenfold starting three years ago, that would amount to thirty years of software progress. Bruno says he has not seen such progress. He believes AI is an important technological advance, but as an outside observer, he has not seen a clear improvement in software production.
  • User experience: Software still has many defects, and there are not many new features. Bruno believes that consumers have not experienced a significant increase in productivity.
  • Code growth: Bruno estimates that AI-generated code is usually far larger than what human programmers produce; for example, a mathematical proof may be 10 to 100 times longer than the version written by a human. This extra code becomes technical debt for the next round of iteration.
  • Error rate: As scale grows, the error rate per line of code rises, eventually making it impossible for AI to complete the task. Bruno believes that projects relying purely on "vibe coding" will reach this threshold quickly, but some simple programs have not yet come close to it.

The Role of AI in Security and Quality Tools

Bruno believes that AI is a useful tool for discovering vulnerabilities, similar to fuzzing and static analysis tools. He acknowledges that these tools can also be used maliciously, and attackers may exploit vulnerabilities faster than defenders can fix them. He notes that the disclosure process for some recent software has been very poor, with attackers already exploiting vulnerabilities before updates were released.

Bubbles and Industry Trends

Blow believes that the current AI frenzy very likely contains a bubble.

  • Historical bubbles: He has lived through two software industry bubbles: the internet bubble and the startup accelerator craze from 2010 to 2017. During a bubble, many companies are of poor quality, but valuable businesses still remain after the bubble bursts.
  • Current software startups: Blow observes that many AI-related software companies merely call agents to complete simple tasks, lacking substantive innovation. He believes that even if these companies grow to scale, they depend on the foundational services of large AI platforms, so they are not commercially stable.
  • Company quality: He estimates that many AI companies have products with low technical content, but it is difficult to predict in advance which companies will succeed.

Blow believes that some technologies in the AI field are real and will create value in the future, but the current frenzy includes a large number of companies that lack actual output, and this part is inevitably a bubble.

Conclusion

Bullo argues that the software industry lacks accountability mechanisms and knowledge-preservation systems, which allows complexity to keep growing and makes systems difficult to improve. He contends that current large language models are not suitable tools for writing high-quality, long-maintainable code, and that there is no visible evidence of their productivity gains. He also acknowledges that AI has practical value in specific tasks and in the field of security tooling.

The Missing Accountability in Software Engineering

Blow argues that the software industry has long lacked genuine engineering discipline, in sharp contrast to the accountability traditions of conventional engineering. The root cause lies in the field's rapid growth and insufficient accumulation of experience.

  • Blow notes that his 2019 talk sparked widespread discussion, with many people agreeing that software is growing increasingly complex and increasingly unreliable
  • For decades, the software engineering community has shown overconfidence in paradigms such as object-oriented and functional programming, yet these have not truly solved the problem of reliability
  • Real engineers must understand real-world constraints and bear responsibility, whereas the software industry lacks bridge-style accountability and a guild tradition
  • Aerospace software has certain established standards, but these have not spread to commercial software, which has long remained in a "Wild West" state

Loss of Experience and Industry Expansion

Rapid expansion of the software industry means many newcomers lack experience, while systems themselves grow more tangled, and knowledge is lost as people move on.

  • If the industry doubled every year, most practitioners would be inexperienced, with average experience of about one year
  • Blow himself studied computer science at Berkeley for four or five years, and only after graduating realized how much he didn't know; he didn't truly accumulate experience until about five or six years later
  • Within companies, the reasons behind key design decisions often depend on a few veteran employees who remember the history
  • Early Unix had numerous vendor variants, and projects like BSD and Linux remained divided for a long time, forming a chaotic ecosystem

Unix History and Design Lock-in

Unix was originally designed to be simple and fast, but over time its complexity has grown severely, and many early designs have become locked in and are difficult to change.

  • Core Unix philosophy includes treating system resources as files, an idea that Linux has further extended
  • The time-sharing design originated in the single-CPU era; in today's multi-core parallel environments, this architecture has caused many security problems, such as Heartbleed
  • Blow cites a classic case in which a government system's password check was cracked character by character by probing page boundaries, which prompted later improvements to password comparison algorithms
  • Once software takes shape, modifying it is extremely costly; the more dependencies it has, the harder it is to change, and the system gradually becomes rigid

The Capability Boundaries of AI Programming

Blow argues that AI programming has advantages in speed and cost, but the code quality is low, and it struggles to handle the internal constraints and trade-off judgments of complex systems.

  • Blow argues that large language models are not good at writing high-quality code; their output is often messy, and they frequently generate many times more code than humans do
  • AI is more like an algorithm capable of high-speed search, good at clearly defined tasks, but lacking creativity and judgment
  • In the spreadsheet case, AI completed the task with about 30 lines of Visual Basic, which is very valuable to non-programmers, but this kind of scenario is one-off use
  • Large programs require programmers to maintain many constraint relationships among nodes, and this is reasoning work that current AI struggles to perform reliably
  • Blow points out that the reasoning explanations AI generates often seem plausible, but the output shows that its trade-off judgments are incorrect

Productivity and the Bubble Debate

Although AI is widely promoted as significantly boosting productivity, Blow sees no corresponding growth in actual software progress, and he believes the current situation may be a bubble.

  • Blow argues that if AI had made productivity 10 times higher for three years, software should have advanced by the equivalent of 30 years, but he believes almost no such progress is visible
  • Operating system interfaces have barely changed fundamentally between 1990 and 2026, with main improvements coming from hardware
  • New features in Linux are limited, and a large amount of newly added code has a negative net effect
  • The open-source community has many redundant forks, causing a great deal of duplicated work
  • Blow believes the current AI boom shows signs of a bubble, with many AI companies lacking substantive innovation and most being low-barrier applications