Engineers wrestle with AI code they can no longer fully understand

The Black Box Problem

A software engineer at Meta recently posted on LinkedIn about a growing unease in the industry: as AI code-generation tools become routine, many developers are shipping systems they cannot fully explain.

"I'm writing code these days and it all feels like a black box," the engineer wrote. "You understand the high level, but not every detail. And maybe this is becoming the new standard. It goes against every instinct of being a software engineer."

The post, which quickly circulated among developers, asked whether others felt the same way and how they were handling the pressure to ship faster while losing sight of the details ¹.

Design First, Then Delegate

One of the most upvoted responses came from a developer going by bitwizeshift, who said the solution is to keep the AI out of the design phase entirely. Instead of asking an LLM to plan a system, the engineer recommends writing out the full hierarchy of abstractions, classes, and interactions yourself, then using the model only to fill in the implementations.

"I don't let the LLM design the code at all, I let it implement a design I have created," bitwizeshift wrote.

The commenter argued that LLMs are generally poor at creating good abstractions, producing code that works initially but becomes hard to maintain as requirements change. By contrast, a human-led design phase — even one that takes a few hours up front — keeps the architecture stable and makes future refactors easier.

This approach is not new in principle. Experienced software architects have long insisted on separating design from implementation. What has shifted is the speed at which teams can now generate code, creating temptation to skip the upfront planning step ¹.

The Culture of Pasting Tickets

Several commenters extended the discussion beyond technique to what they saw as a broader decline in engineering rigor across the industry. swatcoder, another developer, argued that many people carrying "senior" titles at large tech companies have little practical experience with software-design principles, spending most of their time pulling tickets and applying minimal patches rather than thinking about system structure.

Another contributor, birdsongs, described working at a company where reviewing thousands of new or changed lines each week was impossible. "I just approve PRs, because I have to," they said, adding that they had quit caring about code quality after their own complaints were ignored.

birdsongs, who described themselves as a senior engineer with a decade of experience and patents to their name, also shared a recent job-interview experience that highlighted a different kind of disconnect. When asked about "NLP," the interviewer meant "Natural Language Programming" rather than Natural Language Processing, the field that studies how computers understand human language. The candidate's attempt to answer seriously did not save the interview ¹.

Why This Matters Now

The conversation comes as tools like Claude, GPT-5 and other large language models continue to be integrated into everyday development workflows. The promise of these systems is speed: developers can generate boilerplate, tests, and entire modules in seconds. The risk, as the Hacker News thread made clear, is that speed can crowd out understanding.

Engineers who voiced concern in the thread did not argue for abandoning AI-assisted coding. Their point was more specific: the human architect must still own the design, or the codebase will eventually become impossible to maintain, debug, or trust when something goes wrong.

As the Meta engineer's original post suggested, the question is no longer whether AI will change how software is built, but whether the profession will adapt its standards fast enough to keep pace ¹.

This story reports on a trend for information only. It is not advice or encouragement to do any of it.

Key facts

• A Meta engineer publicly questioned whether shipping AI-generated systems they cannot fully understand is becoming the new industry standard.
• Respondents recommended keeping the human in charge of system design and using LLMs only to implement predefined abstractions.
• Multiple commenters argued that many so-called senior engineers lack deep experience with software-design principles.
• One senior engineer recounted an interview where the interviewer confused Natural Language Processing with "Natural Language Programming."

Sources

1. Ask HN: How to deal with shipping AI-written code you don't understand? | Hacker News — news.ycombinator.com