We were working through a system integration between two company where both teams had done what they were supposed to do. Our service produced the expected output. Their system accepted it. The interfaces were behaving as designed.
The problem appeared in the workflow between the two organizations.
A piece of state that mattered to the business decision existed upstream on the client side, but it was not part of the integration contract. Our engineering team had no reason to expect it, and the client team assumed it would carry through. Everyone could point to a component that was working correctly, yet the end-to-end process was wrong.
Getting through it required more than fixing an interface. We had to bring the teams back to the business outcome, trace the decision across both organizations, identify where assumptions changed ownership, and agree on who was responsible for the state at each boundary.
That experience changed how I think about AI skills. Systems thinking used to be something we expected from architects and senior engineers. AI is turning it into an entry requirement. As execution gets cheaper, the hard decisions move to the seams: what context is available, where state lives, what happens downstream, and who owns the outcome when a workflow crosses teams or companies.
There is a catch. Systems thinking without domain depth can make the problem worse. AI makes it easier to build a coherent version of the wrong process.
OpenAI described this shift in its Codex work. In one internal experiment, a team built a production product with no manually written code and estimated that it took about one tenth the time conventional development would have required. The human role moved toward designing environments, specifying intent, and building feedback loops around the agents doing the execution.
That changes what creates value. Producing another component is getting easier. Understanding what happens between components is getting harder.
We saw that in another part of the same kind of workflow. When automation reached a certain level of reliability, human review fell faster than the improvement itself could explain. The reviewers had, reasonably, begun trusting the system. Our caught-error rate collapsed, and it took a while to recognize that as a warning rather than good news.
The local metric improved while the system became less reliable. Fixing it meant changing the interaction between the technology and the people using it, not improving the model again.
Anthropic has documented a similar effect in a different setting. Its telemetry showed users approving roughly 93% of Claude Code permission prompts. As the number of approvals increased, people paid less attention to them. The control was technically still there, but human behavior had changed its effectiveness. Anthropic responded by changing the containment model around the agent rather than assuming more approval prompts meant more safety.
Microsoft Research found another version of this in a study of 860 developers. The developers wanted AI systems with explicit limits on authority, provenance, uncertainty signals, and least-privilege access. The researchers called the pattern “bounded delegation.” What interested me was the boundary. The useful design question was as much about where the AI should stop as what it could do.
I see the same shift in how we think about context.
A team can spend days tuning instructions when the missing information sits in another database, belongs to a different process, is hidden behind a permission boundary, or disappeared three steps earlier in the workflow. At that point, prompt engineering cannot rescue the design.
Sourcegraph describes context engineering as designing the full pipeline that determines what a model sees, including retrieved information, memory, tool definitions, and prior interactions. That framing matches what we see in production. Context is an architecture decision. State, permissions, retrieval, and the tool surface determine whether the model has any chance of making the right decision.
Stripe reached a related conclusion while testing agents on real integrations. Models that could solve scoped coding problems still had to deal with persistent state, failure recovery, testing, and the “glue” between APIs, databases, and front ends. Stripe found that evaluating the isolated coding task missed much of what made the real work difficult.
There is another systems question that I think matters even more: should the workflow exist?
I have been in client conversations where the initial request was to automate an existing process. The obvious response was to make each step faster. The better discussion started when we traced the business decision end to end and asked why each handoff existed.
In one case, much of the information being requested downstream already existed elsewhere in the process. We could have automated the handoff and shown a convincing efficiency number. Instead, part of the handoff disappeared. The measured time saving would have looked similar, but the system became simpler.
This is where I would qualify the argument that systems thinking is the skill everyone now needs.
Systems thinking without domain depth can be dangerous.
I’ve watched a strong engineer design a clean orchestration for a cross-border workflow without knowing that one document, issued by a government agency, changes which filings apply and when they’re due. The diagram was correct. The product would have been wrong in a way that only shows up months later.
I’ve also watched the opposite: someone who knew the rules cold and could not see the loop they were part of, so every fix was local. The people who ship in regulated work carry both, and in my experience that combination is rarer than either half.
There is a harder problem underneath this. We may be making systems judgment more important at exactly the point when AI is removing some of the work through which people used to acquire it.
Mark Russinovich and Scott Hanselman make this case directly in Communications of the ACM. They argue that AI can amplify experienced engineers while creating what they call “AI drag” for early-career developers who do not yet have the judgment to steer and verify the output. Their concern is the talent pipeline: if companies automate junior work and hire primarily experienced people, they eventually remove the apprenticeship that creates experienced people.
I used to assume people would develop systems judgment the same way many of us did, by spending years close to the work and gradually taking on larger problems. I am less convinced that path will survive unchanged.
That changes what I look for in teams. Tool familiarity matters less to me than it did a year ago because the tools depreciate quickly. I care more about whether someone can sit with an engineer, a product owner, a domain expert, and a client and trace a decision across the entire workflow. They need to see where assumptions change, where ownership becomes unclear, and where improving one part creates a problem somewhere else.
That is increasingly what leading AI work looks like to me. The difficult part is rarely getting another component to work.
It is getting the whole system, including the people around it, to work.
