The Observer in the Machine

Posted on

A problem at the heart of the emerging debate about AI safety is one I don’t think we are talking about enough. Dario Amodei, CEO of Anthropic, is now calling for slowing the development of frontier AI models and, among other safeguards, placing independent evaluators inside AI labs with enough access to determine whether new models are safe before release. Independent oversight is clearly a good idea, especially in an industry where commercial incentives point in the opposite direction. But I am increasingly skeptical that it addresses the more fundamental problem.

What happens when the system being evaluated understands the evaluator better than the evaluator understands the system?

Think about how we normally manage risk. Auditors examine companies, regulators inspect banks, engineers test aircraft and cybersecurity teams probe software. In each case there is an implicit assumption that sufficiently competent observers, given sufficient access, can understand enough about the system to determine whether it is behaving as intended. Independence matters because it removes conflicts of interest and lets the observer follow the evidence.

AI is beginning to challenge the second part of that assumption. A sufficiently capable model may know that it is being evaluated. It may understand what the evaluator is looking for, recognize which behaviors will cause it to pass or fail, and reason about what happens after it passes. The evaluation is no longer simply something being done to the model; it has become part of the environment the model itself is reasoning about. We have moved from observer → system to observer ↔ system.
Continue reading →

Your Company Is Buying AI. Is It Learning How to Think?

Posted on
The Thinking Enterprise

Companies are buying AI faster than they are learning how to invest in it. The important return is not only what an initiative delivers today, but what it makes possible next.

The AI Investment Problem

Artificial intelligence has created a difficult capital allocation problem for senior executives. They are being asked to move scarce capital away from established business priorities and into a technology whose eventual impact cannot yet be estimated with any confidence.

Moving too slowly carries an obvious risk. If AI continues to improve at its current rate, companies that fail to develop the required operating capabilities may find themselves at a significant and perhaps irreversible competitive disadvantage. Moving aggressively carries a different risk. Large commitments to immature technologies, individual vendors and poorly understood use cases may create little durable value while leaving behind a costly new layer of complexity.

Most AI strategies avoid this problem rather than address it. They provide lists of potential use cases, estimates of productivity improvement and roadmaps for deploying new tools across every function. The resulting activity creates an impression of progress. It does not answer the underlying investment question.

How should an enterprise invest in AI today without betting the business on a single prediction of AI’s future?
Continue reading →

The Leadership Attention Paradox

Posted on

“The manager’s job is characterized by brevity, variety, and fragmentation.”

— Henry Mintzberg, *The Nature of Managerial Work* (1973)

At the time, this was an observation.

Today, it is a constraint.

Most discussions of modern work start in the wrong place.

They start with overload. Too much to do. Too many decisions. Too many stakeholders. Too much pressure.

All of that is visible. None of it explains the real failure mode.

The deeper issue is not volume.

It is continuity.

There is a structural shift that happens as work becomes more important.

The work itself changes shape.

It becomes less bounded. Less discrete. Less compressible into short cycles of attention. The things that matter most—strategy, judgment, synthesis, communication at the highest level—do not resolve quickly. They require sustained thought. They require someone to stay with them long enough for the structure to emerge.

This is true well beyond leadership. It applies to anyone doing complex, multi-threaded work. The more consequential the work becomes, the more it depends on continuity of attention across time.

And that is where the system begins to break.

Continue reading →

The Long Road to the Threshold

Posted on

1. Novelty Is Usually a Disguise

Most technology ideas arrive in the world disguised as novelty.

They are announced as rupture. As invention. As the sudden appearance of something no one could have imagined before the market finally made it possible to say it out loud.

That is rarely how the deepest shifts actually happen.

The more consequential pattern is quieter. A problem appears early, in primitive form. It is partially solved, then broken again under scale. It reappears in new technical clothing. It is misunderstood as tooling, then rediscovered as architecture, then rediscovered again as governance, then again as human limitation. Decades pass. The surface changes. The underlying failure remains.

I have spent much of my professional life in the shadow of one of those failures.

2. The Mismatch Beneath Modern Software

The failure is not a lack of software. It is not a lack of intelligence. It is not even, in the end, a lack of automation. It is the persistent mismatch between how serious human work actually unfolds and how most systems are designed to contain it.

Real work does not happen in tidy, isolated transactions. It stretches across time. It accumulates context unevenly. It leaves traces in conversations, half-finished notes, strategic intentions, unresolved decisions, documents, revisions, and memory. It is recursive. It is interrupt-driven. It is often ambiguous until quite late in the process. And yet most software continues to behave as though work arrives in cleanly bounded units, ready to be filed, routed, and processed inside static tools designed in advance.

That mismatch has never seemed small to me.

I wrote my first line of code in 1981. Since then I have lived across much of the terrain where complex systems meet real institutional pressure: engineering, architecture, platform design, large-scale organizational technology, data-intensive environments, executive operating contexts, and transformation work inside large enterprises. Across all of it, one lesson kept resurfacing: the hardest problems were almost never purely technical. They were problems of fit. Problems of structure. Problems of continuity. Problems of what happens when human intent meets systems that are too rigid to adapt and too loosely governed to trust.

Continue reading →

Beyond the Codex Moment

Posted on

In my previous post, I described what I called my “CODEX moment”: the realization that the capabilities of agentic engineering tools have crossed a threshold.

For roughly a decade, development tooling improved incrementally. Better IDEs, better automation, better infrastructure abstraction. The tools accelerated individual tasks, but the underlying production model of software engineering remained constant.

Over the past month, that model changed.

The current generation of agentic engineering systems does not simply help write code faster. It allows an architect to design and operate a system in which autonomous agents execute large portions of the engineering workflow.

That is a step-function transition in capability.

But it is also where much of the current conversation begins to miss the point.

⸻

Continue reading →

My CODEX Moment

Posted on

An Inflection Point

I wrote my first line of code in 1981 on a Sinclair ZX80. It wasn’t compiled or linked, and persistence was tenuous at best. You wrote in BASIC, pressed run, and hoped you hadn’t mistyped something, because debugging meant staring at a flickering television screen and reasoning it out manually.

At university I moved to FORTRAN—compilers, linkers, batch jobs, structured thinking. Code became something you constructed carefully, fed into machinery, and waited for a verdict. That mental model stayed with me.

Over the next four decades I wrote and delivered production systems in more than half a dozen languages. I worked in C and C++, shipped enterprise Java, built distributed systems, and stood up cloud infrastructure. I led large engineering organizations, debugged race conditions at 2 a.m., negotiated vendor contracts, and migrated monoliths into microservices.

The stack evolved. The abstractions improved. Tooling became more sophisticated.

But the core production model remained constant: a human translated intent into syntax, and a machine executed it. Software development, at its core, remained a manual craft supported by increasingly capable tools.

Until last week.

Continue reading →

Zombie Governance

Posted on
Empty boardroom room with performance data on screens but no humans in attendance.

When Systems Outpace Judgment

Modern institutions now make more consequential decisions in a single day than leaders once made in a year. Increasingly, those decisions are not deliberated over directly; they are produced by systems. Credit approvals, pricing adjustments, risk allocations, hiring screens, eligibility determinations, capital routing — these unfold at a speed and scale no committee, however capable, could realistically match.

Human judgment does not scale with decision volume. It depends on context, on deliberation, on the uncomfortable work of weighing competing considerations when circumstances resist simplification. Those conditions do not shrink simply because throughput increases. They do not accelerate because markets demand it. Machine execution expands almost effortlessly. Human reflection does not. What was once a manageable imbalance has become a structural challenge.

Operationally, automated systems often produce impressive results. Variance declines. Auditability improves. Decisions become statistically defensible in ways boards and regulators understandably prefer. In many sectors, refusing automation would not signal prudence; it would signal negligence. And yet, as these systems embed more deeply, leaders find themselves accountable for outcomes they did not personally decide and cannot fully explain except by pointing to model logic or policy configuration.

When those outcomes are challenged, the language shifts. Calibration thresholds. Performance tolerances. Compliance adherence. None of this is wrong. But it is revealing. Explanation moves from reasoning to mechanism. Authority remains formally human, yet the practical locus of decision-making has moved elsewhere. The person affected by the outcome rarely sees the deliberation — only the result.

The emerging risk is not spectacular failure. It is something quieter: the gradual displacement of visible judgment from the point where consequences are felt, even as responsibility remains intact. Institutions continue to function, by conventional metrics, to perform well. But the deliberative presence that once accompanied consequential decisions becomes harder to locate, diffused across upstream configuration, policy architecture, and system design.

This essay introduces a term for that condition: Zombie Governance.

Continue reading →

Now’s the Time to Check your Digital Posture

Posted on

Cross-posted from dprism.com. Visit us for additional insights on digital posture, digital growth strategy and modern operating models.

Covid-19 is not the first massively disruptive event we have had to navigate, and it will certainly not be the last. But what we are now managing makes clear that the maturity of an organization’s digital posture directly impacts its resilience and ability to adapt quickly to these types of external shock.

In military strategy, force posture is defined as the combination of materiel, capabilities, placement, infrastructure, personnel, industrial base and the economic wherewithal to bring capabilities to bear quickly. The same strategic analysis can be applied to any organization’s need to compete, grow and adapt to challenges in today’s digital economy. We use the term digital posture to describe that analysis.

Digital posture: The combination of digital products and services, processes, tooling, skills and culture that enables an organization to achieve its strategic objectives and to adapt and respond and be resilient in the face of adversity.

Continue reading →

Five keys to a modern operating model

Posted on

Ask a dozen consultants what the term “digital transformation” means and you’ll likely get a dozen different answers. The term has become so overused and so lacking in definition it has become meaningless. This lack of clarity is problematic for senior executives charged with leading their organizations through increasingly challenging times. There’s no escape from the fact that the world in which we live and operate today is digital. Transformed or not, all organizations operate in an economy where all meaningful transactions and interactions with customers and stakeholders are intermediated in the digital realm. The key question for executives is not whether they can successfully drive a digital transformation, but whether they can build a modern operating strategy and model that can thrive in a digital world.

Continue reading →

Three keys to sustainable (software) development

Posted on

Executives making substantial investments in new technology platforms and capabilities have a not-unreasonable expectation that their investments will have a lifespan of over a decade or more. This is particularly true for leaders of organizations whose businesses are not rooted in the constant change and turmoil of the technology industry.

Unfortunately, such reasonable expectations run counter to the reality of cycle times in the IT industry. Technologies come and go, become fashionable and then become irrelevant over periods as short as six months or a couple of years. Software and technology platforms, development tools and architectures are born, thrive and die – or are replaced – within a decade.

How then should we square the need for sustainable returns on new systems investments with the countervailing cycle of constant technology disruption?

The answer involves understanding three core concepts: iteration, abstraction and…Legos.

Continue reading →