Editor’s note: Bill Hilf is the former CEO of Vulcan/Vale Group, the current chairman of the board of Ai2 and American Prairie, and the author of the new sci-fi novel, The Disruption, which explores the topics of AI and the natural environment. He talked about the book on the GeekWire Podcast, and elaborated on the themes in this companion story.
We are building AI on a civilization scale while still talking about it as if it were a software release.
Which model is superior in which benchmark. Which chatbot sounds the most human. Those questions are important, but the wrong height. AI systems no longer just answer questions. They mediate employment, diagnosis, logistics, finance, and increasingly pieces of public decision-making. We no longer ship products. We are reshaping places.
At this scale, AI is highly connected. Connect failure modes. Urgent behavior. Invasive species. Points to give advice.
Treating space as a product is a category mistake, and it’s already compounded.
I’ve spent three decades building the systems now at the center of this conversation, from scientific computing at IBM to early Azure and large enterprise systems at HP. The operating model was deterministic: specify the system, build it, tune it, manage it. If something breaks, diagnose and fix. This model works until it doesn’t.
At a sufficient scale, distributed systems stop behaving like machines and start behaving more like ecosystems. They are adaptable. They revolve around failure. They develop dependencies that no one designed and interactions that no one fully understands. You can still design and engineer them. But when they are embedded everywhere, connected to everything, and developed in too many layers for any one person to remember, they are no longer just tools.
And the curve is going up. McKinsey’s latest The state of AI states that 88% of organizations surveyed now use AI in at least one business function, up from 55% two years earlier. Gartner predicts worldwide software spending to exceed $1.4 trillion by 2026. In an investor note distributed this year, Thoma Bravo says agent AI could create an estimated $3 trillion in app revenue by turning labor capital into usable software. That’s not a feature upgrade. It’s a system that reassembles itself on the fly, faster than most firms can manage, test, or take apart what they’ve already built.
That realization didn’t just come from technology. It also came from conservation.
Ecology has a name for what happens when you remove a load-bearing layer too quickly: the trophic cascade. The Aleutian fur trade nearly wiped out sea otters in the 18th century. Otters eat urchins. Urchins eat kelp. Remove the otters, and you don’t get an otter-shaped hole. You get an explosion of urchins, fallen kelp forests, and the loss of all the fish nursery the kelp was quietly holding.
That’s the pattern we should be looking at in AI-dependent infrastructure. AI will probably be better than your humans at testing, scoring, and predicting. The real problem is speed. We replace the people who were providing judgment, correction, and prevention, the connective tissue that did not appear in the workflow diagram. A voice in the gray areas, incomparable decisions. Remove that layer faster than the organization can figure out what it was holding, and you get the same cascade.
If we are serious about building a robust AI infrastructure, those patterns must be learned, and some of the lessons are uncomfortable.
Efficiency is limited. In technology, as in ecology, the best-designed system becomes brittle. Loose and wanton. So are fire extinguishers, and so is local autonomy.
In July 2024, a single CrowdStrike configuration update crashed 8.5 million machines worldwide. Airlines, hospitals, 911 centers, banks. $5.4 billion in losses. They returned a negative review in 78 minutes. Recovery took days. Southwest Airlines was not affected. It just wasn’t using the CrowdStrike software. Sometimes the absence of dependence is your fire break. If all the important functions in your stack depend on one model, one provider, or one training pipeline, you haven’t built intelligent miracles. Build the next blackout.
Ecosystems don’t just cascade. They also fail to climb. AI is entering workflows the way invasive species enter ecosystems: through subtle vectors, one use at a time. A flight attendant here, an abstraction layer there, a freelance editor somewhere no one is keeping track of. Each shipment is individually secured. The cumulative effect is something that no one chooses. The revisions and conflicts that keep previous processes honest are designed for human speed. Nothing has changed with the speed of the machine.
The model is not always the same as it was in the lab when it begins to shape the surface it eventually shapes. AI systems do the same when embedded in markets, media, institutions, and human behavior. You don’t control an ecosystem by controlling each organism. You control the conditions that determine whether the entire system recovers or collapses. Those conditions include visibility.
Systems that cannot be tested, studied, or independently evaluated are systems that no one can understand or govern well. Openness is important here, not as a slogan, but as a requirement for analysis and earned trust. The same concept applies to fault tolerance. Before a model is allowed inside critical systems, its operator must prove that the full facility can still operate without it. That means mandatory destructive testing, the way we test banks and bridges.
Builders don’t have to wait for administrators. When an AI layer enters the manufacturing workflow, manufacturers need to know what happens if the model is incorrect, the vendor is down, or the behavior changes after shipping. If the honest answer is “we don’t know,” the layer is not yet ready to carry the load. That’s true for the hospital screening system and the customer support bot. It’s especially true for open-scope agents: software that can program, drive tools, and execute within environments that no one fully controls. In those systems, the quality of the model is a simple question. What is difficult is who is responsible when they fail.
Multi-agent architectures and integration methods can improve resilience, but only if diversity is real. Three agents directing to the same base model may improve reasoning, but they are not three independent defenses. They are one person wearing three hats.
There is a broad strategic implication here. In stable ecosystems, dominant species accumulate their profits gradually. Slow the cycle of disruption and many of those gains are eroded before they mature. That is happening in business moats now. When distractions are greatly reduced, effective questioning ceases to be constructive and becomes inclusive when there is nothing around it. In real-world applications, the ‘best’ model loses the most flexible system.
Recovery is just as important as prevention. In the conservation work I do, the question is how can you never stop change. Disruption is inevitable. The question is what survives, how quickly the system recovers, and what hidden capabilities remain after the shock. We should ask the same of AI-dependent infrastructure. Not just “Is it safe?” but “How does it fail? Who can overwrite it? How far does the failure spread? What grows back after the mistake?”
The thing that breaks, in my experience, is assuming control. Real systems don’t crash cleanly and they don’t recover cleanly. Some parts fail. Others adapt. Some change into things no one intended.
Nature has been using distributed sensing, spatial feedback, and recovery for hundreds of millions of years. It uses the kind of network we’ve been trying to create. Not because the forests are conscious or because the planet is AI, but because the engineering problems are structurally the same: how does a system with no central control maintain coherence, adapt to damage, and continue throughout time?
The question is no longer just what AI systems can do. What kind of world they create around themselves, what kind of world they inherit from us, and whether we are smart enough to create programs that we can still direct.
If we take this seriously, several principles follow. Design diversity before efficiency. Build to recover before working out. Keep people informed, not as a measure of compliance but as stewards of the system, its source of judgment, and its memory of why it exists. Insist on openness, at all levels, as a precondition for trust at scale. None of this slows down the AI. It’s what keeps the AI running the day something fails.
You can turn off the machine.
You have to live within the ecosystem.
