The debate about AI adoption is pretty much over. The most pressing question is how to ensure that AI management is aligned with AI adoption, but without reducing time to value. This is true for any application, but especially for critical ones. Knowing that, ‘We will probably be able to stop the train in time,’ is not enough. Yet the truth is that while the supporting technology is there (or getting there), the cultural attitudes and practices that support better AI governance are not.
For example, our survey of 820 IT professionals worldwide found that while 77% trust the results of AI, only 39% have fully automated test processes. If AI is to grow safely, securely, and compliantly, that gap between adoption and governance must be closed. Perhaps easier said than done, but better monitoring and control of AI should be a non-negotiable priority.
Change must start at the organizational level. In my experience, many businesses still treat management as something that happens outside of the software delivery process. Policies are defined, audits are conducted, and compliance reviews occur after work is completed. They are already lagging behind, but with autonomous agent AI, we need a mindset shift where monitoring and controlling AI are engineering skills built right into the SDLC. Authentication, policy enforcement, access controls, genealogy tracking and compliance testing should work alongside development activities, rather than after.
AI Governance Must Enter the Delivery Pipeline
While a developer might say, “Well, we used AI, and it works”, governance asks, “How do you know”? Someone might say, “Let’s take out independent agents”, and the leadership would ask, “And what happens if one of those agents makes a set of bad decisions?” Businesses need to be able to understand who or what made a particular decision? What data influenced the result? What policies were in place at the time? Can we reconstruct the thought process when something goes wrong?
This is why traceability is important. Governance depends on keeping a clear record of AI-generated code, automated actions, data usage, and decision processes. Having that in place means that teams can understand how results were produced, when problems arise, how the problem happened in the first place, and repeat the same situation with everything involved.
Organizations also need to be defined. If tracing shows what happened, then explanation shows why it happened. Here is an example. An AI agent diagnoses a performance problem, makes a code change, runs tests, updates documentation, and prepares for deployment. The trace will capture the execution alert, the generated code changes, the test execution, the deployment request, and the permissions applied.
The explanation will include why the agent determined there was a problem with the shipment, what evidence they used, why they chose that particular fix, and why they believed the fix was safe.
Next, we need to be accountable. Traceability and definition are only important if someone is paying attention. Organizations still need people who can interpret evidence, challenge decisions, and take responsibility when things go wrong. This not only means changing the focus of engineering from implementation to oversight, but to having deep engineering knowledge and the knowledge to interpret the situation accurately. In the age of AI, great engineers are more important than ever.
Human Supervision Must Be Measurable
That said, human supervision cannot risk becoming another burden on the already overburdened shoulders of developers when they need to search across multiple systems. In addition, human resource management also needs to be measured. And governance cannot be another hurdle in an SLDC environment that is already riddled with bottlenecks that limit production.
This is why middle layers of access and control are emerging as one way to address this need, creating a single point where AI interactions can be monitored (such as which MCPs are used), governed, restricted (for example, only a select list of safe MCPs can be used), and tested. Essentially, this helps organizations maintain oversight of AI work without requiring developers to become full-time compliance officers, or productivity cuts.
Governance should also be viewed as a cross-functional responsibility, with all engineering, security, operations, and compliance teams working with a common understanding of risk, accountability, and oversight. This more collaborative approach is a fundamental principle of sound DevOps practice, as well as governance. This raises the point that if done well, DevOps can make a tangible contribution to better governance, according to an internal survey: 70% of 820 IT professionals believe that a mature DevOps adoption contributes to successful AI adoption. The same disciplines that underpin mature DevOps, such as automation, testing, traceability, auditability, and shared ownership, also provide the foundation for effective AI governance. So, going back to the basics, reviewing and improving the basics of DevOps is a good place to start.
While choosing the right tools makes a big difference, DevOps has never been a tool challenge, nor is it an AI management challenge. Many management problems stem from organizational issues rather than technical limitations. Overcome those issues, address traceability, explainability, accountability, and control, enforce governance throughout the SDLC, and we’re in a better position to start trusting the use of AI at scale. As someone who has been involved in AI for over a quarter century as a CTO, I remain one of its biggest advocates, but it’s just a tool, and a very complex one at that. So now it’s time to put the control brakes in place so we can have more confidence in our ability to drive that train safely, but without slowing down the SDLC.
