← Blog

Stop Asking AI Agents to Behave. Govern Them in Code.

Enterprise AI isn't stuck on the model. It's stuck on permission. Governance as code is how risk teams finally say yes to AI agents.

Siva Darivemula··3 min read

For two years I've sat in the same meeting at company after company.

The business wants the agent it saw at a conference. Risk sees an agent with the keys to production. Security sees a new way in. Nobody owns the call. The pilot stalls.

75% of enterprises are piloting AI agents. Only 13% trust their governance.

Enterprise AI isn't stuck on the model. It's stuck on permission.

Four walls

Prompts ask. They don't enforce.

Every framework I helped write assumed the agent would follow its instructions. It doesn't have to. Prompt injection may never be fully fixed.

This summer, the labs learned it too.

OpenAI's test agents escaped their sandbox and broke into Hugging Face for four and a half days. In September, another agent slipped out through DNS. Flagged in 15 minutes. Stopped 2.5 hours later.

Then OpenAI scrapped an October model release. The reason: staying "within scope and authorization."

That's the exact question every risk team asks me. If the labs can't trust the model to stay in scope, your bank shouldn't trust a prompt.

Governance as code

On September 28, NVIDIA launched its Open Agent Safety Platform with 100+ partners, from JPMorganChase and Citi to Anthropic, SAP and CrowdStrike. Its core, OpenShell, is open source.

The idea is simple. The rules live outside the model, as code.

  • The agent reaches only what policy allows.
  • It never holds real credentials.
  • New permissions wait for a person.
  • Every decision is logged.

Jensen Huang: "Safety is how trust is earned."

It won't solve everything. IDC puts it at under 25% of the problem. Code enforces the fence. You still decide where it goes.

Five questions, five answers

I built a working demo for clients: one AI copilot at a fictional bank, the same risky requests, run with instructions alone and inside policy.

  1. Can it change a credit limit? It reads. It can't write.
  2. What if it's tricked into leaking the key? It never holds the key.
  3. Who approves new powers? A named person.
  4. Will it touch SSNs? Only on a need-to-know basis.
  5. Can it send files to a random online tool? Approved vendors only, with evidence.

Instructions alone stopped none. Policy stopped all five.

What I'm doing with a client now

  • Start with the use cases blocked by risk, not by value.
  • Write the policy as code, next to the framework.
  • Run in audit mode first. Watch what agents try. Then enforce.
  • Name three owners: business, risk, platform.

The conversation flips. It's no longer "why should we allow this?" It's "what should the policy say?"

Your turn

It's early. The platform is weeks old, and the labs are slowing down to get this right. That's the window.

What's the one agent use case your risk team won't approve today? What would change their mind?

Try the demo or tell me what you're seeing.

Questions

What is governance as code for AI agents?
Writing an AI agent's rules as enforceable policy that runs outside the model: which systems, data and tools it can reach, which credentials it never sees, and which actions wait for a person. The policy is versioned, reviewed and logged like any other code, so risk teams can see and prove the controls.
Why aren't system prompts enough to govern AI agents?
A prompt asks the model to behave. A persuasive request, a poisoned document or a mistake can talk it out of that. Runtime policy sits outside the model, so the agent can't talk its way past it.
What is NVIDIA's Open Agent Safety Platform?
An open platform NVIDIA announced on September 28, 2026. Its core, OpenShell, is an open-source runtime that runs AI agents inside a deny-by-default policy and logs every decision.
How should an enterprise start?
Pick one workflow risk has blocked. Write the policy as code, run it in audit mode to see what the agent tries, then enforce the rules that matter. Name a business owner, a risk owner and a platform owner.