When AI Outruns Oversight

Iterate.ai’s board of directors is very small—just the two co-founders who understand all the technical ins and outs of AI. But the San Jose, California-based company also leans on its advisory board members from different segments of the economy to provide additional insight and guidance, including on ethical AI governance.

Jon Nordmark, co-founder and CEO, who operates out of the company’s Centennial, Colorado office, shares the perspective of both boards and how they interact with each other.

What are some of the most pressing topics your board is tackling these days?

AI is moving faster than most governance frameworks, and our board has to stay ahead of that. Our mandate centers on the risks that tend to get overlooked. Things like shared-cloud infrastructure, long-term AI memory and the data exposure that happens when employees assume their tools are private when they are not. Whenseventy thousand ChatGPT conversations showed up indexed in Google search results, that was not a technical failure. It was a governance failure.

We treat AI as both a strategic and a fiduciary topic. On the fiduciary side, the work is understanding how our data moves, where it sits and which policies actually govern its use. We study our vendors’ incentives and build control models that favor resilience over convenience, and we expect our vendors to apply the same scrutiny to us. From the board seat, the goal is to make sure speed never comes at the cost of customer trust or the company’s longer-term position. Those things tend to be more connected than boards want to admit.

There’s also a cost exposure most boards haven’t begun to account for. Agentic AI systems don’t consume a few thousand tokens the way a chatbot does. An autonomous agent handling compliance review or software testing reads context repeatedly, calls tools in loops and runs long workflows. Token consumption at that scale looks nothing like the pilot projections. Boards that never asked about token pricing now have agents running in production, and the economics are catching people off guard.

How are you mitigating some of your company’s toughest challenges?

One of the toughest challenges right now is hiding right underneath the surface. AI agents are making decisions while storing unprecedented amounts of information. And they act with persistence. They tend not to forget and they do not ask permission and that creates a new class of operational risk if you do not or cannot control the environment they’re running in.

My co-founder and I are the board. We are also management. At a company our size, those roles sit in the same two people, and that’s a dynamic that requires deliberate discipline. When we put on the governance hat, the job is to set a standard that applies to all managers, and that includes us. The board seat can’t be seen as a perch above the problem. It is a commitment to ask harder questions than the default management mode tends to encourage, including of ourselves.

The layer we’re most directly focused on is the one below us, meaning the operations leads, product managers, engineers and team leads who are actually making AI adoption calls day to day. They’re choosing vendors and deciding what data gets fed into which system. They’re also spinning up new AI touchpoints without necessarily thinking through where their queries go or how long they get retained. The governance standard we apply is explicitly about holding them to a bar that doesn’t just default to “whatever is fastest.”

The reason this matters is that procurement-level decisions carry fiduciary consequences that most people making them do not (yet) realize. When an employee uploads a contract draft or feeds customer data into a tool that processes it across shared public infrastructure, they are not thinking about data retention policies, vendor terms or the exposure that quietly compounds over six months. That is not a criticism of them, because it’s a structural gap that governance exists to close. By the time most boards even see the AI architecture diagram and workflows, they have already been built and is already creating the exposure.

We also maintain a clear distinction between convenience and control. Convenience can be tempting. It very easy to start a conversation with an LLM that processes data across public GPU farms or to use a shared inference service without questioning what it retains. But that convenience usually comes at the cost of privacy, and most AI governance failures will trace back to exactly that trade-off. Our board’s job, as it should be for other boards regardless of size, is to make sure the people below us are being asked that question explicitly, before the architecture makes the choice for them.

What is new about how you are recruiting board members?

My prior startup had a two-person board, then five, then eight. It included representatives from well-known Silicon Valley venture firms, independent board members and founders, so I’ve seen boards operate at a few different levels.

Ours is small by design. Just my co-founder and myself, with a healthy set of board advisors I’ll get to in a second. Governance is critical, but we don’t want bureaucracy slowing us down. Two lawyers provide oversight and guidance, and we run our finances through accounting firms. The reason we’ve stayed lean is that we don’t want strategic interference, and we don’t want to spend board time teaching people about AI. AI is doubling its capabilities every three months now, up from every seven months over the past five years, according toMETRresearch. That pace dwarfs Moore’s Law, which said chips would double capabilities every 18 months. To compete in that environment, the board has to move fast.

What is new is building out a formalized advisory board. Rather than prioritizing financial representatives, we’re focused on operators with real business experience in the domains that matter for the next decade. This is why we added board advisors likeCathy Halligan,Elaine Boltz,Frank Kollmar andTed Shelton. They each bring a different dimension that matters for the next decade, including retail and consumer insight, global operational scale, generative AI strategy and ethical AI governance. We view these as required capabilities for responsible board oversight, beyond just nice-to-have specialties.

These are people who can help sharpen our focus on private AI and the products we build for secure private environments. We recruit for fluency in AI governance, cybersecurity, regulated environments and digital transformation, because those are the domains where the risk has concentrated.

I’ve served on corporate boards for more than 20 years. Boards that don’t adapt to this AI reality will fall behind the companies they’re supposed to govern. And it isn’t just AI. Quantum computing will add another layer of exposure once quantum machines reach the threshold where they can break current encryption, and the credible forecasts put that window between 2028 and 2032. Architecture decisions being made right now will determine how exposed the company is when that day arrives. Most boards aren’t being asked about it yet, but the timeline is shorter than it looks.

How does your board keep up with the opportunities and the risks of emerging technologies?

Staying current is a governance obligation now. The technology isn’t going to slow down to let anyone catch up, and curiosity has to be built into how the board operates, not treated as a bonus.

Most boards are behind. Surveys showonly about 30 percent of directors believe they’re prepared for modern AI oversight, and nearly 40 percent report no AI training at all. That gap has consequences, even if they haven’t fully surfaced yet. A board that doesn’t understand what it’s approving can’t ask the right questions, and the right questions are the whole job.

We close that gap by staying in direct contact with operators, researchers and people working at the policy level. We look at real incidents, not just scenarios. We evaluate infrastructure exposure, memory retention and regulatory obligations together, not in isolation. We also participate in cross-industry forums, including one we created here in Colorado called IterateOn, because an idea from healthcare or aerospace sometimes reveals a pattern that applies directly to what we’re doing. That cross-pollination makes oversight sharper and more forward-looking.

On the policy side, Colorado’s legislators and governor appointed me to the state’s AI Task Force, which has been actively refining SB 205, the nation’s first sweeping AI bill. That’s put me in the middle of real debates about AI bias and the tension between consumer protection and legislation that could stifle the startup community. That context shapes how we think about risk at the board level in ways that purely internal deliberation wouldn’t.

What are your strategies to ensure the company remains resilient?

You cannot be resilient if you don’t know where your data lives. And you can’t know that if your critical systems sit on shared infrastructure you don’t control, or if your AI stack is retaining memory you cannot audit or erase.

Our board prioritizes architectures that give the company control rather than dependence. We want private AI environments, on-prem and offline options, model portability, and runtime control that lets us swap or isolate models when the market shifts or a vendor changes terms. We design specifically to avoid single-vendor lock-in.

Token pricing is a version of this problem that doesn’t get enough board attention. The shift to agentic AI changes the consumption curve entirely. A single autonomous agent doing a long-running task can generate millions of tokens through recursive loops and tool calls. Multiply that across an enterprise deploying dozens of agents and the cost exposure is significant, and volatile, because you’re subject to whatever a third-party provider decides to charge. A private model running on infrastructure you control has no token meter. The economics are fixed and predictable. That is exactly the kind of leverage our board prioritizes, and it’s part of why the private AI architecture that protects your data also protects your budget.

The questions we push management to answer are whether the company can survive a regulatory pivot, a major vendor outage or a broad industry-level AI incident. None of those are hypotheticals anymore. Resilience in this environment is an architecture decision. It gets built through privacy-first design, genuine transparency around data retention and the operational capability to run workloads outside the cloud when needed. Our job is to build that in now. The board that waits for a headline to start asking these questions will find the architecture has already made the choices for them.