What they described instead, during a lunch roundtable hosted by Thomson Reuters, was something closer to organized chaos.

“The biggest challenge I could see is speed without clarity,” said Venkatesh Alagirisamy, EVP and COO of Nike. “I see a lot of hype around AI that drives a lot of energy within organizations in wanting to adopt AI, but without that clarity, without that sense of purpose, that speed could get us in the wrong direction.”

Welcome to what panelists called the “automation illusion” — the dangerous gap between what AI promises operations leaders and what it actually delivers.

The promise was simple

The way the COOs described it to Fortune Editorial Director Diane Brady the AI pitch was almost too good. Automate the routine. Free up the workforce. Let the machines handle forecasting, logistics, compliance, customer service. Let humans handle strategy.

Aayush Bhatnagar, global head of customer service at Sysco — which moves food to restaurants across North America, generating nearly $84 billion in annual revenue — put it plainly: the goal was to take tribal knowledge baked into decades of human relationships and institutionalize it at scale. “Every piece of broccoli you’re eating has moved an average of 2,000 miles,” he said. The supply chain that makes that happen runs on judgment calls made by people who’ve been doing it for years. AI was supposed to absorb that expertise and multiply it.

And in some ways, it has. Nike launched an internal learning platform 12 months ago — peer-curated, bottoms-up, not mandated from above — and logged 20,000 digital courses taken, with 3,000 live training sessions conducted. Sysco is using AI to rethink how it forecasts and buys. Thomson Reuters is deploying it to help lawyers, tax accountants, and trade professionals work faster.

But this has all come with a big reality check.

The illusion kicks in

Laura Clayton McDonnell, president of corporates at Thomson Reuters, expanded on the automation illusion. “We’re going to move fast, we’re going to get these answers really quickly,” she said. “But what about making sure that output is reliable, it’s accurate, it’s something that you can drive your business on?” That, she added, is where companies really need to pause instead of give in to the need for speed.

For the professionals Thomson Reuters serves — lawyers walking into courtrooms, accountants navigating tariffs, trade teams dealing with sanctions — there is no margin for error. “You cannot be wrong,” McDonnell said. “You just can’t be wrong.” A large language model that confidently produces a plausible-but-wrong answer isn’t a productivity tool in that context, but a liability.

The illusion runs deeper than accuracy, though. The bigger problem is that AI has made the operating environment fundamentally less predictable — precisely the environment COOs are paid to manage.

Olivia Nottebohm, COO of Box, said she has watched it play out inside her own company. Box sells AI products. It runs Box AI internally. It talks about AI constantly. And when Nottebohm looked at the adoption numbers, they were low. “Here we are, an AI company selling AI,” she said, “and I wasn’t even seeing the adoption I was expecting.” When she dug in, she found the answer wasn’t resistance — it was confusion. People didn’t know how. The tools were available. The skills weren’t.

She shared that the company impemented a program called “No Boxer Left Behind.” It worked, but it also revealed a harder truth: even at a tech-forward company, the gap between deploying AI and operationalizing it is enormous. “Really making sure … that people don’t feel disenfranchised, I think that has been the thing that took me the longest to figure out,” she shared, adding that she “should have figured it out sooner.” The company’s mandatory trainings are clear about what Boxers have to learn, “and if you choose to opt out of being on the AI transformation, that’s up to you. But we as an employer are not going to leave you behind.”

The management problem no one has solved

Nothing illustrated that gap more starkly than Bhatnagar’s admission about his team. Four weeks ago, he told the room, he added seven AI agents to his direct reports. They have names. They have defined roles — an escalation agent, a delivery agent, a communications agent. Their performance is reviewed alongside the humans at his weekly business review.

“I lost some sleep that night,” he said, “thinking that our traditional laws of leadership, principles of leadership, do not apply to these agentic agents.” To his point, there is no management literature for that, no HR policy or performance improvement plan you can put an agent on. And yet COOs like him are already accountable for their output — output that can scale instantly and go wrong just as fast.

“How do I train my managers now?” he asked the room. It may have been the most honest summary of where enterprise AI actually stands.

The deeper stakes

Near the end of the discussion, the question hanging over the room became explicit: what happens to the entry-level workers who traditionally built their judgment doing the exact tasks AI is now absorbing?

McDonnell kept returning to the same guardrail: the human in the loop isn’t optional, it’s structural. “I don’t think we’ve found a tool yet that actually can exercise business judgment,” she said. “That’s the difference maker.”

Alagirisamy framed it as the central leadership capability of the moment: learning agility. Not AI fluency, not technical depth, but the organizational muscle to keep adapting as the ground keeps shifting. “Does your team have the learning agility to adapt to this new environment?” he said.

The same week as the Summit, The New York Times reported that Box had created 13 new job categories — AI architects, AI process designers, forward deployed engineers — that didn’t exist two years ago. CEO Aaron Levie pushed back publicly on the “AI kills jobs” narrative, arguing the technology is unlocking work that was previously unaffordable. Nottebohm was describing both things at once in that Scottsdale conference room: the dysfunction of under-adoption and the emergence of roles no one had a résumé for yet.

For COOs, the automation illusion isn’t just about bad AI outputs. It’s about the widening gap between the speed at which the technology is moving and illusion of how much work can be automated, and the reality that it’s much easier said than done.

They came in talking about what AI was doing for them. They left still trying to figure out what to do about it.

For this story, Fortune journalists used generative AI as a research tool. An editor verified the accuracy of the information before publishing. The article has been updated to clarify that Box COO Olivia Nottebohm said her company is committed to not leaving employees behind.