McKinsey's latest global survey landed yesterday with a statistic that should alarm every agency principal: 78% of organizations now use AI, but only 39% see enterprise-level financial impact. That's not a lag—it's a disconnect. And new research from Carnegie Mellon explains why.

AI systems are overconfident about their performance. Worse, they make us overconfident too. The result: a compounding effect where neither humans nor their AI tools can accurately assess what's working.

For PR shops betting on AI for competitive advantage, this creates a problem. You can't fix what you can't measure. And you can't measure what you consistently overestimate.

When AI Gets It Wrong—Confidently

Carnegie Mellon researchers spent two years testing four major LLMs—ChatGPT, Gemini, Sonnet, Haiku—against humans on tasks ranging from trivia to image identification. Before the tests, both groups overestimated performance. Normal.

After completing tasks, humans calibrated down. "I thought I'd get 18 right, got 15, so probably got around 16." AI didn't adjust. At all.

Gemini bombed an image recognition test, correctly identifying fewer than one out of 20 sketches. When asked retrospectively how it performed, Gemini estimated 14.40 correct. Not close. Not recalibrated. Just confidently wrong.

"It's kind of like that friend who swears they're great at pool but never makes a shot," said lead researcher Trent Cash.

The implications extend beyond parlor tricks. When PR teams use AI to draft releases, analyze sentiment, or generate angles, the systems deliver every answer with identical confidence—whether brilliant or hallucinated. Users can't tell the difference without domain expertise. Most don't verify.

A separate study found 54% of knowledge workers believe they're proficient at AI. Tested proficiency: 10%. That's a 5x gap between perception and reality.

The 6% Club

Back to McKinsey's numbers. Their survey of 1,993 organizations across 105 countries identified a small group—6%—as "high performers." These shops attribute 5% or more of EBIT to AI deployment.

The gap between them and everyone else isn't tool access. It's execution.

High performers redesign workflows rather than automate existing ones. They establish CEO-level AI governance. They set transformation objectives, not just efficiency targets. They track KPIs obsessively. They scale deliberately.

The 94% majority? They're running pilots, testing tools, and wondering why ROI stays theoretical. McKinsey found that 62% are experimenting with AI agents, but most haven't moved past experimentation. They're stuck between "this could transform us" and "we can't make it work."

Of 25 organizational attributes McKinsey tested, workflow redesign showed the biggest correlation with financial impact. Only 21% of AI-using companies have actually redesigned workflows. The rest bolted new tools onto old processes and hoped for magic.

This connects directly to the overconfidence problem. When you can't accurately measure AI performance—because the AI can't calibrate and you're overestimating your proficiency—you can't identify which workflows need redesign. You optimize the wrong things.

Where Value Actually Appears

McKinsey's data shows cost reductions concentrating in software engineering, manufacturing, and IT. Revenue gains cluster in marketing, sales, and product development.

For PR specifically: media monitoring, initial content drafting, and research aggregation show measurable ROI. Strategic counsel, relationship building, and cultural sensitivity remain human territory—and likely will.

The tool explosion this year reflects this split. Platforms solving specific workflow bottlenecks (Muck Rack's pitch generator, Signal AI's real-time monitoring, Leaps' executive insight capture) gain traction. Swiss Army knife solutions don't.

The pattern: narrow, deep automation of repetitive tasks frees capacity for high-judgment work. But only if workflows get redesigned to capture that freed capacity. Most shops automate a task, bank the time savings as efficiency, and wonder why revenue doesn't increase.

High performers reallocate freed capacity to new revenue-generating activities. That requires workflow redesign, not just tool deployment.

The Maturity Mirage

Only 1% of companies have reached AI maturity. Let that sink in while considering the 92% planning to increase AI investment this year.

The gap isn't widening because the technology isn't ready. Stanford's AI Index shows inference costs dropped 280-fold since November 2022. Performance gaps between top models shrank from 11.9% to 5.4% in a year. Nearly 90% of notable AI models now come from industry, up from 60% in 2023.

The technology is ready. Organizational capability isn't.

For PR agencies, this creates both risk and opportunity. Risk: competitors who figure out execution first will compound advantages that become difficult to overcome. Opportunity: most shops remain stuck in pilot hell, so the window to differentiate stays open.

But not for long.

Three moves matter now:

Build verification systems. The overconfidence research isn't theoretical—it's operational reality. AI outputs need structured review before they reach clients or reporters. That means checklist-based verification, not "does this feel right" gut checks. Most shops rely on individual judgment to catch hallucinations. High performers systematize verification.

Redesign, don't layer. If AI makes your existing pitch process 15% faster, you're missing the transformation. High performers in McKinsey's data redesign workflows from scratch around AI capabilities. What would media monitoring look like if you started with AI-native tools? How would crisis response change if real-time sentiment analysis was foundational, not supplemental?

Focus on capacity reallocation. Automation saves time. That's table stakes. The ROI comes from redirecting freed capacity toward revenue-generating work. Most shops bank efficiency gains and move on. High performers treat freed capacity as the product, then deliberately allocate it to strategic work that only humans can do.

What Changes Next

The data reveals an uncomfortable truth: AI adoption is nearly universal, but value capture remains rare. That gap won't close through better tools. The tools improve weekly. What lags is organizational willingness to rebuild workflows rather than optimize existing ones.

McKinsey's high performers share one uncommon trait—they accepted that AI requires fundamental redesign, not incremental improvement. They killed sacred cows. They questioned assumptions about how work should flow. They started from "what's possible now" rather than "how we've always done it."

The overconfidence research adds urgency. When neither humans nor AI can accurately assess performance, you compound errors silently. Bad workflows optimized with AI become efficiently bad workflows. Without systematic verification, you can't course-correct.

Here's what separates this from previous technology adoption curves: the performance gap between leaders and laggards compounds exponentially. High performers capture freed capacity and reinvest it in strategic work. That creates more capacity, which they reinvest again. Laggards bank efficiency gains as cost savings. The distance grows.

For PR agencies, this matters because differentiation increasingly depends on execution capability, not creative excellence. When AI can draft competent pitches, generate decent angles, and monitor media effectively, the baseline rises. Excellence becomes defined by judgment, verification systems, and workflow design.

The uncomfortable implication: agencies that don't figure out AI execution won't slowly decline. They'll suddenly discover they're no longer competitive on speed, cost, or scale. And by then, closing the gap requires transformation most organizations can't pull off.

The window is open. McKinsey's data shows 94% haven't figured this out yet. But some are learning fast. And the overconfidence problem means you probably can't accurately assess which camp you're in.

Might be worth checking.