·
5 minute read
Most generative AI pilots don't show a positive return on investment yet — a stat raised in a recent episode of ZoomRx's Decode podcast, citing an MIT Sloan report and describing the failure rate as "either 90 or 95%" across enterprise GenAI initiatives. For pharma insights leaders deciding how much budget and headcount to commit to AI, that number is either a reason to pump the brakes or a reason to ask a sharper question: why are most pilots failing, and is pharma insights different?
John Shortell, a market research and insights leader with 30+ years of experience in pharma, including 15 years focused on the specialty medicines market and specifically oncology, addressed that stat directly on the podcast. His answer isn't dismissive of the number — it's a reframe of what it actually means, paired with a clear-eyed read on where the real risk sits for pharma insights teams specifically.
The MIT Sloan Stat Isn't Surprising - It's Early
Asked whether the 90–95% failure rate surprised him, Shortell's response was immediate:
"Probably not, because we're so early in this cycle... it's under three years ago when this sort of exploded. So that probably means that most of these pilots maybe started 18 months ago, even if we're being generous. If this was any other product — and we use the pharma example — you're not looking for a positive ROI until years 3 to 5. So we're still super early there."
That comparison matters because it's the exact framework pharma insights professionals already use to evaluate their own industry: no one expects a newly launched drug to hit peak ROI in its first 18 months. Judging enterprise AI by the same standard this early in its adoption curve, Shortell argues, is applying an unreasonable timeline, not identifying a genuine failure.
Where Pharma Insights Teams Actually Sit Today
Rather than treat "AI adoption" as one undifferentiated bucket, Shortell broke down how he sees the top 100 biopharma companies splitting their approach to GenAI in insights functions, using three categories:
- Pure cost-cutting (~5%) — Companies treating GenAI primarily as a budget-saving tool, with ROI expectations already attached.
- Active experimentation without premature optimization (~75%) — The clear majority: companies embracing AI, running many experiments, but deliberately not yet burdening early pilots with hard ROI targets.
- Effectiveness-focused, not just cost-focused (~20%) — Companies already treating AI as a lever for better insights and decision-making, not only cheaper ones.
That distribution is itself the answer to the MIT Sloan number: if 75% of the market is intentionally in an experimentation phase rather than an ROI-proving phase, a high near-term "failure" rate isn't evidence AI doesn't work — it's evidence that most pilots were never designed to prove ROI yet in the first place.
The Real Risk: Duplicated Effort Across Functional Silos
If the ROI framing isn't the biggest threat to pharma AI initiatives, what is? Shortell points to organizational structure — specifically, the tendency for every function to build its own AI agents independently.
"Every function is developing their own agents, which is great for early adoption... but I don't see that viable in the long term because there is so much duplication of effort. If I'm building an AI agent to look at competitor A using my data sets, and my medical counterpart is also building their AI agent for the same competitor A using the medical data sets — isn't it obvious that we should probably be collaborating somehow?"
This is a direct application of Conway's Law to AI adoption: an organization's AI systems will mirror its org chart, redundancies included, unless someone actively designs against it. Shortell's expectation is that pharma will eventually consolidate toward a federated model — centralized oversight with departments retaining a high degree of autonomy, rather than either a fully siloed, function-by-function approach or a single top-down mandate.
Interestingly, Shortell himself views a Chief AI Officer hire as a positive signal, not a premature one - "I would think that's good news... I think it's a positive step" — provided it comes paired with a federated structure rather than pure top-down control. His own read on the current mandate is simpler: get people comfortable with the tools first. It was the podcast's host, not Shortell, who pushed back on timing specifically, suggesting such a hire today might be "24 months too early" for most organizations — a reminder that even AI-forward pharma leaders don't fully agree on when centralized AI governance should arrive.
Related reading: How AI Is Transforming Market Research in Pharma
What Happens If the Failure Rate Is Still High in 18 Months?
Pushed on what would cause pharma-specific GenAI pilots to still be failing at a similar rate in another 18 months, Shortell's hypothesis wasn't about the technology, it was about the operator:
"It could be that the users need more technical expertise to take it to that next level... I used to start writing prompts with like one and two sentences. Then I started talking to other folks and I'm going, well, our prompts are like 5 pages long. I'm going, what — you can do that?"
His point: the gap between an AI pilot that stalls and one that compounds in value often isn't the underlying model, it's the sophistication of the prompting, the quality of the data feeding the agent, and how much context the user is willing to provide. Teams that treat AI adoption as a skill to be developed, not a tool to be switched on, are the ones most likely to beat the odds MIT Sloan is describing.
Two Use Cases Already Beating the Odds
Rather than staying purely theoretical, Shortell pointed to two applications already delivering measurable value inside his own organization, evidence that the 90–95% failure rate isn't universal within pharma insights specifically:
- Early message pre-testing. AI agents built on real patient and caregiver conversation data let brand teams pressure-test draft messaging before a single dollar goes into formal creative testing — trimming the number of costly agency rounds needed before research begins.
- Competitive intelligence and conference summarization. AI-powered synthesis of entire conferences and secondary literature has cut both the cost and the turnaround time of tracking competitor activity, compared to manual review.
Both use cases share a common thread: they don't ask AI to replace primary research or human judgment. They ask it to compress the time and cost of getting to a well-informed starting point — which is exactly the kind of narrowly scoped, clearly bounded use case that tends to show ROI fastest.
Watch the Full Conversation
Decode: AI for Life Sciences and Healthcare is ZoomRx's ongoing podcast series exploring how artificial intelligence is reshaping market research, insights, and commercial strategy across pharma and biotech.
ZoomRx is a life sciences consultancy serving 100+ biopharma companies across the globe, helping brand, insights, and commercial teams turn primary research, competitive intelligence, and AI-powered analytics into decisions that move the needle.
Frequently Asked Questions
