CIOs increasingly dump in-house POCs for commercial AI

After thousands of AI proof-of-concept projects have died on the vine, many organizations are scaling back internal efforts in favor of adopting commercial, off-the-shelf AI tools.

About half of companies surveyed by Gartner in late 2023 were developing their own AI tools, but the number fell to about 20% at the end of 2024, says John-David Lovelock, a vice president and analyst at Gartner.

Many organizations are still running a few POCs, but cooling hype surrounding generative AI has many CIOs turning to vendors, whether they be large language model (LLM) providers or traditional software sellers with AI built into their products, Lovelock says.

Many ambitious AI projects started in 2024 or earlier now face internal scrutiny because of the high POC failure rates, Lovelock adds. Recent research by IDC found that 88% of POCs didn’t make it into widescale deployment, while CIOs struggle to define POC success.

“Last year, when CIOs were doing proof of concept work and getting lots of help from service providers and internal resources, there was a fairly high failure rate among companies that already had a good pedigree in artificial intelligence,” Lovelock says. “But for companies that didn’t have that same pedigree, their failure rates were over 50% higher.”

Not enough brain power

The “vast majority” of customers at cloud- and automation-focused Asperitas Consulting are now using off-the-shelf AI tools, says Scott Wheeler, the company’s cloud practice lead. Many Asperitas customers are in the financial services and insurance sectors, often seen as potential beneficiaries of gen AI functionality.

Global tariffs shake up CIOs’ IT agendas

CIOs have long been adept at the art of the pivot — a skill that will serve them well as they cope with the fallout of global tariffs that have sent organizations into a tailspin over the past few weeks.

Already, IT is feeling the impact on infrastructure and supply chains, and CIOs are decreasing capital expenditures and scaling back projects or delaying them altogether.

The new tariffs will not only drive up tech prices but also disrupt supply chains and weaken global IT spending this year, IDC warned in a blog post.

“The situation remains highly fluid and dynamic,” IDC wrote. “A weakening economy will lead to IT spending cuts and delays in the next six months.” The research firm is projecting “a move closer to the previous downside of 5% growth, which reflects a rapid, negative impact on hardware and IT services spending.”

Even though some equipment vendors may try to mitigate the impact, US customers will quickly feel the effect of higher prices, IDC noted. “Lean inventories and rapid manufacturing cycles mean that price hikes will materialize quickly. The broad, unfocused nature of these new tariffs leaves manufacturers little room to adjust.”  

Rising IT infrastructure prices “could balloon budgets and force CIOs to delay or prioritize the most important projects,” says Mark Moccia, a vice president and research director at Forrester. As a result, CIOs and other tech leaders need to “proactively analyze costs, diversify sourcing, optimize inventory, and prioritize the projects that don’t sacrifice critical AI ambitions.”

Bank of America’s big bet on AI started small

Bank of America will invest $4 billion in AI and related technology innovations this year, but the financial services giant’s 7-year-old homemade AI agent, Erica, remains a key ROI generator, linchpin for customer and employee experience, and source of great pride today.

Few used the term agent, let alone agentic AI, in 2018, but the bank built a team of software engineers, linguistic specialists, and banking experts to create the small language model, which has been tuned over the years using customer feedback data from the call center.

Hari Gopalkrishnan, head of consumer, business, and wealth management technology at BofA, says the key to Erica’s success and longevity has been its small size.

“We are not writing essays with Erica. We are not trying to write software. We’re trying to understand short bursts of information a customer asks us because they don’t want to hunt and peck on a menu on a screen that’s got 50 different things, and basically be able to translate, what does the customer really mean when they say, ‘I want to pay a bill’?” Gopalkrishnan says. “How do we understand the short burst of conversation that the customer would want?

“We trained the model to do just that,” he says about Erica, which is built on open-source models. “Over time, it went from a model that was 80% accurate to 85% to well in the north of 90% accurate. It also let us be more predictive in what the model would do.” 

Gopalkrishnan says the Bank of America was able to tune Erica during the pandemic to enable customers to apply for PPP loans and handle a multitude of business and consumer needs. He will embrace generative AI and agentic AI offerings as they evolve but believes that most of the bank’s customers requirements can be built in house.

Explore our Latest Stories

Political engineering 101: The biz-savvy IT leader’s survival guide

Sometimes, even the soundest engineering isn’t sound-enough engineering. So when things go sideways, politics can be a shrewd tool or your undoing.

Agents are here — but can you see what they're doing?

Agentic AI is poised to transform business processes at a scale and pace never seen before. But are enterprises prepared to deal with the risks agentic systems pose?

5 tips for transforming company data into new revenue streams

Enterprise data can be a hidden treasure trove. Here’s how to safely monetize it by zeroing in on actionable value and guarding against common risks.

How to plan for a new business technology operating model

As organizations reinvent for the AI era, business leaders say they’re moving toward a new enterprise technology operating model. Here’s how to get started.