I know that even A.I. cannot tackle an issue it has never been taught to understand or accurately advise a business it knows nothing about. When we met with the company, it didn’t ask us a single question — not about our data, our systems or how we worked. We eventually said no, but it took longer than it should have. Even a room full of people who knew better wanted its promises to be true.
In the meantime, we continued to run pilots — small test versions of tools inside the business before committing to it. I wanted them to work as much as the next person, but the results were the same almost every time. The demo looked like magic. But the tool wanted clean, connected data and decisions made in consistent, repeatable ways. Ours lived in a dozen systems that did not agree, layered with decades of exceptions and workarounds.
This is not just my story. When I compare notes with other chief information officers, the details change, but the ending rarely does. One runs an insurer, another an airline, another a manufacturer. Each tried tools that dazzled in the demo and stalled the moment they hit the mess of a real, decades-old business. Last year a report from M.I.T.’s Project NANDA put a number on it, finding that 95 percent of enterprise generative A.I. pilots never delivered real results.