AI doesn’t fix a broken process. It scales the breakage.

AI applied to a broken process doesn’t fix it. It executes it faster, at higher volume, with less friction to catch the errors. That’s not a metaphor. It’s how the thing works.

I keep seeing the same pattern. An organization has a process that doesn’t work well. Decisions take too long, handoffs are unclear, exceptions multiply, nobody can fully explain the rules. Someone proposes AI. The proposal gets approved. The process gets faster.

The problems get faster too.

What’s actually underneath

I’ve seen this play out more in automation projects than AI ones — I’ve run more of them — but the mechanism is identical.

A law firm wanted to automate client billing. The assumption was straightforward: all the data is in the timesheets, so we extract it, process it, issue the invoice. One field gave me pause — additional costs, entered as free text. The surface problem was parsing: unstructured input, inconsistent formatting. A technical fix.

But I observed the process over several billing cycles before we touched anything. What I found underneath: sometimes the currency wasn’t the billing currency. Sometimes costs were entered with VAT included, so the system would calculate VAT on top of VAT. Sometimes what looked like an additional cost was actually a separate line item — of counsel work that should have been billed differently.

None of that was visible from the outside. None of it would have been caught by automating the extraction. The process had been working — just barely, because experienced people were quietly absorbing the inconsistencies at each step. Automate it, and you don’t remove those inconsistencies. You remove the humans who were catching them.

In a manual process, a confused step produces a confused person who pauses, asks a question, maybe catches the error. Automated, that same step produces a wrong output at scale, consistently, until someone notices. The technology isn’t the problem. The sequence is.

The failure modes have names

This isn’t just a pattern I’ve observed. A 2024 RAND report, based on interviews with 65 experienced AI and machine-learning practitioners, found consistent patterns across AI project failures: the wrong business problem was chosen, the real objective was misunderstood, success metrics didn’t map to actual outcomes, workflow fit was poor, and deployment infrastructure was inadequate.

The healthcare implementation literature shows the same thing. A systematic review of 92 studies found that adoption barriers included data quality and bias, infrastructure limitations, workflow misalignment, inadequate training, financial constraints, and transparency and accountability issues — suggesting that implementation failures are rarely just about model performance.

Public-sector research from the European Commission’s Joint Research Centre is more direct. It distinguishes between initial adoption and operational implementation, and warns that moving beyond pilots into sustainable use creates a different set of organizational, cultural, governance, and trustworthiness challenges — because testing a system is not the same as embedding it into operational practice.

Across sectors, the consistent finding is that the causes of failure are upstream of the model. By the time you’re choosing which AI to use, most of the decisions that determine whether it will work have already been made — or avoided.

The sequence that actually works

There’s no single universally accepted framework, but the research converges on something close to the same order of operations.

First, define the problem precisely. Not “we want to use AI for claims processing.” What is the actual decision being made? Who makes it? What information does it depend on? What does a good outcome look like, and how would you know if you got one? RAND’s finding that teams consistently optimize for the wrong objective is a direct consequence of skipping this step.

Second, make the information usable. French firm-level research on AI adoption found that it was more likely where firms already had digital and organizational complements in place — data security systems, ICT-trained staff, and digital channels for collecting customer information. You cannot model what you haven’t measured.

Third, redesign the workflow. This is Michael Hammer’s original argument from 1990, and it still holds: technology investments disappoint when they mechanize old ways of working instead of redesigning them. His Ford example is the clearest version — the breakthrough in accounts payable wasn’t faster invoice matching. It was redesigning the process so invoice matching largely disappeared. Simplify before you automate.

Fourth, establish governance before scale. Clarify who is accountable for the output. Build in human override. Define what gets monitored and how often. Governance and human-in-the-loop controls should be established before deployment, not retrofitted after something goes wrong.

Only then does it make sense to choose the automation layer. Rules-based automation for stable, high-volume, low-variance tasks. AI where prediction, classification, language, or pattern recognition adds real value — and where the uncertainty that comes with probabilistic output can be governed.

RAND is clear on this too: many organizations choose AI where simpler tools would work better. That’s not a technology failure. It’s a problem-definition failure.

The nuance worth keeping

There’s a version of “fix the process before you apply AI” that goes too far, and it’s worth naming.

AI can sometimes help you see a broken process. That’s different from using it to run one.

A 2024 study by Brynjolfsson, Li, and Raymond looked at a generative AI assistant in customer support. Productivity went up 14% on average — and 34% for novice and low-skilled workers. The researchers’ interpretation: the system captured the tacit knowledge of the best performers and made it accessible to everyone else. AI helped codify and distribute good practice that previously lived only in a few people’s heads. It diagnosed and surfaced what the process should look like.

There’s also reasonable evidence that AI can help with messy documentation — clinical notes, requirements definitions, work artifacts — by giving people a structured starting point or flagging inconsistencies. In those cases, the process was unclear partly because no one had written it down well. AI provided enough structure to start.

So the more precise claim: AI can help reveal, diagnose, and codify a broken or implicit process. But AI used to execute or scale that process before it’s understood is much more likely to amplify the defects than to fix them.

That distinction matters for how you scope a use case. If you’re using AI to analyze your current process and surface what’s inconsistent — that’s a reasonable starting point. If you’re using AI to run the process at scale before you’ve answered the questions in the sequence above — you’re accelerating toward a problem, not away from one.

The institutional version of this

In a regulated environment, the stakes are asymmetric.

A slow, manual, slightly broken process produces localized failures. A person makes a wrong call. Someone catches it. It gets corrected. The exposure is bounded.

An automated or AI-assisted, slightly broken process produces consistent failures, at volume, with an audit trail that shows you knew what the system was doing. The Dutch childcare-benefits scandal is one cautionary case: an algorithmic risk-classification system was embedded in a harsh and legally fragile administrative regime, helping scale false accusations and discriminatory outcomes rather than correcting the underlying process. Australia’s Robodebt program is another — an automated data-matching system applied an income-averaging method that turned out to be unlawful, at scale. In both cases, the automation didn’t create the instability. It removed the human checkpoints that had been quietly absorbing the errors, and made the consequences impossible to contain.

Those are extreme examples. But the mechanism is the same at smaller scale. If your process has unclear decision rules, contested data definitions, or exception-handling that lives in someone’s head — AI makes those problems faster, broader, and harder to walk back.

What to do with this

The next time an AI initiative lands on your desk — as a sponsor, a sceptic, or the person accountable for delivery — run it through the sequence before the conversation gets to model selection.

Can you describe the process precisely, including who decides what and how you’d know if the output was wrong? Is the data clean and structured enough to model? Has the workflow been simplified, or are you automating complexity? Is governance in place before scale, not scheduled for after launch?

If the answer to any of those is no, you don’t have an AI problem yet. You have a process problem. That’s the one to solve first.