Why Most AI ROI Claims Collapse Under Audit

The pattern is identical across every failed model I have reviewed. Someone ships an AI feature. A few months later leadership asks for the ROI. An analyst pulls a number together: "AI saved 3,200 hours this quarter." Finance asks four questions, the model breaks, and the project gets shelved.

The four questions are always the same. What was the baseline. How do you know the AI caused the change and not seasonality, a launch, or a process tweak. What is the full cost, not just the API bill. How long until cumulative cash flow turns positive. If you cannot answer those four in writing, with sources, you do not have ROI. You have a story.

• No baseline: the manual workflow was never measured before AI replaced it, so any "savings" is a guess
• Attribution by vibes: self-reported productivity gains overstate real savings by 30-60% in every controlled study
• Cost amputation: the ROI counts API spend but forgets prompt iteration time, eval runs, observability, and the engineer-hours spent babysitting the system
• Annualizing a single quarter: a six-week pilot result extrapolated to a 12-month savings figure is the most common board-deck lie
• Wrong comparator: "saves $X vs zero" instead of "saves $X vs the tool, salary, or vendor it replaced"
• Vanity metrics: token volume, prompts per day, model calls. None of these are ROI; they are activity counts
• No payback discipline: enterprise AI typically pays back in 12-30 months, but most pilots get killed at 3 because nobody set the expectation

The Three Tiers: Realized, Trending, and Capability ROI

Treating AI ROI as a single number is the fastest way to lose the argument. Mature AI programs separate three tiers and report them as a stack. Realized ROI is the hard dollars already booked. Trending ROI is the trajectory of metrics moving toward booking. Capability ROI is the strategic optionality the investment unlocks. Skipping the second and third tiers is what gets pilots cancelled at the 90-day review.

CFOs do not need the second and third tiers to be precise. They need them to be honest, bounded, and committed to a future booking date. "We have not realized savings yet, but cycle time is down 41% and we project booking in Q3" is a defensible story. "We have not realized savings yet" alone is a death sentence.

• Realized ROI: dollars already in the P&L, signed off by finance, traceable to a system of record
• Trending ROI: operational metrics moving in the right direction with a forecasted booking date
• Capability ROI: strategic positioning, talent retention, data assets, IP, infrastructure that compounds
• Each tier reports its own KPI, its own confidence interval, and its own time horizon
• Mix matters: 100% Tier 1 means you are not investing in the future; 100% Tier 3 means you are dreaming
• Board-friendly format: one slide per tier, dollar number, source system, confidence band, next milestone

Baselines: The Step Nobody Does, Then Regrets

The single most expensive mistake in enterprise AI is shipping a system before the manual version was measured. Once AI is running, the "before" world is gone. You cannot rebuild it from memory, JIRA tickets do not give you cycle time, and HR cannot tell you how many hours people spent on a specific task. Two to four weeks of clean pre-launch baseline data is the difference between a model that holds up and one that gets torched in a board meeting.

Baselines are not a one-time exercise. The right discipline is to instrument the manual workflow at the same moment you scope the AI project, run it for at least one full business cycle, and freeze the numbers in a signed-off baseline memo. Finance signs the baseline. Then AI ships. The "after" measurement uses the same systems of record, the same definitions, the same exclusions. No moving the goalposts mid-flight.

• Time-per-task: stopwatch or time-tracking, not self-report. Use a sample of at least 30 task instances
• Throughput: completions per FTE per week from the system of record, not a survey
• Error rate and rework cost: defect rate times average rework hours times loaded labor rate
• Tool and license cost: every SaaS seat, contractor invoice, and per-transaction fee tied to the workflow
• Cycle time: request to delivery, measured from ticketing system or CRM, not memory
• Quality and CSAT proxies: NPS, ticket reopen rate, dispute rate, refund rate, audit failure rate
• Loaded labor rate: salary plus benefits plus overhead, not just base. Finance usually has a standard number
• Sign-off: baseline memo goes to finance and the executive sponsor; once signed, it is the contract

Attribution Methods That Survive a Finance Review

Attribution is where consulting decks go to die. The honest answer is that most AI systems run in environments with too many confounders to attribute changes cleanly, and the right response is not to fake precision but to pick the tightest method the workflow allows. Listed below in order of credibility. If you can run an A/B test, run an A/B test. If you cannot, pick the next-best method and document the limitations in the model.

Avoid self-reported productivity surveys for anything finance will book. They overstate savings by 30-60% in every controlled study and they collapse the first time an auditor or a skeptical board member asks for the methodology.

• Randomized A/B with cohorts: half the team uses AI, half does not, for 2-4 weeks. Gold standard
• Stepped rollout: launch by region, team, or product line on a staggered schedule. Near-gold when A/B is not possible
• Difference-in-differences: compare delta in AI cohort against delta in matched non-AI cohort. Good for natural experiments
• Pre-post with matched baseline: rigorous if the baseline window is long enough to smooth seasonality
• Time-and-motion: measure the same task before and after with stopwatch precision. Strong for narrow workflows
• Cost-per-output: total fully loaded cost divided by units produced, tracked monthly. Strong for high-volume work
• Funnel deltas: conversion or completion rate at each step. Strong for customer-facing AI
• Shared credit rules when multiple initiatives overlap. Finance writes the rule; engineering does not get to claim 100%
• Avoid: self-reported productivity surveys, "users say it saves them an hour a day," and "we surveyed N managers"

Total Cost of Ownership Most Models Get Wrong

Anthropic and OpenAI API spend is the most visible AI cost and usually the smallest one. The real bill is the people, the time, and the operational scaffolding around the model. A finance-grade TCO model has at least eight line items, only one of which is the inference bill.

The ratio that matters: for most enterprise AI workloads, inference is 15-35% of fully loaded year-one cost. Engineering build is 30-50%. Ongoing operations (evals, drift watch, prompt iteration, observability) is 20-30%. If your model has inference as the dominant cost, the model is wrong.

• Model inference: API tokens, fine-tuning, embedding, plus rate limits and overage. Use 90-day actuals, not pilot data
• Engineering build cost: loaded engineering hours for design, build, integration, and stabilization, typically 3-12 person-months for a real system
• Ongoing engineering: prompt iteration, eval runs, dependency upgrades, model migrations. Budget 20-30% of build cost per year
• Observability and evals: LangSmith, Langfuse, Braintrust, or in-house, plus the human labelers who maintain the eval set
• Data work: cleaning, labeling, embeddings, retrieval index maintenance, often the largest hidden cost
• Security, compliance, legal: DPIAs, contract review, vendor assessments, SOC2 scope additions
• Change management: training, documentation, user adoption, support load increase in the first 90 days
• Opportunity cost: what the engineering team would have shipped instead. Real, even if uncomfortable to write
• Risk reserve: 10-15% of year-one cost for incidents, rollbacks, model deprecations forcing rework

The Metrics Finance Actually Books

Finance teams do not book "productivity gains." They book hard dollars: reduced spend, increased revenue, deferred capex, freed headcount that gets either redeployed or eliminated. Soft metrics like employee satisfaction matter for retention models but do not show up in the cash flow statement.

The most defensible AI ROI dollar amounts come from four buckets: vendor and tool consolidation, headcount avoidance, revenue uplift with clean attribution, and risk-and-loss reduction. MIT's 2025 report found the highest-ROI generative AI work was actually in back-office automation (BPO elimination, external agency replacement, ops streamlining), not the customer-facing marketing tools where most budget lands.

• Tool and vendor consolidation: BPO contracts ended, SaaS seats reduced, external agencies replaced
• Headcount avoidance: planned hires not made, with a written counterfactual in the headcount plan
• Headcount redeployment: roles moved to higher-value work, with the dollar uplift quantified
• Revenue uplift: incremental sales, conversion lift, upsell rate, with A/B attribution
• Margin improvement: lower cost-to-serve, lower cost-per-transaction, lower cost-per-acquisition
• Working capital: shorter DSO, faster close, faster invoice processing, faster collections
• Risk reduction: chargebacks avoided, compliance fines avoided, claims denials reduced, fraud caught
• Asset capitalization: under US GAAP and IFRS, some internal AI development cost can be capitalized; talk to your controller

Payback Periods, NPV, and the Numbers Boards Compare

A finance-grade AI business case computes the same metrics as any other capital project: payback period, NPV, IRR, and a sensitivity analysis. Skipping these is what makes AI projects feel different and therefore expendable when the budget cycle tightens.

Realistic ranges based on 2025-2026 deployment data. Narrowly scoped finance and ops pilots (AP, AR, close, reconciliation) often pay back in 3-6 months with 100-300% year-one ROI. Customer-facing AI typically pays back in 9-18 months. Platform and enablement work (developer assistants, internal search) pays back in 12-30 months, sometimes longer.

• Payback period: months until cumulative cash flow turns positive. Most enterprise AI: 12-30 months
• NPV at the corporate hurdle rate: discount future savings, subtract present cost. Negative NPV at 12% hurdle is a kill signal
• IRR: the discount rate that zeros NPV. Compare to other capital projects competing for the same dollars
• Sensitivity: re-run with adoption at 50%, savings at 70%, inference cost at 2x. If only the best case pays back, the case is fragile
• Risk-adjusted: multiply expected benefit by probability of full realization, often 0.5-0.7 for early-stage AI
• Time-to-value milestones: not just terminal ROI, but checkpoints at 30, 90, 180 days
• Decision rule before you ship: define the kill threshold in writing. "If 6-month adoption is below X, sunset the system"

Reporting Cadence That Keeps Programs Alive

Most AI programs are not killed because they failed. They are killed because nobody had a clean monthly report and the next quarter's budget review found a louder project. A monthly AI ROI report, owned by an executive sponsor and signed off by finance, is the single highest-leverage thing you can build after the system itself.

The report is one page. Top half: the realized, trending, and capability ROI tiers with current dollar values, deltas from last month, and confidence bands. Bottom half: the kill thresholds, the milestones, the risks, and the asks. If you cannot fit it on one page, finance will not read it, and if finance does not read it, your program is invisible.

• Monthly one-page ROI report owned by an executive sponsor, signed off by finance
• Quarterly board update with the same numbers and a rolling 12-month forecast
• Annual portfolio review: every AI initiative ranked on realized ROI, payback progress, and capability value
• Live dashboard for operators (engineering, ops) with cost-per-output, latency, eval scores, drift signal
• Public kill criteria: written, dated, with the executive sponsor's signature. Removes politics from sunset decisions
• Annotation log: every system change (model swap, prompt update, infra migration) annotated on the cost and quality trend lines

How I Engage on AI ROI Work

I work with CFOs, COOs, VPs of Engineering, and AI executive sponsors who need a credible ROI model fast, usually because of a board meeting, a budget cycle, an investor diligence ask, or a stalled program that is about to lose funding. Engagements run from a single audit and rewrite of an existing business case (1-2 weeks) to a quarterly fractional AI officer role overseeing measurement across a portfolio of AI initiatives.

The first session is always free. Walk in with whatever model you have, however incomplete. You will leave with a written list of the specific holes, the order to fix them in, and an honest take on whether the program is worth defending or worth quietly winding down to free the team for something better.

FAQ