An AI product roadmap is a ranked plan of the AI features a product team will discover, test and ship, where each item must pass explicit gates for business value, data readiness, eval quality, cost per task and risk before it moves forward. It replaces fixed launch dates with evidence that the feature works and pays back.
Key Takeaways
- Most AI pilots never reach production. MIT NANDA's July 2025 study found only 5% of custom enterprise AI tools got there, so the roadmap's first job is to say no.
- Score every AI idea on six criteria: problem value, eval-ability, data readiness, unit economics, risk and workflow fit. A score of 1 on eval-ability or risk is a knockout.
- Price features by cost per successful task, not cost per model call. A cheap call that fails half the time is an expensive feature.
- Move items through gates (discovery, eval-gated beta, general availability, operate) instead of promising dates for probabilistic features.
- Budget for model change. Prices, tokenizers and model versions shift during a product's life, and each shift needs a fresh eval run.
This guide is for CPOs, product heads and founders deciding where AI belongs in next year's plan. Cyberdunes runs product management consulting and the AI product management bootcamp, and we build AI features into our own platform, so the scoring model below is the one we use when an AI idea asks for a roadmap slot.
Why Do Most AI Features Never Pay Back?
Most AI features fail because teams pick the technology before the problem, then discover too late that they cannot measure quality or afford the cost at scale. The evidence from 2025 and 2026 is consistent on this.
- MIT NANDA's preliminary report, The GenAI Divide: State of AI in Business 2025 (July 2025), estimated that despite $30 to 40 billion in enterprise investment, "95% of organizations are getting zero return." It found that 60% of organizations evaluated enterprise-grade AI tools, 20% reached pilot and just 5% reached production. The study is small (52 organizational interviews, 153 survey responses and a review of 300+ public initiatives), so treat it as directional.
- The same report blamed "brittle workflows, lack of contextual learning, and misalignment with day-to-day operations," not model quality. Those are roadmap decisions, not engineering accidents.
- Gartner predicted in June 2025 that over 40% of agentic AI projects will be canceled by the end of 2027 "due to escalating costs, unclear business value or inadequate risk controls." Gartner also said "many use cases positioned as agentic today don't require agentic implementations."
- In July 2026, Gartner's George Brocklehurst said buyers "want better outcomes and adding more AI features often creates more cost, not better outcomes."
Read together, these point to three failure modes we see in roadmap reviews: an AI feature with no owner metric, an AI feature nobody can test, and an AI feature whose cost grows faster than its value. The scoring model below is built to catch all three before engineering time is spent.
How Is an AI Product Roadmap Different From a Normal Roadmap?
An AI roadmap plans for uncertainty that a normal roadmap does not have: outputs vary, quality must be measured statistically and the model underneath can change without your team shipping anything. Four differences matter most.
| Dimension | Conventional feature | AI feature | What the roadmap must add |
|---|---|---|---|
| Output | Deterministic: same input, same result | Probabilistic: same input can give different answers | Quality targets as pass rates, such as 90% of test cases passing |
| Done means | Feature complete and QA passed | Eval thresholds met on real cases, then held in production | Eval gates between stages instead of launch dates |
| Cost | Mostly fixed after build | Variable per use: tokens, retries, tool calls | A cost-per-successful-task ceiling for each item |
| Dependency risk | Your code changes when you change it | Model prices, versions and behavior change on the vendor's schedule | Re-evaluation on every model or prompt change |
The dependency risk is concrete. Anthropic's pricing documentation notes that its newer models use a tokenizer that "produces approximately 30% more tokens for the same text," lists several older models as retired, and records that a scheduled price increase for one model was canceled. Prices can move in either direction: Stanford's 2025 AI Index found the inference cost of a system performing at GPT-3.5 level "dropped over 280-fold between November 2022 and October 2024." A roadmap that assumes today's model and price will hold for three years is planning on sand.
How Do You Score an AI Feature for the Roadmap?
Score each candidate from 1 to 5 on six weighted criteria, apply two knockout rules, and let the total decide whether it gets a beta, a discovery spike or a parking spot. The weights reflect what kills AI features in practice.
- Problem value (25%). How often does the problem occur and how much does it hurt? Tie it to one metric you already report, such as handle time, conversion or days sales outstanding. A 5 moves a board-level number; a 1 is a demo.
- Eval-ability (20%). Can you write down what a good output looks like and collect 50 to 100 real examples to test against? A 5 has objective pass/fail checks; a 1 means only the user's future regret can grade it. Our guide on how to build AI evals shows how to write these rubrics without code.
- Data readiness (15%). Does the feature have the data it needs, with the right permissions and freshness? A 5 is clean, owned and current; a 1 needs a data project first.
- Unit economics (15%). Does cost per successful task sit well below value per task? A 5 has a 10x or better margin; a 1 loses money on every use.
- Risk, inverted (15%). What does a wrong answer cost in money, compliance, safety or brand? A 5 is low and reversible; a 1 is regulated advice or irreversible action.
- Workflow fit (10%). Does it sit inside a workflow people already use, and does value show in weeks? A 5 needs no behavior change.
Decision rules: 3.8 or above earns an eval-gated beta. 3.0 to 3.7 earns a time-boxed discovery spike to raise the weakest score. Below 3.0 is parked and rescored next quarter. Any 1 on eval-ability or risk is a knockout regardless of total. Keep the weights fixed for at least a quarter, or the model becomes a tool for justifying favorite ideas.
What Does the Scoring Model Look Like in Practice?
Here is the model applied to three ideas for a fictional mid-market accounting software company. The numbers are illustrative, but the pattern is one product teams meet often: the most exciting idea scores worst.
| Feature | Why it scored this way | Total | Decision |
|---|---|---|---|
| Invoice field extraction | High volume, objective ground truth (the ledger), owned data, cheap per document, errors caught at review | 4.50 | Commit to beta |
| Help-center answer bot | Clear deflection metric, help articles exist, answers testable against a question set; risk of wrong answers to customers keeps risk at 3 | 3.85 | Eval-gated beta with human review on billing topics |
| "AI CFO" cash-flow advice | Exciting in demos, but no ground truth for good advice (eval-ability 1) and regulated, high-stakes output (risk 1) | 2.30 | Park; knockouts apply |
The arithmetic for the answer bot: (4 x 0.25) + (4 x 0.15) + (4 x 0.20) + (4 x 0.15) + (3 x 0.15) + (4 x 0.10) = 3.85. Parking the AI CFO is not a permanent no. It becomes a discovery question: could a narrower version, such as flagging overdue invoices, score 3 or more on eval-ability and risk?
How Do You Calculate the Unit Economics of an AI Feature?
Calculate cost per successful task: model cost per attempt, multiplied by an overhead factor, divided by the success rate your evals measure. This one number tells you whether a feature can scale and which model to use.
Take the help-center bot. Assume four turns per conversation, each sending about 6,000 input tokens (instructions, retrieved articles and history) and receiving 400 output tokens: 24,000 input and 1,600 output tokens in total. At Anthropic's October 2026 list prices of $2 per million input tokens and $10 per million output tokens for Claude Sonnet 5.5, that is $0.048 + $0.016 = $0.064. Double it for retries, guardrail checks and logging and you get $0.128 per conversation. If evals show 60% of conversations resolve, cost per successful resolution is $0.128 / 0.6 = about $0.21. On Claude Haiku 5.5 ($0.10 and $0.50 per million), the same math gives about $0.011.
Three lessons follow:
- Success rate moves cost as much as price does. Raising the resolution rate from 60% to 80% cuts cost per success by a quarter without touching the model.
- Do not assume costs only fall. Gartner predicted in January 2026 that by 2030 the cost per resolution for generative AI in customer service "will exceed $3," driven by data center costs, vendors pivoting to profitability and "increasingly complex use cases that consume more tokens."
- Set a ceiling before beta. Write the maximum cost per successful task into the roadmap item. If production cost crosses it, the item returns to discovery.
Cost per task also shapes how you charge for the feature. If you plan to charge per result, read our companion guide to outcome-based pricing for AI products before you set the price.
What Should an AI Product Roadmap Template Include?
A useful AI roadmap template tracks each item's stage, the gate it must pass next, its cost ceiling and its owner, on one page. Dates still exist, but they attach to gate reviews, not to launches.
Copy this one-page template into your planning tool. One row per AI item:
| Field | What to write | Example (help-center bot) |
|---|---|---|
| Problem and metric | The user problem and the one metric it moves | Repeat how-to tickets; ticket deflection rate |
| Score and knockouts | Total from the scoring model; any 1s | 3.85; none |
| Stage | Discovery, eval-gated beta, GA or operate | Eval-gated beta |
| Next gate | The measurable condition to advance | 90% pass on 120 test questions for two weeks |
| Cost ceiling | Maximum cost per successful task | $0.30 per resolved conversation |
| Risk tier and guardrail | Low, medium or high, and the control | Medium; billing questions go to a human |
| Model dependency | Model and version, and the fallback | Primary model plus smaller fallback, both on the eval set |
| Owner and review date | One accountable person and the next gate review | Support PM; first Monday of next month |
The kill switch matters as much as the launch gate. If an AI feature's value or margin stays below its line for a full quarter, retire it and free the team. The MIT NANDA report found that users prefer general chat tools for simple tasks "but abandon it for mission-critical work due to its lack of memory," so plan for the feature to learn: grow the test set from every production failure.
How Do Eval Gates Replace Launch Dates?
An eval gate is a pass threshold on a fixed set of real test cases that a feature must meet, and keep meeting, before it moves to the next stage. It turns "is it ready?" from an opinion into a number.
- Discovery exit. The prototype beats today's baseline on 50 to 100 real cases offline. If people currently resolve 70% of invoices correctly on first pass, the AI must beat 70% on the same set.
- Beta entry. Pass rate meets the agreed threshold, high-risk failure types are at zero, and cost per success is under the ceiling.
- Beta exit. Pass rate and cost hold for two consecutive weeks with 5 to 10% of users, and the online metric beats a control group. Our guide to A/B testing covers how to set up the comparison.
- Operate. Every model upgrade, prompt change or retrieval change reruns the full eval before release. A model swap that saves 40% on cost but drops pass rate below the gate does not ship.
This is where most teams under-invest. Writing 100 test cases feels slower than writing code, but it is the only way to know whether next month's model change helped or hurt. If your product managers have not built evals before, the AI product management masterclass covers the method in a working session.
Should You Build AI Features or Buy Them?
Buy or partner when the capability is not your differentiator and a vendor can show eval results on your data; build when the AI feature is the product. The MIT NANDA data leans toward partnering for most enterprise use cases.
In its interview sample, external partnerships with "learning-capable, customized tools reached deployment ~67% of the time, compared to ~33% for internally built tools." The authors note these are self-reported outcomes that may not account for confounding variables, so use the finding as a prompt, not a rule. In roadmap terms, add a build-or-partner field to every item scoring 3.8 or above, and ask any vendor for its eval results on 50 of your own test cases before you sign.
Gartner's April 2026 prediction raises the bar further: by 2028, "over half of all enterprises will stop paying for assistive intelligence (such as copilots and smart advisors) and instead will favor platforms that commit to workflow results." If you sell software, your AI roadmap should favor features that complete a measurable piece of work over features that merely suggest. Our guide to AI agents for marketing teams shows what that shift looks like inside one function.
How Do You Build Your First AI Product Roadmap in 90 Days?
Spend the first month collecting and scoring ideas, the second running discovery spikes on the top three, and the third putting one feature into an eval-gated beta. Each step has an owner and an output.
| Weeks | Step | Owner | Output |
|---|---|---|---|
| 1 to 2 | Collect every AI idea from sales, support, product and leadership; write each as a problem plus metric | Head of product | Idea backlog of 15 to 30 items |
| 3 to 4 | Score all items with the six criteria in one 2-hour session with product, engineering, data and legal | Product lead | Ranked list with knockouts flagged |
| 5 to 8 | Run discovery spikes on the top three: baseline, test set, prototype, cost estimate | PM plus engineer per item | Go or no-go on each, with eval results |
| 9 to 12 | Launch one item into eval-gated beta with 5 to 10% of users | Feature PM | Two weeks of pass rate, cost and metric data |
| 12 | Quarterly review: rescore the backlog, update weights only if strategy changed | CPO | Next quarter's AI roadmap on one page |
Connect the AI roadmap to your wider plan rather than running it as a side track. Our posts on product strategy consulting and the AI-era product management lifecycle cover how these decisions fit into strategy and delivery.
How Cyberdunes Helps Product Teams Prioritize AI
Cyberdunes works with product leaders in two ways. Our product management services team runs the scoring session with your leadership, builds the first eval sets and sets up the one-page roadmap with gates and cost ceilings. For teams that want to own the method, the AIPM bootcamp, certified with Ritors Product Nexus, trains product managers in AI discovery, evals and AI unit economics on their own backlog.
Talk to us about your AI roadmap
Sources
- The GenAI Divide: State of AI in Business 2025, preliminary findings (MIT NANDA, July 2025)
- Gartner Predicts Over 40% of Agentic AI Projects Will Be Canceled by End of 2027 (Gartner, June 2025)
- Gartner Predicts GenAI Cost Per Resolution for Customer Service Will Exceed Offshore Human Agent Costs by 2030 (Gartner, January 2026)
- Gartner Expects Most Enterprises to Abandon Assistive AI for Outcome-Focused Workflow by 2028 (Gartner, April 2026)
- Gartner Says $234 Billion in Enterprise Application Software Spend Is at Risk from Agentic AI (Gartner, July 2026)
- Pricing (Anthropic documentation, accessed October 2026)
- The 2025 AI Index Report (Stanford HAI, 2025)
