AI Prototype-to-Production Failure Database 2026

Original incident synthesis by Uvik Software

Research disclosure: Uvik Software publishes this database and provides AI application rescue and production-hardening services. No organization can pay for inclusion, exclusion, a lower control-priority score or a more favorable classification.

Executive answer: The first release contains 14 verified public records across court decisions, regulator actions, company postmortems, peer-reviewed research and investigative reports. 9 records fall into the high or urgent control-priority bands, 14 are coded as multi-causal, and 13 reached users or operational production before the failure was documented. The most repeated gaps are task-relevant evaluation, accountable human review, observability, permission or environment isolation, recovery controls and authoritative grounding.

Key findings

• 14 records are included in the versioned candidate dataset.
• 10 records have an evidence-confidence score of 85 or higher.
• 9 records score 60 or higher on control priority; this is a prevention-priority measure, not a vendor ranking.
• 14 records combine more than one root-cause or failure layer.
• 13 records occurred after a public or operational release; one record documents a launch gate that was reversed before broad production.
• Evaluation is the largest primary category with 7 records.
• Healthcare contributes two independently researched examples where real-world performance or fairness differed materially from the original optimization target.
• The dataset includes both model failures and ordinary production-engineering failures around configuration, rollback, state, permissions and ownership.

Evaluation is the most common primary category in the candidate record set, but most incidents also include secondary system or operating failures.

What qualifies as a database record

A record needs a stable public source or an explicitly authorized assessment. The source must establish that an AI or algorithmic system reached a real operating workflow, public pilot, customer interface, filed professional output or production release gate. The database separates three layers:

The candidate release favors court, regulator, company and peer-reviewed sources, with carefully labeled investigative records where no primary postmortem exists.

Failure taxonomy

A visible model error is often only the trigger. Production impact depends on architecture, state, permissions, evaluation, recovery and ownership.

All published candidate records

The control-priority score is not a severity ranking of organizations. It estimates how urgently an engineering team should address the coded control pattern when designing a similar system.

How scoring works

Evidence confidence

Control priority

Each component is available in the CSV. The included Python script recalculates both totals and blocks a final public scoring mode until source-owner preview and security-lead approval are complete.

The score helps prioritize preventive engineering work; it does not certify products or assign blame.

Recurring missing-control families

Independent evaluation and accountable review appear repeatedly, but production isolation and rollback are equally important in agentic and stateful systems.

Evaluation failures

The dataset includes several different evaluation problems:

• The Epic Sepsis Model performed materially worse under independent real-world validation than its developer-reported result.
• A healthcare risk algorithm optimized cost rather than health need, reproducing structural racial disparities.
• Gemini’s image-generation tuning failed in historical and context-specific prompts and was paused.
• Government and airline chatbots provided authoritative-looking guidance that contradicted law or policy.
• Rite Aid deployed facial recognition without adequate accuracy testing, subgroup analysis or false-positive monitoring.

These are not one failure mode. They show why evaluation needs to match the actual decision, user population, authority level and impact of the production workflow.

State, permission and environment failures

Agentic systems add a separate class of risk: the model can trigger tools or mutate durable state. The reported Replit incident illustrates why a code freeze written in natural language is not a production control. Development and production data need technical separation, scoped credentials, explicit destructive-action approval and tested recovery.

The same principle appears in autonomous systems. Cruise’s post-collision behavior required a deterministic minimal-risk state and trustworthy event reporting. Prompt instructions or model intent cannot replace a state machine for irreversible or safety-critical actions.

Reliability and recovery failures

The OpenAI Redis incident was not a model-quality failure. It was a production configuration failure in an AI service. Staging, canary and dual writing did not reach the connection threshold that exposed the misconfiguration. The corrective pattern is conventional but essential: typed configuration, parity checks, load testing, telemetry and a rehearsed rollback.

This is why the database includes ordinary distributed-systems failures. AI prototypes do not become production-ready merely because model outputs improve.

Ownership and authority failures

Air Canada’s tribunal case and Mata v. Avianca show a common operating mistake: generated text entered a high-authority workflow without a named person or system boundary responsible for verification. The answer may come from a model, but the organization still owns the policy, legal filing, customer promise or production action.

Source and rights workflow

1. Identify a stable public source or a first-party assessment with explicit publication rights.
2. Record the evidence class, source date, rights status and capture date.
3. Extract only the facts needed to explain the engineering pattern.
4. Keep source facts separate from Uvik Software interpretation.
5. Run technical review of taxonomy, root causes and corrective controls.
6. Run legal and privacy review where the record contains confidential or potentially identifying information.
7. Send a private preview to the named organization, vendor or source owner when feasible.
8. Publish only after required approvals and version every material correction.

Public evidence, editorial coding, rights review and versioned corrections remain separate steps.

Anonymization and client material

Version 1.0-rc1 contains no client records. Future Uvik Software assessment patterns can be included only when publication rights are explicit and re-identification risk is acceptably low. Removing a company name is not sufficient. Industry, geography, architecture, dates, team size, customer count and unusual failure details can identify an organization when combined.

When useful technical detail cannot be preserved safely, the record must remain private.

How to use the database

• Turn each missing-control field into an architecture-review question.
• Create hidden tests and failure-injection scenarios from recurring triggers.
• Prioritize rare high-impact controls separately from common low-impact defects.
• Review whether state, permissions, evaluations, observability and rollback have named owners.
• Use the corrective-control field as a starting point, not as an environment-specific design approval.

Production-readiness checklist

1. Move business rules, permissions and irreversible actions into deterministic services.
2. Define explicit state transitions, idempotency, approvals and replay behavior.
3. Create offline, subgroup, adversarial and online evaluation baselines.
4. Threat-model prompts, tools, data access, tenant boundaries and secrets.
5. Add timeouts, retries, circuit breakers, backups, fallbacks and kill switches.
6. Set cost, token, latency and uncertainty budgets with telemetry.
7. Require tests, provenance, versioned prompts, dependency controls and architecture documentation.
8. Assign a production owner, incident route, support model and change-governance process.

Download the data

• Download failure records CSV
• Download taxonomy CSV
• Download source register CSV

JSON note: JSON is intentionally not generated in this package. The production site should generate it automatically from the final CSV and preserve the same field names, dataset version, methodology version and capture date.

Limitations

• The dataset is a curated evidence set, not a random sample of AI projects.
• Public incidents overrepresent visible, regulated and newsworthy failures.
• Source technical detail varies substantially.
• Root-cause and missing-control fields are editorial coding, not admissions by the named organization.
• The score prioritizes preventive controls and cannot measure total financial, human or reputational harm.
• Source-owner preview and final approvals are not complete in version 1.0-rc1.

Corrections and submissions

A correction must identify the record ID, disputed field, current value, proposed value and a stable public source. Confidential submissions remain private unless the submitter grants explicit publication rights and the record passes technical, legal, privacy and re-identification review.

Commercial handoff

Prototype works but the production boundary is unclear? Uvik Software audits code, agents, state, permissions, evaluations, observability and recovery, then produces a keep, change and rebuild plan under NDA. Request an AI production-readiness assessment.

Primary sources

• BC Civil Resolution Tribunal – Moffatt v. Air Canada
• EEOC – iTutorGroup settlement
• FTC – Rite Aid facial recognition action
• Zillow – Zillow Offers wind-down
• Google – Gemini image-generation postmortem
• NHTSA – Cruise consent order
• JAMA Internal Medicine – Epic Sepsis Model validation
• PubMed – racial bias in healthcare risk algorithm
• The Markup – NYC MyCity chatbot investigation
• Microsoft – Recall preview update
• Mata v. Avianca sanctions order
• OpenAI – ChatGPT Redis outage postmortem
• AI Incident Database – reported Replit agent incident
• Time – DPD chatbot incident
• AI Incident Database research paper