AI Transformation Assessment Engine — Evidence-Gated Scanner Architecture

The Forensic Analysis Pipeline

The pipeline separates extraction, verification, scoring, synthesis, and validation. The scanner extracts candidate evidence. The verifier checks whether the source material really supports it. The mathematics layer scores only verified data. The synthesizer receives locked metrics as boundaries. The fact-checker and sanitation layer then remove unsupported or privacy-unsafe claims before the report is shown.

Why This Order Matters

Agentic Verification: The Engine Decides What the Evidence Allows

The engine is agentic within guardrails. It does not simply pass text from one model call to the next. It evaluates intermediate results, compares them against the source, and makes bounded process decisions about what should happen next. Those decisions are not open-ended autonomy; they are evidence-gated controls inside the assessment workflow.

Three Roles. Three Constraints.

The engine behaves like a small assessment team. Each role has a bounded mandate, a different stage responsibility, and a different failure mode. The roles are not decorative prompt personas; they are control surfaces that keep parsing, scoring, synthesis, and validation separate.

[1] Core AI Transformation Vocabulary
AI operating model, AI governance, AI lifecycle,
agent, agentic workflow, autonomous decision,
human-in-the-loop, prompt, model gateway,
evaluation, groundedness, hallucination, RAG,
retrieval quality, red teaming, guardrail,
AI value hypothesis, Impact Statement,
Launch-and-Learn, service area ownership

[2] Architecture, Data, and Safety Signals
API boundary, brownfield integration, service
boundary, model registry, prompt registry,
rollback, observability, drift, lineage,
semantic context, data product, data access,
privacy, masking, purpose limitation, Agent
Behavioral Contract, Secure-by-Design

[3] Value-Creation and Operating Signals
service blueprint, value stream, capability map,
service catalog, decision rights, portfolio
learning, kill / continue / pivot, business case,
value realization, adoption, learning loop,
Sense & Respond, platform-as-product,
Kickstart, Building the System

Two Streams. One Diagnosis.

Every domain has two opposing streams. Stream A measures maturity: evidence of capability, ownership, operating proof, architecture, governance, data quality, learning, and value realization. Stream B measures anti-patterns: evidence of friction, fragility, decision fog, brittle integration, context-poor data, unsafe autonomy, or tool adoption without value proof.

Anti-pattern absence is positive only when verified. A zero anti-pattern score can mean tested absent, which is useful evidence of clearance, or not assessed, where the source simply did not cover the issue. Low burden with low coverage means unknown, not excellent.

The 50-Criteria AI Transformation Knowledge Base

Five domains. Each domain holds 5 maturity indicators paired against 5 anti-patterns. Each criterion carries 3 evidence questions. The result is a 50-criterion, 150-question scanner for whether AI can become safe, measurable, service-owned value creation.

0-3 Evidence Scale and AI Maturity Ladder

The scanner asks what the submitted material proves. A statement that “AI is strategic” is not enough. A pilot is not enough. A platform purchase is not enough. Higher scores require operating proof, measured feedback loops, tested controls, ownership records, or traceable production behavior.

Evidence Check Statuses

Readiness Classification

Evidence Categories

The score asks how strong the evidence is. The category tag asks what kind of evidence it is. Every evidence quote is tagged with exactly one category: Policy, Process, Operational, Automation, Accountability, Architecture, Data, Governance, Learning, or Value.

Source Rules and Knowledge Base Guardrails

The engine treats the uploaded source pack as untrusted customer evidence and the remote Knowledge Base as internal methodology context. Those two sources are never allowed to collapse into one another.

{
  "source_evidence": "Only uploaded assessment material can prove current state.",
  "reference_kb": "Rubric context, validation questions, false-positive checks, risk framing, and tactics only.",
  "forbidden_kb_uses": [
    "customer_current_state_claim",
    "audited_finding",
    "evidence_quote",
    "score_justification_without_uploaded_source",
    "visible_document_name",
    "visible_company_or_organization_name"
  ],
  "general_process_boundary": "Generic IT/process maturity is adjacent context unless it directly proves an AI Transformation criterion.",
  "privacy_output_rule": "Write fluent anonymized prose from the start. Do not emit company names or placeholder tokens."
}

What to Submit

The assessment accepts any combination of documents. More document types provide richer signal across all five domains. Absence of entire document categories is itself a maturity signal. Do not manufacture missing documents for the assessment; absence is part of the diagnosis.

The Firewall Between AI and Roadmap

Phase 2 contains no generative AI. It applies deterministic arithmetic to evidence-checked audit logs. Unsupported scanner output has already been rescanned or downgraded before this layer runs.

Phase 3 Output Contract

Phase 3 produces structured JSON consumed by the dashboard. It cannot fabricate optimism because the visual scorecard, classification, evidence density, domain scores, anti-pattern findings, and source gaps are locked by deterministic validation.

{
  "phase_3_strategy": {
    "executive_summaries": {
      "transformation_lead": "operating model, decision rhythm, learning, portfolio, and adoption lens",
      "technology_lead": "architecture, platform, lifecycle, observability, safety, and data lens",
      "business_service_owner": "value stream, service ownership, customer impact, and capability lens"
    },
    "evidence_summary": "supported findings only, with adjacent context clearly labeled",
    "domain_diagnosis": {
      "A": "Adaptive Operating Model",
      "B": "Enterprise AI Architecture & Platform Readiness",
      "C": "AI Strategy, Governance & Value Realization",
      "D": "Data Foundations, Ownership & Accessibility",
      "E": "Business Capability & Service Architecture"
    },
    "planning_decision": "GO | WARN | BLOCK | NO_GO",
    "findings_mode": "used when confidence is LOW",
    "remediation_roadmap": [
      "Kickstart: diagnose, select domain, run vertical slice, validate value and safety",
      "Building the System: integrate architecture, platform, data, teams, rhythm, governance, and learning"
    ]
  }
}

The Report Is Checked After It Is Written

The engine treats generated strategy as another artifact to verify. The first complete report draft can still be rejected, rewritten, or blocked if it exceeds the evidence, leaks internal reference material, misuses a tactic, or turns uncertainty into advice.

Real Model Roles and Fallback Chains

Model routing is centralized in src/models.ts. The production configuration uses stronger reasoning models for forensic audit, targeted rescan, synthesis escalation, roadmap creation, fact-checking, and quality-gate explanation. Fallbacks are ordered by task shape: fast safety checks can start with Gemini Flash, independent verification uses Gemini Pro-class models, and high-stakes synthesis routes through Claude Sonnet/Opus and GPT-5.5 profiles.

The Difference Between Tool Adoption and Value Creation

A generic AI-readiness survey asks whether the organization has a strategy, tools, pilots, data, and governance. The AI Transformation Engine asks whether the documents prove that those things operate together as a value-creation system.