Engineering Maturity Models at Scale in Regulated Financial Services: What to Anchor On, What to Borrow, and How to Keep the Evidence Honest

Overview

The maturity model a large financial services estate should anchor on in 2026 is not a maturity model at all. It is DORA's outcome-metric research programme, now in its second decade and drawing on more than 39,000 respondents, paired with the CNCF Platform Engineering Maturity Model for the delivery mechanism and TMMi for the one domain, testing, where a staged model still has sector-specific empirical support. Everything else in the field, CMMI appraisals, consultancy velocity indices, vendor DevOps ladders, is either receding from core engineering practice or contested by named critics on the record. Sources: Google Research (2024) (↗); CNCF TAG App Delivery (2024) (↗); American Journal of Computer Science and Technology (2024) (↗)

The defining shift of the past eighteen months is that the anchor itself moved. DORA's 2025 report retired the elite/high/medium/low performer tiers that a decade of internal transformation targets were built on, replacing them with seven team archetypes after its own 2024 data showed medium performers beating high performers on change failure rate. Any central standards function still setting divisions a target of "reach elite" is benchmarking against a model its authors no longer trust. At the same time, two consecutive DORA reports found AI coding adoption correlated with worse delivery stability at team level even as individual output rose, which converts engineering hygiene from a cost-centre virtue into the precondition for capturing the AI productivity banks are already reporting to investors. Sources: RedMonk (rstephens) (2024) (↗); RedMonk (rstephens) (2025) (↗); Google Cloud Blog (2025) (↗); DORA / Google Cloud (2024) (↗)

Regulation supplies the deadline pressure but not the method. The EU's Digital Operational Resilience Act applied in full from January 2025 to more than 22,000 entities; the UK's operational resilience regime under PRA SS1/21 and FCA rules reached its transitional deadline in March 2025, with a fresh 2026 wave creating a single incident-reporting framework. None of these texts mandates continuous delivery or forbids it. They demand evidence of governed, repeatable resilience, which telemetry-driven delivery metrics can satisfy better than a change advisory board that approves over 90 per cent of what it reviews. Sources: Council of the European Union (2022) (↗); Sidley Austin (legal insight) (2025) (↗); Global Regulation Tomorrow (Norton Rose Fulbright) (2026) (↗); Team Topologies (2026) (↗)

The hard problem is therefore not model selection but programme design: segmenting thousands of teams of uneven maturity so uplift lands where it is needed, and building measurement a risk function and internal audit will accept as control evidence rather than a dashboard nobody trusts.

Timeline

H2 2022

• EU Council adopts the Digital Operational Resilience Act

H2 2023

• CNCF publishes the Platform Engineering Maturity Model
• McKinsey's developer productivity framework draws sustained public rebuttal

H2 2024

• CrowdStrike outage puts a price on third-party fragility
• DORA 2024 finds AI adoption correlates with worse delivery stability
• Medium performers beat high performers on change failure rate

H1 2025

• EU DORA applies in full
• UK operational resilience transitional deadline passes
• Treasury Committee tallies 803 hours of bank IT outages

H2 2025

• METR RCT finds experienced developers 19% slower with AI tools
• DORA retires performance tiers for seven archetypes and an AI Capabilities Model

H1 2026

• UK regulators issue a single incident and third-party reporting framework
• Platform engineering surveys quantify the golden-path adoption gap

Key Findings

DORA is the shared vocabulary, but it is an outcome framework, not a maturity ladder, and it keeps changing. Its metric set has moved from four keys to five, replaced MTTR with failed-deployment recovery time, and in 2025 abandoned ranked clusters entirely. The correct use, per Martin Fowler's still-standard framing of maturity models as "wrong but hopefully useful", is to treat the metrics as the dependent variable and the DORA capability catalogue as the list of things to fix next, not as a scoreboard for inter-divisional league tables. Level-based models invite gaming through certification incentives; outcome metrics invite gaming through Goodhart's law; the mitigation for both is using them for sequencing rather than reward. Sources: DORA (Google Cloud) (2026) (↗); martinfowler.com (2014) (↗); Computer Things (Hillel Wayne) (2023) (↗)

The CAB finding is the single most load-bearing result for regulated firms. DORA's published research reports no evidence that formal external change approval reduces change failure rates, while correlating it with larger, riskier, less frequent releases; the underlying Accelerate research found externally approved organisations 2.6 times more likely to be low performers. Team Topologies' banking governance analysis adds that CABs in regulated banks approved over 90 per cent of major changes, with some firms rejecting none in a year, and argues for automated peer-review controls as both faster and more auditable. Where a firm cites regulation to defend a manual CAB, that is the excuse pattern: the regulation requires governed change, not a committee. Sources: DORA (dora.dev) (2023) (↗); Information Safety (2022) (↗); Team Topologies (2026) (↗)

Borrow TMMi for testing, retire CMMI ambitions elsewhere. A peer-reviewed survey of sixty financial institutions found TMMi Level 3 the most commonly achieved level, with benefits concentrated in software quality and testing productivity and near-universal parallel use of test automation. That is the only staged model with a sector-specific empirical base; CMMI evidence remains single-case ROI studies, and formal process-capability appraisals persist mainly in outsourced testing contexts rather than core bank estates. For regression automation in legacy estates, TMMi supplies the assessment scaffold while risk-based test selection does the prioritisation work. Sources: American Journal of Computer Science and Technology (2024) (↗); Engineering, Technology & Applied Science Research (2017) (↗); arXiv (2019) (↗)

SAFe is widespread in banks and contested nearly everywhere it is studied. An academic case study of a large full-service bank two years into SAFe found persistent friction in management, requirements, quality assurance and architecture; ICSE-SEIP work across seventeen companies calls it demanding and expensive; empirical comparison gives it roughly 53 per cent market share among scaling frameworks while contesting its effectiveness. Team Topologies co-author Matthew Skelton disputes SAFe's claim to have integrated his framework. The pragmatic reading: borrow Team Topologies' stream-aligned and enabling-team constructs for the operating model, and do not let a scaling framework's planning cadence enlarge batch sizes the metrics are trying to shrink. Sources: arXiv / Springer (RISE Research Institutes of Sweden) (2021) (↗); ACM/ICSE-SEIP (2022) (↗); arXiv (2023) (↗); Pretty Agile (2023) (↗)

Golden paths are the delivery mechanism for uplift without line authority, and the adoption data is unusually sharp. The State of Platform Engineering Volume 4 found well-designed golden paths drive voluntary adoption above 80 per cent against roughly 20 per cent for platforms without them, while only 13.1 per cent of organisations reach an optimised measurement stage and 29.6 per cent measure nothing. Spotify reports Backstage adoption above 90 per cent internally, but multiple independent sources report other organisations stalling below 10 per cent, and Gartner's 2025 Market Guide warns explicitly of "Backstage disillusionment". A Microsoft case study of an unnamed bank with a 120-person central engineering team frames golden paths as a negotiated compromise, not a mandate. Sources: bex.co (2026) (↗); platformengineering.org (2026) (↗); Cortex (2025) (↗); Cortex (citing Gartner) (2026) (↗); Microsoft Learn (2025) (↗)

AI adoption makes sequencing the hygiene fixes more urgent, not less. DORA's 2024 data associates a 25 per cent increase in AI adoption with a 1.5 per cent throughput decrease and 7.2 per cent stability reduction, attributed to larger batch sizes; METR's randomised trial found experienced developers 19 per cent slower with early-2025 tools despite believing otherwise. IT Revolution and Simon Willison's practitioner accounts converge on the amplifier reading: AI compounds the advantage of divisions with strong version control, small batches and automated testing, and compounds the instability of divisions without them. DORA's 2025 AI Capabilities Model names exactly those foundations. This gives the segmentation logic teeth: fix release and regression automation in the weak divisions before their AI adoption outruns their controls. Sources: Google Cloud Blog (2024) (↗); METR (2025) (↗); IT Revolution (2025) (↗); DORA / Google Cloud (2025) (↗)

The rollout mechanisms with documented effect are enablement-shaped, not mandate-shaped. Target's dojo model ran immersive challenges for more than 70 teams and remains the clearest documented pattern for a central function uplifting teams it does not own. Swiss financial infrastructure firm SIX describes a five-plus-one-dimension DevOps model (skills, organisation, process, infrastructure, architecture, plus mindset) used to break silos, reported independently via InfoQ. The CNCF model itself warns that each maturity level demands more funding and staff time and that the top level should not be a goal; its cited value is the improvement list, not the score. Sources: Nerd/Noir (2023) (↗); InfoQ (2023) (↗); CNCF TAG App Delivery (2024) (↗); blog.eisele.net (2025) (↗)

Audit survivability depends on telemetry-derived, tamper-evident metrics, and the academic base for that is only now forming. Research on automated DORA metrics computation is described in the ACM literature as scarce beyond industrial proofs of concept, with one validated open-source pipeline against a 37-microservice estate; a 2026 arXiv extension adds a deployment-consistency metric DORA's dashboard cannot express; other current work proposes auditable DevOps automation via VSM and goal-question-metric structures. Self-assessed scorecards, by contrast, have no peer-reviewed evidence of effectiveness despite widespread commercial adoption, per a 2026 Frontiers multivocal review. Metrics designed as control evidence must be computed from pipelines, not surveys. Sources: arXiv (2026) (↗); arXiv (2026) (↗); Frontiers in Computer Science (2026) (↗)

Evidence & Data

The independently sourced numbers cluster on the cost side. The House of Commons Treasury Committee found nine of the UK's largest banks and building societies accumulated at least 803 hours, more than 33 days, of unplanned IT outages across at least 158 incidents between January 2023 and February 2025, with Barclays alone expecting £5 million to £7.5 million in compensation for one January 2025 mainframe outage. Parametrix estimated banking-sector direct losses from the July 2024 CrowdStrike incident at $1.15 billion to $1.4 billion, with cyber insurance covering only 10 to 20 per cent. Sources: UK Parliament, Treasury Committee (2025) (↗); Reuters (via Cyprus Mail) (2025) (↗); Bloomberg (2024) (↗); Cybersecurity Dive (2024) (↗)

On delivery performance, DORA 2024 recorded the high-performance cluster shrinking from 31 to 22 per cent of respondents while low performers grew from 17 to 25 per cent. The 2025 DORA report claims internal platform adoption near 90 per cent, ahead of Gartner's forecast of 80 per cent of large engineering organisations by 2026, up from 45 per cent in 2022. Faros AI telemetry across more than 10,000 developers showed individual AI output gains without matching organisational delivery gains, corroborating DORA's survey result from an instrumented dataset. Sources: DORA / Google Cloud (2024) (↗); Google Cloud Blog (2025) (↗); Gartner (2026) (↗); Faros AI (2025) (↗)

Bank-reported AI figures deserve a lower evidentiary tier because they come from earnings calls. Reuters reported JPMorgan engineers gaining 10 to 20 per cent productivity from an internal coding assistant against a technology budget later reported near $19.8 billion; Citigroup's CFO cited 9 per cent software development productivity improvement and roughly 100,000 developer hours saved weekly; Goldman Sachs tied AI efficiency to headcount constraints in an internal memo. None of these firms has published change failure rate or lead time alongside the productivity claims. Sources: Reuters (via Investing.com) (2025) (↗); Yahoo Finance (bank earnings call reporting) (2025) (↗); Reuters (2025) (↗)

Signals & Tensions

Two irreconcilable frames for the same engineers. Banks present AI to investors as a headcount and cost story, while resilience regulation demands a stability and change-failure frame; the financial press has not reconciled them and neither, visibly, have the banks. Sources: Reuters (2025) (↗); Bank of England (2026) (↗)

The missing middle of the evidence base. Outage costs are independently documented and vendor tooling is heavily marketed, but the causal link between maturity investment and fewer outages is asserted by consultancies, not demonstrated in independent reporting. McKinsey's Developer Velocity relaunch drew specific published rebuttals from the Pragmatic Engineer and LeadDev, a rare case of a named framework being publicly contested. Sources: McKinsey & Company (2023) (↗); The Pragmatic Engineer (2024) (↗); LeadDev (2025) (↗)

Engineering-intelligence tooling oversells its AI layer. An independent buyer evaluation found LinearB's headline AI feature is a rule-based YAML engine unable to distinguish AI from human contribution, and Jellyfish's AI summaries trace shallowly to source data. Treat vendor measurement claims as unverified until checked in-house. Sources: tianpan.co (2025) (↗)

Regulators are drifting from blocker to enabler, slowly. Legal analysis of SS1/21 frames the requirement as evidence of governed, repeatable resilience rather than any methodology, and vendor DORA-compliance mappings increasingly position CI/CD telemetry as the evidence layer, though those mappings are self-interested. Sources: Data Matters (Sidley Austin law firm blog) (2025) (↗); jfrog.com (2026) (↗); Microsoft (2024) (↗)

Measurement of AI productivity is becoming impossible to control. METR reported in February 2026 that its follow-on experiment produced an unreliable signal because developers refused to work without AI tools. The window for clean baselines has closed, which weakens every future before-and-after claim. Sources: METR (2026) (↗)

Open Questions

1. Will auditors and regulators formally accept telemetry-derived DORA metrics as control evidence, or will firms have to run parallel attestation regimes indefinitely? The auditable-automation literature is one paper deep. Sources: arXiv (2026) (↗)

2. Does any bank have credible before-and-after numbers? Multi-bank practitioner accounts date largely to 2018-era DevOps Enterprise Summit reports; post-2022 named evidence with measured deltas on release frequency or change failure rate is essentially absent from independent sources. Sources: InfoQ (2018) (↗)

3. What replaces the retired DORA tiers as an internal target-setting device? Seven archetypes describe rather than rank, and no published account yet shows a large firm running target-setting on them. Sources: RedMonk (rstephens) (2025) (↗); Agility at Scale (2026) (↗)

4. Why is SAFe ubiquitous in banks yet nearly undocumented in independent literature, and does its planning cadence measurably enlarge batch sizes in regulated estates? Sources: arXiv (2023) (↗)

5. Do scorecards change behaviour or just reporting? No peer-reviewed evidence of scorecard effectiveness exists despite widespread commercial adoption of Backstage, Cortex and OpsLevel. Sources: Frontiers in Computer Science (2026) (↗)

6. How much unsupervised agent-driven change can a regulated estate safely permit, given the gap between 50 per cent and 80 per cent reliability time horizons in METR's data? Sources: METR (2026) (↗)

7. Which of the current measurement stack survives to 2029? DORA's five keys and platform golden paths look durable; performance tiers are already gone, and engineering-intelligence AI features look like tooling fashion until independently validated. Sources: DORA (Google Cloud) (2026) (↗); tianpan.co (2025) (↗)

The uncomfortable conclusion for any central standards function is that the strongest evidence in this field concerns what not to do: do not target retired tiers, do not defend manual CABs on regulatory grounds the regulation does not support, do not trust self-assessment, and do not let AI adoption run ahead of release and test automation in the divisions least equipped to absorb it. The affirmative playbook, golden paths, dojos, telemetry-as-control-evidence, rests on thinner but consistent ground. That asymmetry is itself the finding: the discipline survives audit by proving what it stopped doing.