AI IS SKEWING YOUR DORA SIGNALS — FIX VISIBILITY BEFORE YOU “OPTIMIZE” THE WRONG THING
AI coding tools are distorting DORA signals unless teams make AI work visible and instrument quality, not just speed. A thoughtful piece argues that DORA numbe...
AI coding tools are distorting DORA signals unless teams make AI work visible and instrument quality, not just speed.
A thoughtful piece argues that DORA numbers can look better while systems get shakier when AI speeds up code generation but pushes defects past review and into production DevOps.com. The pattern: higher deployment frequency, compressed “coding time,” and longer, leakier review/MTTR tails.
A counter-move is to make AI work visible. Shopify’s approach — an internal agent that runs in public channels so others can see, reuse, and critique the work — turns one person’s good prompts and judgment into team learning instead of chat silo drift Nate’s Substack.
This also addresses “shadow AI” risk: many employees are using personal AI accounts at work, which hides the activity, complicates measurement, and introduces compliance issues TechRadar.
DORA can look “up and to the right” while quality quietly degrades when AI accelerates code beyond review capacity.
Making AI work public reduces shadow AI, improves reuse, and gives leaders realistic delivery and reliability signals.
-
terminal
Segment DORA by author type (human vs AI-assisted) and correlate with defect density, rollback rate, and review dwell time.
-
terminal
Pilot a public-by-default AI agent channel; log non-sensitive prompts/outputs and measure reuse rate and mean time saved.
Legacy codebase integration strategies...
- 01.
Insert an AI gateway/proxy (SSO, DLP/redaction, audit logs) so ChatGPT/Claude traffic is visible and policyable.
- 02.
Tighten CI: require tests/docs for AI-heavy diffs, elevate approvals on risky files, and watch MTTR for AI-touched services.
Fresh architecture paradigms...
- 01.
Design agents to work in shared channels with searchable transcripts and opt-in artifact publishing for fast team learning.
- 02.
Instrument pipelines from day one to tag AI-assisted commits and expose quality funnels alongside DORA.
Get daily SHOPIFY + SDLC updates.
- Practical tactics you can ship tomorrow
- Tooling, workflows, and architecture notes
- One short email each weekday