Every light was green
Picture a team six months into an AI-native pipeline. Agents turn approved intents into specs, specs into plans and plans into merged code. Every build passes. Latency is flat. The error budget is untouched. Then the support queue doubles, and a feature that took two days to ship is barely used.
Nobody can find the fault, because nothing broke. A label changed meaning, a step moved behind a menu, and a familiar pattern quietly became a new one. No monitor was looking at people.
That scenario is my extended version of the hypothetical in Nielsen Norman Group’s The Role of UX in Agentic Software Factories, written by Tanner Kohler and published on 30 October 2026.[1] The argument is simple. The new AI-native SDLC playbooks treat UX as background context for the agents. They miss the parts of UX that decide what is worth building and check whether it works.
Last month I wrote that we should map the gates, not just the steps. The software factory has plenty of gates. None of them look at users.
- Context is not a gate. A UX skill file can shape what an agent generates. It can’t refuse a change that hurts people.
- The factory’s scoreboard is about code. Speed, merge rates and incidents. User outcomes aren’t on it, so they can drift.
- Put UX gates where judgment changes the outcome. Before a spec exists, before risky changes ship, and in monitoring.
What the playbook proposes
On 21 August 2026, Anthropic’s Applied AI team published The AI-native SDLC playbook, written by Louis Claxton.[2] Its starting point is that code is no longer the bottleneck. When agents build fast, the slow parts become everything around the build: planning, review, testing and deployment.
The answer keeps the six familiar stages but turns the line into a loop. Every stage ends by committing an artifact to version control, and the next stage starts by reading it.[3] The chain of commits becomes the audit trail: who asked for what, what the agent made, and who approved it.
| Stage | What it commits | Who signs off |
|---|---|---|
| Plan | intent.md, a plain-language problem that anyone can submit | The product owner accepts or rejects it |
| Design | spec.md, generated from the intent with the organisation’s brand, security, compliance and UX skills | The product owner reviews it, and mocks up front-end work in an AI design tool |
| Build | plan.md, then the code | An engineer approves the plan, then the agent builds |
| Test | Tests and evals, checked against the earlier artifacts | Engineers oversee the agent’s checks |
| Deploy | The pull request and its review findings | A tech lead or architect for higher-risk changes |
| Maintain | An incident record, which becomes a new intent.md | Monitoring wakes the agent when a metric breaks its band[2] |
Anthropic isn’t alone. NN/g notes that Google and OpenAI have published similar, less prescriptive versions.[1] OpenAI’s guide is the warmest towards design: it keeps the design system and the final direction of the user experience with the team, and expects agent-built prototypes to reach customers sooner. But neither guide gives user research or usability testing a step of its own.[4]
Anthropic’s playbook is the most detailed of the three. It’s a serious piece of engineering governance, and a lot of it is good design. People stay accountable for the calls that need judgment. Hooks can allow an action, block it, or pause it until a named person approves.[2] That last one is a gate in exactly the sense I used before.
Every gate in the loop has an owner. None of the owners is UX.
Context is not a gate
UX does appear in the playbook. Skills for brand, security, compliance and UX guide the agent as it writes the spec.[2] What UX doesn’t get is a stage, an artifact or an approval. NN/g’s reading is that UX has become context, and that context “can encode only what we already know.”[1]
This isn’t NN/g turning against context. In July 2026 the same group called context the new UX deliverable, and named the practice of curating it UX-context design.[5] I agree with both pieces. Context is how research reaches an agent. It just can’t decide anything on its own.
That line carries the whole argument. A skill file can hold your heuristics, your design system rules and your content style. It can’t hold the need nobody has found yet, or the confusion that only shows up when a real person tries the task.
NN/g walks through the stages one by one. Here’s the gap at each, in short.[1]
- PlanNothing checks that an intent rests on a real user need. Anyone can submit one, the product owner becomes the only filter, and business priorities win by default.
- DesignAI mockups answer how it looks, not whether it meets the need. Someone has to keep the design system machine-readable, and the playbook doesn’t say who. User research isn’t mentioned at all.
- TestThe agent checks its work against the team’s own artifacts. If a wrong assumption entered at Plan, every later check confirms it.
- MaintainMonitoring watches technical health. A feature people abandon doesn’t trip a single alarm.
None of this is a bug in the playbook as an engineering document. It’s an ownership gap. And in a fast system, ownership gaps don’t stay small.
Why it compoundsDrift, one good commit at a time
Sidney Dekker’s Drift into Failure describes how complex systems slide toward breakdown through small, everyday decisions, each of them reasonable on its own.[6] An agentic factory is built to make many small changes quickly. NN/g gives the same warning: lots of well-meant agent changes can slowly wear a user experience down.[1]
Each change passes. The sum doesn’t, and no single commit is to blame. Switch the UX gates off in the illustration at the top of this page and you can watch it happen.
What the factory measures
Teams get what they measure, and the playbook is precise about measurement. Every stage comes with suggested metrics.[2] Read them as a designer and a pattern shows up.
| Stage | The playbook measures | Add a user signal |
|---|---|---|
| Plan | Time from first conversation to a committed intent, and the share accepted | Share of intents that cite user evidence |
| Design | Time from intent to spec, and spec rework after build starts | Problems found by a UX review before build |
| Build and test | First-pass merge rate, CI and eval pass rates | Task success in a quick test, for flagged changes |
| Deploy | Review time, gate wait times and DORA measures | Task completion and error rate after release |
| Maintain | Time from breach to a triage intent, and repeat incidents | Task abandonment, conversion drops and support spikes |
Every metric in the middle column is about the speed and health of the code. They’re good metrics. But a team can hit all of them and still ship something people can’t use, and nothing on the board will say so.
The evidenceEngineering’s own research agrees
NN/g’s piece is an argument, not a study. The data comes from somewhere else. DORA, Google’s long-running research programme on software delivery, surveyed nearly 5,000 technology professionals for its 2025 report on AI-assisted development.[7] Its headline finding is that AI works as an amplifier: it magnifies a team’s strengths and its dysfunctions alike.[8]
7
AI capabilities // DORA 2025
Foundations that decide whether AI helps or hurts. A user-centric focus is one of them.[9]
40%
Higher organisational performance
for teams with a strong focus on users, across DORA’s research.[8]
The finding that matters here is the warning. Without a user-centric focus, DORA found that adopting AI can make team performance worse, not better. Teams simply move faster in the wrong direction. DORA has a name for it: the feature factory trap.[8]
That’s an engineering research programme concluding that user focus decides whether the factory pays off. A UX gate is one way to make that focus structural instead of hoping for it.
Three UX jobs, one covered
Kohler frames UX work in an AI-native SDLC as three roles: deciding what is worth building, evaluating what gets built, and adding context to what the AI generates.[1] The playbook covers the third one well.
The first two jobs are where UX earns its keep. Discovery research finds the needs nobody wrote down. Usability testing finds the problems a team can’t see because they built the thing. Both depend on people outside the team, which is exactly what a closed loop of artifacts leaves out.
You could ask whether agents can stand in for the users too. NN/g has tested synthetic users and recommends them for desk research and early hypotheses, not for final decisions.[10] In a 2026 survey study, synthetic respondents also bunched around the most common answers and dropped the long tail that real people give.[11] The long tail is where discovery lives.
There’s a second loss that’s easy to miss. NN/g’s Director of Research, Maria Rosala, argues that even if AI could produce research as good as an expert’s, the team would still lose the learning. The value of research is partly what a team absorbs by watching people struggle, and a report can’t carry that.[12] A factory that reads findings but never watches users gets the report and misses the learning, at scale.
“Indifference towards people and the reality in which they live is actually the one and only cardinal sin in design.”
Where the UX gates go
NN/g makes a useful point about AI and process in general: AI compresses the design process, it doesn’t delete it. Experienced designers still move through the same steps, only faster.[13] A UX gate is that compression applied to the factory. The judgment stays, at the speed of the loop.
Gating every change would turn a fast factory back into a slow one, and the playbook is right to avoid that. The aim is a few UX gates, placed where judgment about people changes the outcome. I ask four questions, the same way I place any gate.
- 01Is there evidence of a real need?If an intent can’t point to research, support data or observed behaviour, it isn’t ready for a spec. Send it to discovery first.
- 02Does it change how people find their way?Navigation, information architecture, labels and familiar conventions. People expect your product to work like the others they already use.[14]
- 03What does a wrong move cost the user?Money, data, trust or safety. The higher the cost, the closer to real people the check should be.
- 04Can a tool judge it?Accessibility rules, token use and content lint can run in CI. Save human attention for what tools can’t judge.
The answers set a level of UX check, from U1 to U4. Try it on a change.
- 01Is there evidence of a real user need?
- 02Does it change navigation or a convention?
- 03Could a wrong move cost money, trust or safety?
- 04Can a tool check it?
Automated checks
There’s evidence behind it, and it changes nothing people rely on to find their way.
Let the pipeline run its accessibility and design system checks, then log the change. No person needed.
U1, automated checks. Change an answer to see the level move.
None of these need a new stage. U4 belongs at the Plan gate, where the product owner already accepts intents. U2 and U3 belong at the Design and Deploy gates. What’s new is an owner and a question at gates that already exist.
MaintainMake a user problem count as a breach
The playbook’s Maintain stage already wakes the agent when a metric leaves its band, and the agent writes up what it finds as a new intent.[2] Feed user signals into the same loop. A spike in task abandonment should page someone the way a spike in server errors does.[1] And the intent it creates should reach a researcher before it reaches an agent, because the agent will happily fix the symptom.
Six moves to start this week
NN/g closes with six practical moves.[1] Here they are, with how I’d apply each one.
- 01Rewrite the intent template around usersAsk who has the problem, what they’re trying to do and what shows it’s real. Let researchers file problems before anyone proposes a fix.
- 02Share the acceptance callAdd user-centred acceptance criteria, and let design accept intents alongside product.
- 03Write down when UX review is requiredNavigation, IA, conventions and high-risk tasks. Make the triggers a hook in the pipeline, not a habit someone has to remember.
- 04Alert on user signalsTask abandonment, conversion drops and support spikes, treated like server errors and routed to research.
- 05Own the UX contextDraft the UX skills yourself, put your name on them and version them. Context nobody owns goes stale.
- 06Make research machine-readableKeep insights and the design system where agents can read them, so a spec can surface evidence that contradicts it.
The first move does the most work, so here is the template I’d start from. It fits in the same repo as every other intent.
# intent.md (user-first)
## The problem
Who has it: ______________ (a real user group, never "users")
What they're doing: ______________
Where it breaks: ______________
## Evidence
Source: research / support tickets / analytics / observation
Link: ______________
Strength: strong / some / a hunch
(a hunch goes to discovery, not to an agent)
## What changes for people
Task affected: ______________
Changes navigation, IA or a convention? yes / no
Cost of a wrong move for the user: low / money / trust / safety
## How we'll know it worked
Before release: ______________ (e.g. 4 of 5 people finish the task)
After release: ______________ (the user signal to watch, and its alert)
## UX check
Level: U1 automated / U2 UX review /
U3 usability test / U4 research first
UX owner: ______________ (a person, with a backup)
If the evidence field says “a hunch”, the intent isn’t ready for an agent. That one rule does most of the work.
Regulated work already has this gate
In the industries I design for, none of this is new. Where a use error can hurt someone, regulators stopped treating usability as context a long time ago.
| Domain | What it expects | In the factory |
|---|---|---|
| Medical devices, US FDA guidance (2016) | Use-related risk analysis, formative evaluation during design, and a final human factors validation test with representative users on critical tasks[15] | U3 before release for anything that touches a critical task |
| Medical devices, IEC 62366-1 | A documented usability engineering process across the design life cycle[16] | UX checks committed as artifacts in the same audit trail |
| Nuclear control rooms, NRC NUREG-0711 | Twelve human factors elements, from task analysis to verification and validation and human performance monitoring after go-live[17] | User signals in Maintain, owned by people, not only by monitors |
NN/g puts it plainly for these industries: process is “a safeguard against harm,” not bureaucracy.[13]
The playbook itself notes that regulated organisations can’t accept under-reviewed code, and that the new process has to fit around systems that auditors already trust.[2] The same logic applies to use. If your product sits under one of these regimes, the UX gates aren’t optional. Design them in now, as artifacts the loop can read, rather than bolting them on before an audit.
Speed moved the bottleneck. It didn’t move the responsibility.
An agentic factory can build anything quickly. Deciding what’s worth building, and checking that it works for people, still needs someone who has met them.
Fair questions
Isn’t this just UX defending its turf?
Partly, and it’s fair to say so. But the case doesn’t rest on jobs. It rests on blind spots. A loop that only checks itself against its own artifacts can’t find the need it never wrote down. And it isn’t only designers saying so: DORA’s engineering research reaches the same conclusion.
Won’t UX gates slow the factory down?
Badly placed ones will. Most changes should be U1 and handled by tools. The few that reach a person are the ones where a wrong call costs more than the wait.
Can’t AI run the usability tests?
AI can help plan, run and analyse them. It can’t stand in for the participants, and it can’t do the watching for the team. NN/g’s work on synthetic users found them useful for hypotheses, not for decisions.
So is the playbook wrong?
No. It’s an engineering playbook, and a careful one. NN/g does point out that AI labs benefit when teams adopt these models, which is worth keeping in mind. The fix isn’t to reject it. It’s to give UX an owner, an artifact and a gate inside it.
Where the facts come from
- The Role of UX in Agentic Software Factories (opens in new tab), Tanner Kohler, Nielsen Norman Group, October 2026. The critique, the three UX roles and the six moves.
- The AI-native SDLC playbook (opens in new tab), Louis Claxton, Anthropic, August 2026. Stages, artifacts, hooks, metrics and the note on regulated organisations.
- Anthropic says AI has changed the SDLC (opens in new tab), DevOps.com, 2026. The artifact chain from intent to incident record.
- Building an AI-native engineering team (opens in new tab), OpenAI. Seven SDLC phases, and where design sits in them.
- Context Is the New UX Deliverable (UX-context design) (opens in new tab), Tony Alicea, Nielsen Norman Group, 2026.
- Drift into Failure (opens in new tab), Sidney Dekker, 2011.
- Announcing the 2025 DORA Report: State of AI-assisted Software Development (opens in new tab), Google Cloud, September 2025. Nearly 5,000 respondents.
- User-centric focus (opens in new tab), DORA. The 40% figure, AI as an amplifier, and the feature factory trap.
- Introducing the DORA AI Capabilities Model (opens in new tab), Google Cloud, September 2025. The seven capabilities.
- Synthetic Users: If, When, and How to Use AI-Generated “Research” (opens in new tab), Nielsen Norman Group.
- Synthetic users vs human respondents (opens in new tab), Nielsen Norman Group, 2026. Mode collapse: the common answer kept, the long tail lost.
- Don’t Outsource the Learning: Why Human-Led Research Still Matters in the Age of AI (opens in new tab), Maria Rosala, Nielsen Norman Group, 2026.
- Design Process Isn’t Dead, It’s Compressed (opens in new tab), Nielsen Norman Group, 2026.
- Jakob’s Law (opens in new tab), Laws of UX.
- Applying Human Factors and Usability Engineering to Medical Devices (opens in new tab), US FDA, February 2016.
- FDA human factors guidance webinar (opens in new tab), US FDA. Lists ANSI/AAMI/IEC 62366-1:2015 and its usability engineering process.
- Human Factors Engineering Program Review Model, NUREG-0711 Rev. 3 (opens in new tab), US Nuclear Regulatory Commission, 2012.