Lore

spec-driven development

Spec-driven development: the AI engineering workflow at Notion | Ryan Nystrom

Ryan Nystrom argues that AI coding agents (Codex above all) have collapsed the old tradeoffs of software engineering management — waiting for meetings, waiting for review, manually compiling status updates — into a single workflow where you dictate intent, let AI draft and then build from a spec, and let scheduled/background agents handle synthesis and small fixes, producing a rare 'win-win-win' of more relaxation, more fun, and more output; the video further claims this only pays off if organizations also invest heavily in CI pipeline speed, which it frames as a hard mathematical ceiling on how much value agentic engineering can produce.

How I AI · 2026-05-11 · English

Key ideas

  1. The video frames the old engineering workflow (writing design docs, debating implementations in meetings, waiting for review) as slow and often low-value, and proposes replacing it with dictating ideas by voice, having AI turn them into a spec, and having AI build directly from that spec — removing the wait for meetings and review.

  2. Nystrom proposes eliminating manager meeting-prep entirely by having a scheduled AI agent gather Slack activity, closed tasks, merged PRs, and prior meeting transcripts into a daily 'pre-read,' arguing this lets him work up to the minute before a meeting with zero prep.

  3. Both host and guest argue this automated pre-read 'democratizes' visibility: because the agent surfaces everyone's work regardless of who talks most, quieter but capable contributors get equal recognition.

  4. They argue managers staying hands-on with code (rather than spending most non-meeting time synthesizing reports) is a burnout-protection mechanism that AI newly makes feasible at scale, and extend this into a claim that engineering leaders at all levels should return to writing code — 'the era of the hard skill' rather than soft skills.

  5. Background agents on pre-provisioned VMs (Notion's internal 'Boxy' tooling) let engineers hand off small, loosely specified fixes (a few sentences plus a screenshot) via a comment mention and receive a working PR with self-tested verification minutes later, illustrated by a real 'copy link to tab' feature request.

  6. At the organizational level, Notion rewrote its own AI agent harness by first writing markdown specs (borrowing 'skills and progressive disclosure' from coding-agent design) rather than starting with code, and checked these specs into the repo as a living 'agent specs' library that Codex can then one-shot-build from and that also functions as the changelog.

  7. The source argues the engineer's role is shifting toward being a 'systems thinker and architect' whose main design responsibility is defining the verification loop (how correctness will be checked), since a hazy verification loop is treated as the root problem to fix before anything else.

  8. The source claims CI/pipeline speed is not a minor DevEx nicety but a 'true mathematical limit' on how much shippable value either humans or swarms of agents can produce, citing Stripe's ~1,300 agent PRs/week as evidence that slow CI makes high agent throughput pointless, and urges engineering leaders to allocate time to pipeline speed specifically because of AI.

  9. Nystrom offers a concrete counter-sycophancy prompting tactic — telling the model 'you're wrong, defend your argument' — as a technique for surfacing genuine reasoning instead of blanket reassurance, especially in domains (like CI internals) where he says he lacks deep expertise.

  10. Voice-to-spec-to-build pipeline — A workflow where the engineer dictates a feature idea into Whisper, hands the transcript to Codex to draft a written spec, then points Codex at that spec file and says 'build it.'. Apply: Start any new feature by voice-dictating how it should work, have an AI coding tool turn the dictation into a spec document, then have the same tool implement directly from that spec.

  11. Notion AI custom agent (scheduled) — A Notion AI agent configured to run automatically at a set time (e.g., 9am daily) rather than only on manual trigger. Apply: Configure a custom agent to run right after a recurring template (like a stand-up doc) is generated so it can populate it before anyone needs to prep.

  12. Sub-agents / map-reduce fan-out — Instructing a Notion AI custom agent to dispatch multiple sub-agents in parallel, each pulling from a different data source, then combining the results. Apply: Tell the agent explicitly to 'fan out' across sources (Slack, task database, metrics tool, meeting transcript) as separate sub-tasks rather than processing them sequentially in one pass.

  13. MCP (Model Context Protocol) integration — A connection standard letting an AI agent pull live data from an external tool, demonstrated via a Honeycomb MCP feeding CI metrics into the agent's context. Apply: Wire an MCP connection to whatever external system holds ground-truth metrics or data (e.g., an observability tool) so agent output reflects current numbers instead of stale training knowledge.

  14. Notion 'workers' — A separate Notion automation feature described as more code-like than the natural-language custom agents, mentioned but not deeply used by Nystrom. Apply: Reach for workers instead of a natural-language custom agent when the automation logic is closer to deterministic code than to a described process.

  15. Screenshot-as-spec configuration — Giving an agent a screenshot of a query/tool interface and asking it to figure out and update its own instructions from the image rather than typing out the underlying query. Apply: When you don't know or don't want to write out an exact query/config, hand the agent a screenshot of the relevant UI and ask it to infer and encode the needed instructions itself.

  16. Scoped agent permissions — Granting an agent view-only access to shared/company-wide databases but edit access only to the specific page/database it owns (e.g., the meetings database). Apply: When configuring any autonomous agent, default to read-only on shared resources and grant write access narrowly to just the artifact the agent is responsible for producing.

  17. Natural-language-editable workflows vs. deterministic workflow builders — A contrast between older rule-based automation tools and current AI agents whose behavior is changed by editing plain-language instructions. Apply: Prefer building automations as natural-language agent instructions so that changing the trigger, order, or data access is a quick text edit rather than a rule-logic rebuild.

  18. Boxy (internal VM/software-factory tooling) — Notion's internal fleet of VMs pre-installed with Codex and Claude Code, invokable from a task or comment (e.g., by @mentioning 'Codex') to autonomously implement a fix. Apply: Set up pre-provisioned dev-environment VMs with coding agents installed so a short task description (a few sentences plus a screenshot) triggered from a comment can produce a PR without the requester touching a local environment.

  19. Preview environment / preview URL on PR — An automated step where the agent's PR includes a live preview URL of the change alongside the code. Apply: Configure the CI/agent pipeline to deploy a preview build automatically so reviewers (or the requester) can click through to verify behavior without pulling the branch.

  20. 'I don't get this' as a code-review technique — Using a stated confusion or suspicion ('I've done my best to investigate and I'm pretty sure this is wrong') as a legitimate, low-effort review action that prompts the agent to explain or fix, or can be handed to a teammate. Apply: When reviewing agent-generated code you don't fully understand, respond with your genuine confusion or doubt as the review comment instead of trying to reverse-engineer full understanding yourself.

  21. 'Explain it like I'm five' prompting — A recurring prompt line asking the agent to explain its reasoning or a failure (e.g., a CI issue) in maximally plain, non-expert language. Apply: When working outside your expertise (e.g., CI internals), explicitly tell the agent you don't understand and need it explained at a basic level before accepting its fix.

  22. Background-agent-on-VM strategy — The practice of hooking coding agents to remote VMs so they can run tasks without needing a local dev environment spun up. Apply: Give engineers a standing background agent connected to a VM so any incoming task or fix request can be delegated immediately instead of waiting for a local environment setup.

  23. Skills + progressive disclosure — A design pattern borrowed from coding agents in which capabilities/instructions are only loaded/disclosed as needed rather than dumped into one bloated system prompt, adopted by Notion to fix 'tool and instruction fatigue.'. Apply: When an AI product's system prompt grows bloated with tools and instructions, restructure it into discrete 'skills' that are progressively disclosed to the model only when relevant.

  24. Agent specs subfolder / spec library — A checked-into-the-repo folder of markdown documents specifying how each part of an AI agent (e.g., Notion AI's 'ask mode') should behave. Apply: Maintain a version-controlled folder of plain-English specs per feature/mode so both humans and agents can read the current intended behavior and its edit history.

  25. Ask mode (restricted read-only agent mode) — A Notion AI mode that bans all mutating tools, permitting only reading and answering, defined as an example spec in the agent specs library. Apply: Define a named restricted mode for an agent (allow-list of read-only tools only) when you need guaranteed non-destructive behavior for a given context.

  26. Spec-driven development — A workflow in which a comprehensive spec (including code pointers and a verification section) is written first, then handed to a coding agent to implement in one shot, with the spec (not the code) treated as the primary artifact to edit going forward. Apply: Write specs detailed enough to include where relevant code lives and how to verify success, then instruct the coding agent to build directly from the spec; for future changes, edit the spec and let the agent resync the code.

  27. Custom CLI tooling for agent self-verification — Internally built command-line tooling that lets an agent run and query the product itself (Notion AI) — sending test queries, toggling modes, inspecting transcripts — to verify its own implementation. Apply: Before asking an agent to implement a spec end-to-end, build lightweight CLI hooks that let the agent exercise the actual product to check its own work automatically.

  28. Version-controlled specs as documentation + changelog — Treating the spec file's edit history in version control as both the current source of truth and the record of how a feature's behavior has changed over time. Apply: Update the spec (with a date) whenever behavior changes instead of only changing code, so the spec's git history functions as a human-readable changelog.

  29. Verification loop as core design deliverable — The claim that the most important part of writing a spec is defining how correctness will be verified, ahead of describing behavior itself. Apply: When drafting a spec, prioritize writing a concrete verification section (what to test/check) before or alongside the behavioral description, and revisit it first if anything is unclear.

  30. Parallel/round-robin agent management — Firing off multiple coding agents on different tasks simultaneously and checking back on them between other work rather than watching one agent iterate live. Apply: Kick off several agent tasks in parallel before a meeting or other work block, then review and steer each one when you return instead of babysitting a single agent session.

  31. One-shotting as a workflow goal — Deliberately aiming to get a complete, correct solution from an agent in a single pass rather than iterating live with it. Apply: Invest extra effort in the upfront spec/prompt so the agent can produce a finished result in one attempt, freeing you to move to other tasks immediately.

  32. Git work trees for parallel Codex sessions — Running many git work trees simultaneously (with local dev servers on sequential ports, e.g., 3000-3009) to support multiple concurrent Codex tasks. Apply: Use git work trees plus separate local ports per task when running several coding-agent sessions on the same repo at once so they don't collide.

  33. Codex 'projects' concept — An organizing feature in Codex for grouping work by project, used as a mental model when juggling many different projects. Apply: Structure agent work using the tool's project concept to keep context and history separated across the different codebases/initiatives you run.

  34. Using Codex as automated code/security reviewer — Applying Codex specifically to review pull requests for correctness and security issues, valued for being tireless with tedious review work. Apply: Route PRs through Codex as a first-pass reviewer for both general code quality and security concerns before or alongside human review.

  35. Fast CI/CD as a throughput multiplier — The claim that CI pipeline duration directly caps how many iterations (human or agent) can be completed, illustrated by contrasting a 1-hour vs. 3-minute CI loop. Apply: Treat CI runtime reduction as a direct lever on both human and agent iteration speed, and prioritize it before scaling up agent usage.

  36. Counter-sycophancy prompting — Explicitly telling the model 'you're wrong, you need to defend your argument' to force cited, specific reasoning instead of a reflexive 'it's totally fine' response. Apply: When you suspect a model is just agreeing with you or reassuring you without justification, push back directly and demand it defend its position with concrete evidence.

  37. Escape/interrupt-and-steer — Interrupting an agent mid-generation (hitting Escape and saying 'no') to redirect it rather than letting it finish an unwanted direction. Apply: Interrupt an agent as soon as you see it heading the wrong way rather than waiting for it to complete and then correcting afterward.

Insights

The claimed payoff of the automated stand-up agent is reframed as roughly 20 minutes/day of literal time saved, but the more important benefit is described as protecting cognitive bandwidth from context-switching to ingest scattered updates, not raw time savings.

Nystrom claims natural-language agent workflows are cheaper to iterate on than older deterministic workflow-builder tools, since updating behavior just means rewriting the instructions rather than reconfiguring rule logic.

He describes configuring an MCP-based agent by handing it a screenshot of a query interface rather than typed data and asking it to reverse-engineer its own instructions from the image.

He contrasts Codex and Claude Code not just on capability but on 'personality' fit to task: Codex's blunt, non-affirming tone is described as well-suited to 'explain it to me' debugging use, while Claude Code's warmer tone suits a different mode.

He claims Codex tolerates long-running, context-heavy tasks ('can grind for hours') better than Claude Code, which he says 'loses the plot' once its context window fills — offered as a first-person technical comparison, not a benchmark.

The source claims that at points in his career, roughly 70% of his job consisted of gathering the same information and reformatting it for different audiences (exec vs. team versions), framing this as the specific tedium AI removes.

Version-controlled specs are presented as having a secondary audience beyond engineers — plain-English specs are described as usable by non-engineers (e.g., for marketing/release notes) in a way raw code is not.

The source ties fast CI to engineer psychology, not just agent throughput: it claims slow CI causes engineers to batch changes into 'monster changes' reviewed more cautiously, whereas fast CI enables a tighter ship-feedback-iterate loop that predates agents and is merely amplified by them.

«I literally don't know what I'm doing here. You got to explain it like I'm a five-year-old.»

— 00:01

«I basically one-shotted this.»

— 00:26

«no more waiting for the meeting, no more waiting for review.»

— 00:38

«Your AI, your agent is never going to complain when you ask it to do this five minutes before the meeting starts.»

— 00:53

«It is more relaxing and it's more fun and I feel like I'm getting more done. It's weird to have this like win-win-win.»

— 00:58

«Give me the whole triangle.»

— 01:07

«I also think it sort of democratizes that like sharing.»

— 11:44

«I am such an introvert. I'm a crazy introvert. I will avoid humans with every fiber of my being.»

— 12:16

«this is the era of the hard skill.»

— 15:56

«That's like that was I feel like 70% of my job.»

— 22:25

«This is one, two, three paragraphs, four like four sentences and a screenshot. Not a lot.»

— 28:01

«Let's not start with code. Like let's just start with specs.»

— 32:48

«Build it. And it basically one shotted this because the entire spec file is so comprehensive.»

— 34:16

«The spec is the source of truth, the spec as the change log.»

— 36:11

«No more waiting for the meeting, no more waiting for review. Ship it, have a verification loop.»

— 38:41

«it would just kind of like lose the plot really quickly»

— 40:23

«it's just like a really good code reviewer. It's a really good security reviewer.»

— 42:13

«you might as well be throwing all those PRs in the trash.»

— 44:36

«there is just a true mathematical limit on your capacity to ship code to production that is a reflection of how fast your CI pipeline is.»

— 44:45

«So allocate time to your pipeline, please.»

— 45:13

«you're wrong. Like you need to defend your argument.»

— 45:56

«Escape all caps no.»

— 46:46

Reception

Reception is mixed, with substantive skepticism toward spec-driven/AI-generated workflows balanced by a few neutral or appreciative comments.

The video is a detailed first-person practitioner account of agentic-AI workflows inside Notion — naming specific internal tools (Boxy, custom agents, agent specs) and concrete prompting/permissioning techniques rather than staying abstract — but every efficiency and quality claim rests on the guest's own anecdotes and is not independently demonstrated on screen.

47:53

↳ How I AI · YouTube

Watch original