Lore

AI agents

5 Rules for Building AI Agents That Work in Production | Nan Yu & Jacob Shumway

Building production AI agents works best when you minimize explicit prompting and instead give the model minimal instruction plus tools to load its own context and goals, letting it reason autonomously in a loop — a thesis the video grounds end-to-end in Linear's own internal agent, which evolved from a hacky prototype into a production system built on dynamic 'skills' loading, a small set of guaranteed 'hero' use cases, and multiple non-chat entry points into users' existing workflows.

Peter Yang · 2026-08-09 · English

Key ideas

  1. Central argument: the bottleneck in applied AI isn't model capability but underuse — overprompting makes results worse, so the proposed fix throughout is minimal instruction plus context-loading tools, letting the model reason toward a goal rather than following a scripted prompt.

  2. Origin narrative as evidence: Linear's agent began as a scrappy internal experiment meant to automate 'boring' work, and its scope expanded organically once real usage (sales-call note dumps, vague Slack instructions) revealed unanticipated demand — used to argue that products should ship early and let real usage define scope rather than upfront specification.

  3. The team distinguishes the engineering definition of 'agent' (LLM-in-a-loop) from the product definition (packaged subsystems behind a facade), framing production agent-building as needing to solve both at once.

  4. Architectural lesson drawn from failure/success contrast: exposing the model to everything (all actions, raw GraphQL schema) caused hallucination and overload, while narrower, dynamically-loaded scopes succeeded — used to argue agent quality comes from curation, not raw capability.

  5. Evaluation philosophy pushes back on maximal eval coverage: over-evaluating agent behavior where variance is legitimate produces false signals, so evals should target only where consistency genuinely matters, and are best sourced from real dogfooding feedback rather than designed upfront.

  6. Product accountability argument: agent output must be correctly filed into the existing system of record (right project, status, assignee), not just produced, because otherwise it becomes undiscoverable — extending traditional software ownership/traceability norms into agentic work.

  7. Reliability argument specific to agent products: since an agent's behavior space can't be scoped the way a UI is, the proposed quality bar is to guarantee a small set of 'hero' paths (create a ticket, manage tickets, create a PR) while tolerating variance elsewhere — an explicit departure from conventional 'eliminate all unintended behavior' QA thinking.

  8. Strategic argument for native domain agents over generic chatbots: decomposing users' actual end-to-end workflow to find integration 'entry points' is what justifies a native agent's existence, since a chatbot interface alone gives no reason to prefer it over Claude or ChatGPT.

  9. Business-risk framing: leadership reportedly isn't worried about MCP/agent usage cannibalizing the polished UI because the product is inherently 'multiplayer,' with different user types (e.g., support staff via Zendesk/Intercom) legitimately using only a narrow surface — argued as proof that being 'opinionated' about product doesn't require restricting access points.

  10. Forward-looking claim treated as a working assumption: the human-owned 'edges' of agent-assisted work will keep shrinking as models improve, so the stated roadmap (proactivity, longer-running memory) is designed around the agent progressively owning more of a project's full lifecycle.

  11. LLM-in-a-loop — Jacob's core definition of an agent as an LLM called repeatedly in a loop rather than once, per [01:01]. Apply: Architect a loop where the model repeatedly calls tools and reasons, instead of a single one-shot prompt/response exchange.

  12. Goal-driven context-building loop — The pattern of defining a goal/milestone, giving the agent tools to build its own context, then looping question-answer turns until enough information exists to synthesize a final response [01:17]. Apply: Define the target outcome rather than pre-loading context into the prompt, and let the agent pull in what it needs turn by turn.

  13. Skills architecture — A meta-tool that lets the agent dynamically load task-specific 'skills,' each bundling its own tools and instructions and encoding the team's opinions, e.g. how to set issue priority [10:08]-[10:29]. Apply: Group product actions into narrow, loadable skill bundles the agent requests only when relevant, instead of exposing every action at once.

  14. Native agent architecture — Building the agent directly into the product rather than as a CLI or MCP server with skills bolted on, enabling hundreds of dynamically loaded skills with tight, opinionated control [10:43]. Apply: When a product has many surfaces and strong opinions to enforce, build the agent natively rather than exposing generic tool access via a separate protocol.

  15. Router pattern — A small router model that classifies incoming requests and dispatches roughly 80% of early use cases to one highly optimized issue-creation subprompt [16:24]-[16:35]. Apply: Route common, well-defined intents to a narrow specialized prompt instead of always invoking the full general-purpose agent.

  16. Two-tier model routing — Combining a small router model with a big general-purpose model/prompt for broad, open-ended requests, and narrow prompts for known specific use cases [16:46]-[16:58]. Apply: Size the model to the task: use a lightweight model for classification/simple cases and reserve the large model for genuinely open-ended reasoning.

  17. Biggest model first, then optimize down — The practice of starting development with the largest available model, building evals and success criteria, then downsizing the model once performance is validated [17:12]-[17:32]. Apply: Prototype and validate feasibility with the most capable model, then swap in smaller/cheaper models once evals confirm they still meet the bar.

  18. Mixed objective/subjective evals — Combining deterministic checks, e.g. a status field always set correctly, with subjective quality checks, e.g. whether a description is well-structured [18:09]-[18:34]. Apply: Write deterministic assertions for facts that must always hold, and pair them with subjective scoring for open-ended quality dimensions.

  19. LLM-as-judge — Using a scoring LLM, built from an accumulated dataset, to judge subjective qualities like whether the right information was extracted [18:39]-[18:52]. Apply: Build a judge model/prompt from real examples to score subjective outputs at scale, while manually spot-checking the judge's own accuracy.

  20. Consistency-scoped eval principle — The claim that evals work best when enforcing consistency where it genuinely matters, since over-evaluating high-variance agent behavior produces false signals [19:07]-[19:29]. Apply: Before writing an eval, ask whether consistency is actually required for that behavior; skip evaluating dimensions where legitimate variance is acceptable.

  21. Self-assessed interjection check — An internal decision step where the agent evaluates whether it should proactively answer/interject in a conversation, e.g. Slack, rather than always responding [22:48]-[23:03]. Apply: Give the agent an explicit 'should I act now?' gate before autonomous or unprompted actions to avoid over-eager, unwanted behavior.

  22. Capability-gap feedback loop — When a user asks for something the agent can't do, it calls a tool that reports the gap, which is auto-ingested into an existing or new tracked issue [25:08]-[25:31]. Apply: Instrument the agent to self-report missing capabilities as structured signals that automatically feed the product backlog.

  23. Minimal-instruction design principle — The recurring principle of giving the agent as little explicit instruction as possible and instead giving it tools to load context itself, since over-instruction causes overfitting on unimportant details [00:07], [24:16]-[24:52]. Apply: Write prompts around goals/principles rather than exhaustive rules or micromanaged formatting instructions, and rely on the model to fetch what it needs.

  24. Human-attribution/accountability pattern — The default practice of assigning agent-created work to a human owner, except for fully system-triggered cases like a monitoring alarm the agent handles end-to-end [14:56]-[15:38]. Apply: Attribute agent output to a responsible human by default so work stays owned and discoverable, reserving fully autonomous handling for well-defined automated triggers.

  25. System-of-record placement principle — The design principle that an agent's goal isn't just completing a task but correctly filing the result into the existing tracking system, i.e. right project, status, assignee [14:16]-[14:28]. Apply: Require agent actions to terminate in the system of record's normal structures, not just in ephemeral chat output, so results stay discoverable.

  26. Hero use cases / 'hot paths' reliability framework — Identifying a small set of core, heavily demoed use cases, e.g. create a ticket, manage tickets, create a PR, that must be made 'rock solid,' while accepting variance elsewhere as inherent to the technology [32:41]-[33:44]. Apply: Define and harden a small number of guaranteed 'hero' workflows as your reliability bar, rather than trying to make all agent behavior uniformly reliable before shipping.

  27. Workflow decomposition / entry points ('on-ramps') — The method of breaking down users' actual end-to-end multi-step workflow to find natural integration points, since 'no one sits down and one-shots their whole job' [34:07]-[35:07]. Apply: Map out where work already happens, e.g. Slack, meeting notes, project docs, and hook the agent into those moments rather than routing everything through a single chatbot.

  28. 'Collection of subsystems' framing — Describing the agent not as one chatbot but as multiple integration points/subsystems, with the in-app chatbot as just one necessary-but-non-primary way to interact with it [34:41]-[35:07]. Apply: Design an agent product as a set of distributed entry points plus a chat interface for follow-ups, rather than centering the product on a single chat window.

  29. Third-party escalation/plug-in pattern — An integration pattern where external user types, e.g. customer support, interact with the system entirely through a plug-in surface like Zendesk or Intercom, without touching the core UI [36:29]-[36:52]. Apply: Let peripheral user types interact via lightweight plug-in surfaces embedded in their own tools, since that narrow usage still feeds valuable input to the rest of the system.

  30. Proactivity + longer-running memory — The stated next development direction — agent awareness that persists across a project's full lifetime, days to a quarter, so it can proactively coordinate people and keep documents current [37:12]-[37:55]. Apply: Extend an agent's memory/awareness horizon beyond single interactions to an entire project lifecycle so it can push work forward without being re-prompted each time.

  31. MCP (Model Context Protocol) — Referenced as the alternative integration approach to building a native agent, exposing tools for external general-purpose agents to connect to, contrasted with Linear's native-agent choice [34:00], [31:38]-[31:51]. Apply: Consider MCP when you want to expose your product's actions to third-party general-purpose agents, while recognizing it won't embed your own product opinions the way a native agent can.

Insights

The 'lazy prompt' emergent behavior — users typing vague commands like 'linear do the right thing' or a single emoji and having the agent correctly infer intent — arose naturally from the context-access architecture, surprising even the builders and serving as unplanned validation of the minimal-instruction thesis.

The system can be fully self-triggering with no human in the loop at all (e.g., a monitoring alarm causes Linear to file and attempt to fix its own bug), revealing that the stated 'agent work gets attributed to a human' rule is actually a spectrum rather than a fixed policy.

Underneath the 'smart autonomous agent' narrative, 80% of early usage was funneled through a small router to one narrow, hyper-optimized issue-creation subprompt — a mundane, deterministic reality behind the flashier framing.

The in-app chatbot, despite being the most visible and marketed surface, is explicitly described by the team as secondary — the real value comes from entry points embedded where work already happens (Slack threads, meeting debriefs, project docs), a claim that runs counter to where chat-agent marketing usually puts its emphasis.

A single incident — a user addressing the agent as 'dude' and the agent refusing to respond over informality — illustrates how quickly narrow behavior-tuning gets bolted on in reaction to individual feedback signals.

The host's own admission that he now trusts the agent's polished markdown plans enough to stop reading them is offered as a live example of the 'slop' risk the team says it is trying to prevent by embedding its own product opinions into the agent.

Capability gaps are turned into a self-perpetuating pipeline: when the agent lacks a tool for a request, it reports that via a tool call, which is auto-ingested into the backlog (deduped against an existing issue or filed as new) — making part of the roadmap effectively self-authored by the agent's own limitations.

«The models are really smart, but we're just not using them enough.»

— 00:00

«If you overprompt these things, you're more likely than not going to make it worse.»

— 00:03

«Give it as little instruction as possible. Give it the tools to load context. Don't give it context.»

— 00:07

«Here's all the context. Just figure out what the right thing is. Write the issue and work on it.»

— 00:13

«Computers can do a lot of work for us. So let's get rid of all the work we don't want to do and give it to computers.»

— 00:25

«At a high level, an agent is really just calling an LLM in a loop.»

— 01:01

«Slack by itself isn't enough. But if you combine Slack with the ability to read your codebase with the ability to read your project description and your other tickets and stuff like that, then all of a sudden you can do something.»

— 15:29

«When you think about what is linear's goal, its goal is to like put a ticket in the right place and also accomplish the task.»

— 14:16

«I think evals are most successful when you're ensuring consistency somewhere that consistency is important. But consistency is not always important for agents.»

— 19:16

«we had a user call the agent dude once and then the agent was like, 'Oh, okay. I'm not going to respond to you because that was like you're not being formal with me.'»

— 23:47

«you also just want to like give it as little instruction as possible to be honest. Give it the tools to load context. Don't give it context.»

— 24:18

«AI is a it's not an endto-end solution it's a middle's solution.»

— 29:16

«it just turns to slop, dude.»

— 30:35

«no one no one like sits down at their desk and oneshots their whole job. Like that that's not that doesn't happen, right?»

— 34:19

«an agent is almost like a employee and then like employee doesn't only work in one app, right? You should talk to them from like, you know, Slack or wherever you guys work.»

— 35:44

«I run a project in linear. I can take for granted that it's wellrun, right? Like that's that's the that's the goal that we're looking for.»

— 37:55

Reception

Mostly thoughtful and appreciative engagement with the agent-building content, tempered by a couple of complaints about a slow, rambling intro.

A credible, detail-rich practitioner interview that maps out real architectural and product tradeoffs behind a shipped production agent (skills loading, routing, eval philosophy, hero-use-case reliability), though it remains a promotional showcase of Linear's own product with an embedded sponsor segment.

38:27

↳ Peter Yang · YouTube

Watch original