A framework for choosing which AI model tier to route a task to, combining Pareto efficiency (price vs. performance) with the task's upside potential.
Core split: functions with bounded upside (e.g., closing the books, routine admin) should run on cheap, RL-tuned mid-IQ open-weight models; functions with unbounded upside (sales, research, drug discovery) justify frontier-model cost, since a stronger model occasionally produces a disproportionately valuable result.
Complication — upside is hard to calculate in advance: verifiability alone isn't a sufficient routing signal, because a seemingly low-stakes, bounded interaction (a routine support call) can turn out to be the first thread to a company-changing insight. The heuristic is therefore two-dimensional: how large is the plausible upside, and how confidently can that upside be estimated beforehand — not just whether the output is easy to verify.
Related to Loop Engineering (this is the model-selection layer inside a cascading loop) and to Intelligence Is Not the Bottleneck (bounded-upside tasks are exactly the ones where more intelligence doesn't move the needle).
Source: Anish Acharya (a16z), "Why companies are becoming a series of loops."
A named tactic from Hamel Husain and Shreya Shankar's evals course: once Error Analysis (Manual Trace Review) and a validated eval have established a task's quality bar, test whether a cheaper model variant (e.g. swapping a 'nano' or 'mini' model in for a full GPT-5-class model in part of the pipeline) still clears that bar, and substitute it in to cut cost. The eval is what makes the swap safe — without a validated quality bar, cost substitution is just guessing.