A methodological preference for measuring the share of users whose engagement crosses a defined threshold, rather than an average across all users, when trying to gauge broad delight.
Averages are distorted by a small number of heavy users — Gibson Biddle: "the problem with average is you could have a thousand freaks that watch 40 hours in a month but that's not really helpful." Netflix instead tracked things like the % of members watching ≥30 hrs/month, and — for the original 2007 streaming launch — the % of members who watched at least 15 minutes of a stream in a month. 15 minutes was chosen because it was the length of the shortest TV episode: the smallest unit of delight a member could experience.
Apply: when a proposed success metric is an average, ask whether a small cohort of outliers could be driving it, and replace it with a "% of users past threshold X" metric instead.
See also DHM Framework (Delight, Hard-to-Copy, Margin-Enhancing) (Delight is what this metric is trying to detect).
Tom Verrilli frames this at Whatnot as a general instinct to distrust averages: an aggregate usage metric can look mediocre while hiding a feature that is 100% of the use case for a smaller subgroup. He pairs it with a warning that "it's too easy otherwise for growth to hide all sins" — top-line growth numbers can mask exactly this kind of subgroup-level problem or opportunity.
Tom Verrilli (Whatnot CPO) names this the mistake he's "really scarred by": using an overall usage percentage to justify deprecating a feature, when that percentage can hide a small subgroup for whom the feature is 100% of their use case. A feature used by only 3% of users, by average logic, looks safe to cut — but for that 3% it may be the entire reason they use the product, and the fallout compounds outward (their own downstream activity, referrals, content) rather than staying contained to that slice.
Quote: "averages mean nothing to the individual is probably the thing that I've like really scarred by."
Related: Bezos Anecdote-Over-Data Rule (the credible-individual-account version of the same blind spot).
Из тем: Netflix, Consumer Science, and Strategy Storytelling