Lore

Hypothesis-First Approach to Consumer Science

Overview

Before running a consumer-science experiment, state an explicit, falsifiable hypothesis for why it should work — e.g., Gibson Biddle's Netflix example: "my hypothesis is that news will work because we're about educating and informing." This turns the resulting data into a test of a stated belief rather than an open-ended fishing expedition, and it applies specifically to the consumer-science leg of Three Forces of Inventing the Future (Consumer Science, Strategy, Culture).

Apply: before launching a new product test, write down the specific reasoning for why it should work so the resulting data can confirm or refute that stated belief, not just report a number.

Netflix Example: Social Movie Recommendations

Social/friend-based movie recommendation features failed repeatedly at Netflix, even though the same social mechanics work well for music and books. The hypothesis was falsified for two concrete reasons: "your friends have sucky movie taste," and people don't want their viewing habits (e.g., binge-watching Cake Boss) visible to friends. As Biddle put it, "the list of failures is equal to the successes and it really points out how hard that consumer science is" — a working hypothesis-first discipline still produces roughly as many disproven hypotheses as confirmed ones.

Applying Consumer Science to Yourself

Biddle closes his own talks by having the audience scan a QR code to a SurveyMonkey survey: rate the talk 0–10, name one thing they liked, and name one way to improve it. He frames this as applying the same hypothesis-first consumer science to himself — collecting quick structured feedback after a talk or release to iterate the next version, rather than assuming it landed.