A widespread misconception, per Adam Mosseri (Instagram), is that recommendation algorithms have a detailed semantic understanding of user interests — that the system 'knows' a user likes surfing the way a human would describe it. In reality, systems like Instagram's have long relied on illegible embedding vectors: "a big ass number that correlates with surfing," opaque even to the engineers who built the system. What's changed is that LLMs can now translate these opaque vectors into natural-language descriptions, making previously illegible model internals legible after the fact — not because the underlying representation changed, but because a translation layer was added on top. Related to Seeing Like a State: Legibility Critique (illegibility as a property system designers often can't avoid).
Instagram's 'your algorithm' feature is a concrete application: an LLM translates a user's previously opaque embedding vector into natural-language interest labels — Mosseri's example is 'deep pourover coffee snobbery' — so users can see, and edit, what the ranking system believes they're interested in.