Lore

Music Track as Model Input for Choreography Sync

Instead of describing rhythm or tempo in words, the actual audio track is embedded directly into the generation prompt as an input, with an instruction that the character's movement (e.g., a dance) should follow its beat. This sidesteps the unreliability of text-described timing — the model has the literal waveform to lock onto rather than an approximation like "upbeat" or "fast-paced."

Pairs with move-by-move-choreography-scripting-style explicit move sequencing: the named moves give the what, the embedded track gives the when. Also complements Blocking Lock via Position Reference (a schematic/map fixes where) as another case of feeding the model a literal reference artifact instead of relying on text to encode something text is bad at conveying — timing, in this case.

Apply: When a scene requires movement synced to music, attach the actual audio file as a prompt input and instruct the model to time the action to its beat, rather than writing a text description of the rhythm.