Lore

Redundant colour carry-through in an image-to-image pipeline

The artist keeps control of colour in Blender because the pipeline passes colour information through several independent channels. Structure is handled separately by the ControlNet maps.

Colour channels

  1. Caption: the detailed caption names the colours it sees (florence-2-detailed-captioning).
  2. Rewrite instruction: the LLM is told to 'keep the important information about the colors' (llm-caption-rewrite-for-render-prompts).
  3. Image-to-image latent: the input image is encoded and partly noised, so its colour layout seeds the result (img2img-denoise-tuning).

Structure channel

Depth and canny ControlNets carry geometry and outlines (depth-and-canny-controlnet-maps-from-input, multi-controlnet-stack-strength-tuning).

Rationale

The redundancy is intentional. He says it is 'Never Enough we're working with AI': any single channel can be ignored or distorted by the model, so several are stacked.

Failure mode

If the model takes too much freedom with colours, as in his demo, the first lever to try is a lower denoise. That keeps the output closer to the input latent.

See also Blender to ComfyUI SDXL image-to-image render pipeline, Component-level color naming (true color accuracy).