Lore
Florence-2 detailed captioning
A Florence-2 vision-language model, loaded through a custom node that downloads and loads it, is used to turn the input image into a text description. In Blender to ComfyUI SDXL image-to-image render pipeline this caption seeds the prompt, so no one has to type a description of the scene by hand.
How to wire it
- Connect the prepared image (sdxl-input-image-preparation) to the Florence caption node.
- Set the task to a detailed caption.
- Pass the output text to the LLM rewrite step (llm-caption-rewrite-for-render-prompts).
Notes
- The caption of a flat, shadowless render describes a flat scene, not an attractive one. Rewriting it is therefore a required next step.
- The caption is one of the places colour information enters the prompt (Redundant colour carry-through in an image-to-image pipeline).
- The Florence nodes may need manual installation from GitHub rather than the ComfyUI Manager.