Lore
LoRA (low rank adaptation) for targeted improvement in the SDXL render graph
A LoRA ("low rank adaptation") is a small add-on trained on top of a main model to teach it one specific task. In the Blender to ComfyUI SDXL image-to-image render pipeline it is the way to patch a weakness of the base checkpoint without switching models.
What it does
- Improves one narrow area, such as nice interiors, or clouds if the base model renders them poorly.
- Also adds extra prompting. Using a LoRA changes what the prompt contributes, so the prompt may need adjusting.
- Stefan uses an interior LoRA made for SDXL 1.0 because his test scenes are real-estate interiors.
How to apply
- Pick a LoRA built for the same base model as the checkpoint, here SDXL 1.0. See addon-base-model-compatibility-civitai.
- Read its description to see what it improves.
- Load it between the checkpoint and the sampler. See Juggernaut XL checkpoint and the image-to-image sampler graph.
- Compare runs with the same seed and unchanged input, so the LoRA is the only variable. See K sampler settings: CFG, steps and seed.
SDXL is relatively old, so many LoRAs and ControlNets already exist for it. That is the reason it was chosen. See Choosing the base model by add-on ecosystem (SDXL vs Flux vs SD 3.5).