Lore

Flux & ComfyUI: Local Model Tuning

Из Read: AI Photo Editing

This chapter is the local, node-based track of the toolkit: Flux run through ComfyUI (or hosted inference), where an edit is tuned by turning numbered inputs rather than by painting masks or rewriting prompts. It covers what a LoRA is and the three jobs LoRAs do here — photorealism, 8-step speed, and your own trained likeness — plus the Flux Tools family's unusual model architecture, the Model Sampling Flux node's Max/Base Shift and Width/Height controls, the batch-sweep workflow that makes tuning systematic instead of anecdotal, and the Detail Daemon sampler for pushing fine detail. Several of its numbers are empirical or explicitly unconfirmed, and the chapter says so where that is the case.

Why this track is different: the knobs are numbers

Everywhere else in this book an edit is a decision you make with your hands and your eyes — a mask boundary, a slider, a prompt, a layer stacked on another layer. The Flux/ComfyUI track works differently. Here the image is produced by a graph of nodes with typed inputs, and the interesting work is choosing values for those inputs: a LoRA weight, a guidance scale, a shift value, a step count. That makes the work more reproducible than prompt-wrangling — the same seed and prompt with a different number is a controlled comparison — and it makes it more tedious, which is why a chunk of this chapter is about automating the comparison rather than eyeballing it.

The foundational unit is the LoRA. A LoRA (Low-Rank Adapter) is a small fine-tuning add-on — roughly 2 to 500MB — layered on top of a foundation model like Flux to target one specific improvement: a style, character consistency, photorealism. It is not a retrained model; it is a patch applied to one. That size is what makes the whole approach practical: you can keep a stack of them and swap for the job at hand, as Flux Realism LoRA & Guidance-Scale Tuning for Photorealism describes for photorealism specifically.

One consequence is worth stating up front because it decides which tools you can even consider. LoRAs can only be applied through LoRA-capable tooling — ComfyUI's node graph, or hosted inference on fal.ai. A generator without LoRA support (Glif is the named example) cannot reach this level of realism regardless of how good the prompt is. That is a hard capability boundary, not a matter of skill, and it is the practical reason to keep a local or hosted-Flux route open at all.

The second foundational lever is a single node. Model Sampling Flux (ComfyUI node) sits between the model/LoRA loader and the KSampler and changes how Flux renders, independent of seed and prompt. Between LoRAs and that node, nearly everything in this chapter is a variation on the same move: hold the creative decision fixed and change a number. Note that this track is genuinely self-contained — the compositing craft and the finishing stacks that make an image sit believably in a scene live in Manual Compositing Craft: Light, Color & Depth and Generative Fill & AI Compositing Toolkit, and this chapter does not attempt to bridge to them.

Photorealism: the realism LoRA and the guidance number under it

The flux-realism LoRA (from Excel lab/XLabs) is the targeted answer to Flux's tendency toward synthetic-looking people: it specifically adjusts how skin, hair and wrinkles are rendered, pushing output toward photorealism. It is hosted on fal.ai alongside flux-1-pro and flux-1-dev, and it is not free — roughly $0.32 per generation against about $0.05 for flux-pro. Realism, on this route, costs about six times a plain generation.

But the LoRA alone does not get you there, and this is the part most easily missed. Realism also depends on the guidance/CFG scale. fal.ai's default of 3.5 at 28 inference steps produces shiny, plasticky skin — the exact artifact you brought the realism LoRA in to fix. Lowering guidance to around 2 removes that artifact, and ~2 is the reported sweet spot for realistic-looking skin. In other words the realism failure people blame on the model is often a parameter left at its default, and the fix is one number. The full detail on both halves of that pairing is in Flux Realism LoRA & Guidance-Scale Tuning for Photorealism.

There is one non-numeric cue in this otherwise numeric section, and it is a compositional one: off-center, 'amateur snapshot' framing reads as more authentic than polished professional-photo composition. A too-perfect composition now paradoxically signals AI generation. This inverts the instinct the rest of the book trains — that the goal is a professional-looking frame — and it is worth holding as a live tension rather than a rule, since it applies to convincing someone an image is a photograph, not to making an image look good.

Two other jobs for a LoRA: speed, and your own face

Realism is one use of the LoRA mechanism; the chapter carries two more, and they are worth seeing side by side because they are the same mechanism aimed at completely different problems.

The first is speed. Flux Turbo Lora (8-step workflow) is a LoRA that lets Flux produce usable images in as few as 8 sampling steps. The compressed step count buys you a much faster iteration loop and costs you some quality and detail — which is precisely why it is discussed here rather than in isolation. The recommended response to Turbo's detail loss is not to abandon Turbo and revert to a slower non-Turbo workflow, but to claw the detail back with the Max Shift/Base Shift controls on the Model Sampling Flux node (see Max Shift / Base Shift). Load the Turbo LoRA with the Flux model, generate fast, then fix detail with a parameter instead of with steps.

The second is likeness. Flux Personal LoRA Training via Replicate covers training a personal Flux LoRA — teaching the base model one specific person's face — directly on Replicate.com using the 'AI toolkit for flux Lora training' model by Replicate contributor Luca Taco. This replaces the older Dream Booth / Stable Diffusion 1.4 workflows people used to run in Google Colab. The practical requirements:

One prompting note comes with an explicit caveat attached: placing the trained trigger word as the very first word of the prompt appears to produce more reliable inclusion of the likeness than burying it mid-prompt. The concept body flags this as an informally observed, unconfirmed effect — not documented model behaviour — and it should be carried that way rather than repeated as fact. A finished image/prompt pair from this workflow can be fed into Runway Gen 3 as a first frame to animate it.

Flux Tools: four models that refuse to behave like a family

Black Forest Labs ships four Flux Tools models, and the trap is assuming they are variants of one thing. They are not. Each is a distinct model type with its own installation folder and its own conditioning mechanism, and getting the type wrong means the file lands in the wrong directory and the wrong knob controls its strength. Flux Tools (Depth/Canny/Redux/Fill) Model Architecture & Placement lays out all four; the shape of it is:

Two things about this are genuinely surprising and worth memorising. Depth and Canny being LoRAs rather than ControlNets changes both where the file goes and which slider tunes strength — a LoRA-weight slider, not a ControlNet-strength slider. And guidance has no single family-wide default: Fill needs roughly three times the guidance of Depth or Canny. Even LoRA weight is tuned in opposite directions between two models that otherwise share a workflow — up past 1.0 for Depth, down below 1.0 for Canny. This material comes from the "FLUX TOOLS - Run Local" tutorial, filed 2026-08-06.

Model Sampling Flux: re-rendering without re-rolling

The most distinctive lever in this chapter is a single ComfyUI node. Model Sampling Flux (ComfyUI node) is inserted between the model/LoRA loader and the KSampler, and it changes how Flux renders an image independent of seed and prompt. Its placement relative to the LoRA does not matter — before or after behaves the same. It exposes four controls: Max Shift, Base Shift, and Width/Height. The value of this is that it breaks the usual all-or-nothing loop: instead of re-rolling the seed and losing the composition you liked in order to fix one bad detail, you keep seed and prompt fixed and change how the model renders them.

The primary swept value is Max Shift, with Base Shift typically held constant as a baseline — 0.5 is the example given. Changing them alters fine detail and can shift overall composition, while colour palette and broad composition tend to stay consistent, which is exactly the behaviour you want from a fix-one-thing control. Max Shift / Base Shift records the demonstrated fixes: correcting a fish's shape, seating a tail correctly behind clothing instead of clipping through it, and making a facial expression read as more 'sinister' — all without touching seed or prompt. A workable starting move is to hold Base Shift at 0.5 and sweep Max Shift across roughly 0.75–1.75. The values can also be inverted, with Max Shift set lower than Base Shift, for different and sometimes useful results.

The node's Width/Height settings are the part most likely to mislead. Despite the naming they are not the resolution of the output or latent image; they affect how the model renders internally, and Model Sampling Flux Width/Height is explicit that the exact mechanism is unconfirmed even by the source demonstrating it. What is observable is the interaction: raising Width/Height narrows the usable range between Max Shift and Base Shift before output quality degrades. At 2048x2048 there is far less headroom to play with than at 1024x1024, which is why 1024x1024 is the recommended default — it gives the widest practical range to experiment inside. Raise it only when you are deliberately narrowing that range or exploring how it changes fine detail, and expect to need a tighter sweep when you do.

There is no single ideal configuration for any of this. The concept bodies are consistent on that point: tuning here is empirical, which is a polite way of saying you have to test it — and testing it by hand, one value at a time, is where the next section comes in.

Sweeping a parameter instead of guessing at it

If a parameter has no correct value and must be found empirically, the honest workflow is to generate the whole range at once and look at it as a grid. ComfyUI Batch Parameter-Sweep Workflow is that workflow, and it originated for exactly the problem the previous section leaves you with — finding a Max Shift value — but the pattern is general.

The construction has four moves. Convert the target parameter into a node input rather than a typed-in constant. Drive it with a float-range node that steps across a sequence — 0.75 to 1.75 in 0.1 increments is the worked example. Convert each resulting float to a string. Then overlay that string as text onto the corresponding output image, extending the canvas by roughly 50px to make room for the label.

That last step is what makes it a workflow rather than a batch job. A grid of twelve near-identical images is useless if you cannot tell which one was 1.15; a grid where each image carries its own value burned into it can be compared visually with no manual bookkeeping and no risk of mislabeling the winner. And because nothing in the pattern is specific to shift values — float range, string conversion, text overlay — it transfers to any numeric ComfyUI parameter you want to characterise, including the guidance and LoRA-weight numbers from earlier in this chapter.

Detail Daemon: pushing detail through the noise schedule

The last lever in this chapter reaches into the sampling process itself. Detail Daemon Sampler (ComfyUI Detail Control) is a ComfyUI custom node and sampler that controls how much fine detail is added to a Flux, SDXL or SD1.5 generation by adjusting the noise schedule during sampling — again independent of prompt or model choice. Installation goes through ComfyUI Manager: copy the node's GitHub URL, choose "Install from git URL", restart ComfyUI. It ships with example workflows, and the "comparing detailers" one is the recommended starting point. The repo's own notes document every sampler setting plus suggested value ranges, and those ranges differ by model — Flux, SDXL and SD1.5 are not interchangeable here.

Detail can be weighted toward the start of generation, producing bigger and rougher details, or toward the end, producing smaller ones, via a strength parameter and schedule. Raising strength follows a predictable arc, and knowing the arc is most of the skill: subtle changes at low values around 0.1 (improved eye and feather detail), then progressively more background detail, then overexposure, then eventual conversion of the output into a drawing or illustration rather than a photograph. Along the way it can also fix anatomical problems — restoring a missing limb is the named example — and add texture like fur or glow before it degrades. The background-detail band is specifically useful with the Turbo workflow from earlier, since Flux by default renders strong background bokeh with very little detail in it.

Two alternates do the same kind of job with fewer knobs. The Lying Sigma Sampler takes three values — dishonesty factor, start, stop — and is simpler than Detail Daemon but correspondingly harder to control precisely. Multiply Sigma also takes three values (factor, start, end) for adjusting the same noise/detail schedule. The material describes all three but does not rank them beyond noting Lying Sigma's control tradeoff.

Открытые вопросы

Концепты

Источники