Lore

AI rendering

We Built an AI Render Engine for FREE

Mickmumpitz argues that AI rendering has matured enough to turn rough 3D layouts into finished-looking shots in minutes. A free, open-source, locally run ComfyUI workflow built on a merged Skyreels/VACE video model, driven by outline and depth passes plus reference images, does this. The trade-off is control and simulated physics in exchange for speed, which the video says suits rapid prototyping and previs.

Mickmumpitz · 2026-02-25 · English

Key ideas

  1. The video opens by saying that two years ago he predicted AI is the future of rendering, and that progress has surprised him. AI can now restyle a 3D layout from a prompt and reference images. It also generates smoke, water splashes and explosive debris from the movement in the scene, and it goes from rough layout to fine render in minutes.

  2. The video separates generative AI rendering from traditional rendering. Traditional rendering calculates how light bounces off surfaces. Here a neural network predicts what the image should look like from patterns learned across millions of training images. The 3D scene contributes only extra information (depth maps, outlines, pose data, called control nets), and the model interpolates the reference image's style over the shot. He says he got a community note on X for calling this 'ray tracing'.

  3. The video frames the central difficulty as finding one model with three properties. It must follow scene geometry precisely, keep the freedom to invent new detail, and understand reference images well enough to generate new scene information in areas that were previously obstructed. He says every model they tested had only some of these, and they were close to giving up before finding a model merge.

  4. Which 3D passes you feed the model sets how much freedom the model has. Outlines preserve composition but leave room for new detail, and depth enforces the geometry more strictly. Control-net strength and the blend between passes move this dial further. The advice throughout is to start with the looser setting, test, and tighten only if needed.

  5. Style can be guided either by a single reference start frame, which renders a bit faster, or by multiple references (characters plus an environment) combined through a natural-language prompt. Swapping the reference image or the prompt changes the whole look of the shot without redoing the 3D work.

  6. The base workflow is limited in shot length (up to 120 frames at 24 fps, around 5 seconds, with VRAM and quality degrading beyond that). Longer shots depend on an advanced, Patreon-only workflow that chains generations in batches. The video presents this as the way to keep long shots smooth and consistent.

  7. The video's concluding evaluation is that speed is the real benefit: you can explore about 10 visual directions in the time one traditional render takes. The cost is control, and the physics are unsimulated, so results are sometimes convincing and sometimes off. The remedy is to rerun with a new seed or adjusted prompt. He judges the quality impressive for rapid prototyping and previs.

  8. The video presents the whole approach as accessible. It uses free open-source tools running on your own computer, with RunPod templates as an option for those without a powerful GPU. The work is funded by Patreon supporters, who also get advanced workflows and example files.

  9. Generative AI rendering with control nets, reference image and prompt — A neural network predicts the image from learned patterns instead of simulating light. It is fed depth maps, outlines or pose data from the 3D scene (control nets), plus a reference image and a prompt, and interpolates the reference's style over the shot. Apply: Export control passes from your 3D scene, supply a reference image (start frame or multiple references) and a prompt, and let the video model generate the shot, swapping the reference image to change style.

  10. Inner Reflections' Skyreels + VACE merged model — A model merge by Inner Reflections that combines Skyreels reference-to-video (strong at understanding reference images but unable to take control nets) with VACE (follows control nets precisely but has a weaker reference feature). Apply: Use the merged model as the engine of the free AI renderer workflow so that references and control nets work in tandem and new areas revealed by camera movement still match the reference's style.

  11. Outline pass in Blender (Workbench + Freestyle) — A white-on-black outline render used as a control net that preserves the exact composition while leaving the model freedom to generate detail between the lines. Apply: Switch the render engine to Workbench with flat lighting, single black color and white outlines. For more detail, enable Freestyle as a render pass in the View Layer tab, set its color to white and thickness to 2, deactivate use alpha, view it in the compositor with a viewer node, and save it with a file output node.

  12. Depth pass in Blender (Z pass) — A depth-map control pass that makes the model follow the scene geometry more precisely than outlines do. Apply: Activate Z in the View Layer, render, connect the depth output through a normalize node, invert it, add a curve node to raise contrast and separate the character's depth, then connect a file output node and save the image sequence.

  13. Exporting and merging both passes — Rendering both the outline and depth passes, then testing them separately or blending them to combine precision and freedom. Apply: Export both for every scene and test which works better. The AI renderer workflow has a node that blends them with an adjustable blend value, and he starts with outline-only for more freedom.

  14. Free AI preparation workflow (outline and depth from a viewport render) — A free workflow that creates approximate outline and depth maps from a render of whatever is in your viewport, without needing proper 3D passes. Apply: Render out your layout from the viewport and run it through the preparation workflow, accepting that the result is an approximation and not as good as properly rendered passes.

  15. Single start-frame generation (cloud or local) — Creating one styled first frame from the outline or depth image, using ChatGPT or Nano Banana or the local Z Image Turbo + control net workflow in ComfyUI, to serve as the style reference. Apply: In ComfyUI, drag the workflow in, install missing custom nodes via Manager, download and load the models, drop in the first-frame control image (or load a video and take only its first frame), write a simple prompt, set control net strength as low as possible, run, and change the seed to explore variations.

  16. Image sequence to H.264 video conversion in ComfyUI — Converting rendered image sequences (outline, depth) into H.264 video files so they are easier to handle as workflow inputs. Apply: Use a Load Images node feeding a Video Combine node from the Video Helper Suite, set the frame rate (24 fps in the demo), name the file, choose the H.264 format and run; or use any editing program.

  17. Free AI renderer workflow setup and single-reference run — The main free ComfyUI workflow that takes control-pass videos, a reference image and a prompt and generates the rendered video. Apply: Drag the workflow into ComfyUI, install missing custom nodes in Manager and restart, download models using the links in the notes beside the loaders, set resolution (square for the lizard), do not skip frames, load the outline pass (and depth if wanted), add the reference image and a prompt, then run.

  18. LLM-assisted prompt refinement — Using a large language model to refine or restyle the shot prompt. Apply: Describe the shot in a simple prompt with small details (e.g. wrinkles on the suit), copy it to an LLM and refine it, or ask it to change the style, as he did by asking Claude to change the style to anime.

  19. Multi-reference mode with character sheets and environment — Combining several reference images (characters and an environment) into one video, with a prompt describing who is where and what they do. Apply: Generate character sheets with Z Image Turbo (control net deactivated, empty latent node as the latent image). Select two reference input nodes and press Ctrl+B so the workflow switches to reference image mode, add a third reference for the environment, set the frame load cap (81) and 720p 16x9, and prompt in natural language with left/right positions, making it more specific if outfits drift.

  20. Advanced iterative batch workflow with overlapping start frames (Patreon) — A custom node pack that splits a long video into batches and runs them one iteration after another, using the last frames of the previous iteration as the start frames of the next for smooth, consistent long shots. Apply: Set frames per iteration between 41 and 121 depending on GPU (81 recommended), set the number of start frames (11) for blending, set total iterations (4 in the demo, 5 would be needed for the longer shot), optionally resume from a later iteration if only the ending is off, and set the seed.

  21. RunPod cloud templates for low-VRAM users — Ready-to-use RunPod templates that handle the setup of these workflows for people without a powerful GPU. Apply: Run the workflows on RunPod using the prepared templates; the video says signing up through the creators' link gives a random credit bonus of $5 to $500 after spending the first $10.

  22. Iterate by seed and prompt instead of simulation — The video's approach to correcting unsimulated physics or off results: regenerate rather than adjust a simulation. Apply: When splashes or details look off, run the workflow again with a different seed or an adjusted prompt, and use alternate start frames (such as a mossy forest or anime style) to explore other visual directions quickly.

Insights

The 'two languages' explanation makes the model merge concrete. Skyreels understands references but not control nets, and VACE follows control nets but has weaker references. VACE also fails to generate new detail matching the original style when the camera moves to a new area of the scene. The merge combines the strengths of both.

Mickmumpitz recommends exporting both the outline and depth passes for every scene, because you cannot know in advance which will work better. They can also be merged to get the best of both.

Blender's Freestyle outlines can be 'super slow for some reason', so the video starts with Workbench flat lighting (black surfaces, white outlines). Workbench only outlines one object at a time, which is why Freestyle is added back when more detail is needed.

The depth pass needs post-processing to work as a control net: normalize it, invert it, and apply a curve to raise contrast so the character's depth is better separated.

A start frame does not need to match the control image perfectly. In the demo the lizard's tail sat in a different position than in the outline image, and he says this is usually not a problem because it is only a reference. Changing the seed can also fix such details.

Control-net strength for the start-frame model should be set 'as low as possible', which he says gives maximum quality.

Mickmumpitz uses an LLM as a prompt translator. He pasted the realistic prompt into Claude with the instruction to change the style to anime, and the resulting video even added details such as blinking.

Detail drift in multi-reference shots, such as the fox's outfit changing, is addressed by writing a more specific prompt. For small details like wrinkles on a suit, you add them to the prompt.

Overlapping frames carry continuity between batches in the advanced workflow. 81 frames are generated per iteration, the last 11 become the start frames of the next, and a failed ending can be fixed by resuming from a later iteration.

The multi-reference demo scene was built simply by downloading Mixamo animations (transcribed as 'Migamo') into Blender and rendering an outline pass. This suggests that a rough layout with stock animation is enough input.

For the 300-frame stress test with tentacles and a moving camera, he loaded only the outline pass (no depth blend) and a stormy-ocean start frame. The result was effects and water splashes that he calls consistent.

«Instead of mathematically simulating light transport, a neural network predicts what the image should look like based on patterns it learned from millions and millions of training images.»

— 01:13

«It's just like the model understands two languages.»

— 02:43

«The outline pass is good if you want to preserve the exact composition, but give the model freedom to generate new detail between these lines.»

— 02:50

«Once two of these image inputs are selected, the workflow automatically switches from start image mode to reference image mode.»

— 10:36

«81 is a sweet spot.»

— 13:13

«I think the speed here is the real benefit. You can explore 10 visual directions in the time it takes up to set one traditional render.»

— 14:08

«But of course, you're trading control for speed.»

— 14:16

Reception

Mostly interested and appreciative, with many questions about hardware requirements and broken workflow links, plus a few thanks and praise comments, but also some criticism of the output quality as not final-grade and a few AI-skeptic complaints.

A dense, step-by-step ComfyUI and Blender tutorial that pairs a clear explanation of why a merged Skyreels/VACE model solves the geometry-versus-reference problem with concrete settings. It is framed by the author as a fast previs and prototyping tool rather than a replacement for traditional final rendering.

15:29

↳ Mickmumpitz · YouTube

Watch original