Comfy UI

ComfyUI Updates

ComfyUI Updates [Sep 13, 2024]

ComfyUI now supports the new Stable Video Diffusion model for generating videos from images. With ComfyUI, you can create 1024×576 videos, 25 frames long, on a GTX 1080 with 8GB of VRAM. I’ve also successfully tested it on my AMD 6800XT with ROCm on Linux.

Stable Video Diffusion example

For workflows and detailed instructions on using these models, visit the video examples page.

If you’d like to use the workflow I used to generate the video shown above, you can save it and drag it into ComfyUI.

LCM

LCM models allow for sampling in very few steps. Recently, Loras have been released to convert standard SDXL and SD1.x models to LCM.

For instructions on using LCM in ComfyUI, refer to the LCM examples page.

Kohya Deep Shrink

The _for_testing->PatchModelAddDownscale node introduces downscaling to the UNet, which can be scheduled to occur only during the initial timesteps of the model. This approach allows for generating consistent images at higher resolutions without requiring a second pass.

To use this feature, add the node to your workflow and double the resolution at which you’re generating.

Support for ZSNR V Prediction Models

The new ModelSamplingDiscrete node lets you properly sample v_prediction ZSNR models.

Additionally, the RescaleCFG node implements the rescale CFG algorithm from the ZSNR paper and should be used with these models for accurate sampling.

To use a ZSNR v_pred model, load it with the standard checkpoint loader node, then chain it with the ModelSamplingDiscrete node set to v_pred and zsnr. Follow this by adding the RescaleCFG node.

Other ComfyUI Updates

  • The Load VAE node now supports TAESD, a fast and compact VAE used for high-quality previews. If you have the taesd_encoder, taesd_decoder, taesdxl_encoder, and taesdxl_decoder in the models/vae_approx directory, “taesd” and “taesdxl” options will appear on the Load VAE node. This is especially useful for those seeking maximum speed with LCM.
  • SaveAnimatedWEBP: A node to save image batches as animated WEBP files. These files include metadata and can be reloaded in ComfyUI to restore the workflow.
  • DOM Element Clipping: Nodes can now render on top of multiline text boxes. If this slows down your UI, you can disable it in the settings.
  • The UI now supports workflows in API format.
  • New UI Color Schemes: Available under settings -> color palette.
  • RepeatImageBatch: A node for repeating batches of images.
  • ImageCrop: A node for cropping images.
  • LatentInterpolate: Located under latent->advanced, this node uses NLERP to interpolate between two different latents.
  • heunpp2 Sampler: This sampler, from a specific repo, is highlighted in a paper discussing the quality improvements from using different samplers within a single generation. ComfyUI has long supported chaining advanced sampler nodes together. For a complex example, see the Noise Latent Composition Example page.
  • COMBO Primitive Nodes: These nodes now include a filter to control what they will increment or randomize.
  • FlipSigmas: A new node that flips the sigmas passed to the custom sampler node. It can be used to “unsample” an image.

Cool ComfyUI-Related Projects