Ctrl-Adapter / README.md
hanlincs's picture
Update README.md
ca4af5f verified
metadata
license: mit

This repo contains the pretrained checkpoints for Ctrl-Adapter

CTRL-Adapter is an efficient and versatile framework for adding diverse spatial controls to any image or video diffusion model. It supports a variety of useful applications, including video control, video control with multiple conditions, video control with sparse frame conditions, image control, zero-shot transfer to unseen conditions, and video editing.

See also: https://github.com/HL-hanlin/Ctrl-Adapter



Description of Model Checkpoints

SDXL

Depth map

  • Ctrl-Adapter/sdxl_depth/diffusion_pytorch_model.safetensors

Canny edge

  • Ctrl-Adapter/sdxl_canny/diffusion_pytorch_model.safetensors

I2VGen-XL

Depth map

  • Ctrl-Adapter/i2vgenxl_depth/diffusion_pytorch_model.safetensors

Canny edge

  • Ctrl-Adapter/i2vgenxl_canny/diffusion_pytorch_model.safetensors

Soft edge

  • Ctrl-Adapter/i2vgenxl_softedge/diffusion_pytorch_model.safetensors

Sparse control with user scribbles

  • Ctrl-Adapter/i2vgenxl_scribble_sparse/diffusion_pytorch_model.safetensors

Multi-condition control trained on depth map, canny edge, normal map, soft edge, segmentation, line art, and openpose

  • Ctrl-Adapter/i2vgenxl_multi_control_adapter/diffusion_pytorch_model.safetensors
  • Ctrl-Adapter/i2vgenxl_multi_control_router/diffusion_pytorch_model.safetensors

SVD

Depth map

  • Ctrl-Adapter/svd_depth/diffusion_pytorch_model.safetensors

Canny edge

  • Ctrl-Adapter/svd_canny/diffusion_pytorch_model.safetensors

Soft edge

  • Ctrl-Adapter/svd_softedge/diffusion_pytorch_model.safetensors