Yizuo Peng1,2 · Xuelin Chen3 · Kai Zhang1,* · Xiaodong Cun2, *
1 Tsinghua University 2GVC Lab, Great Bay University 3Adobe Research
Arxiv | Openreview | PDF
We perform controllable video relighting with a user-provided light trajectory. Where we inject the light map on the noisy latent of VDM using the light map injection module. Then, in each denosing step, we design a geometry-aware relighting module to produce the relighted results frame-wise. Thus, the VDM can help to generate consistent video results with controllable lighting.
| Input Video | Light Map | Relighted video |
|
|
|
|
|
|
|
|
|
|
|
|
git clone https://github.com/GVCLab/LightCtrl.git
cd LightCtrl
conda create -n LightCtrl python==3.10
conda activate LightCtrl
pip install -r requirements.txt
IC-Light:🤗Huggingface ;
SD RealisticVision: 🤗Huggingface ;
Animatediff Motion-Adapter-V-1.5.3: 🤗Huggingface ;
CogVideoX-2b: 🤗Huggingface
Video relighting with user-defined light setting:
python relight.py --config "configs/animatediff_relight/music_girl.yaml"LightCtrl also supports the CogVideoX backbone:
python cog_relight.py --config "configs/cog_relight/blackman.yaml"If you find our work helpful, please leave us a star and cite our paper.
@inproceedings{ peng2026lightctrl,
title={LightCtrl: Training-free Controllable Video Relighting},
author={Yizuo Peng and Xuelin Chen and Kai Zhang and Xiaodong Cun},
booktitle={The Fourteenth International Conference on Learning Representations},
year={2026},
url={https://openreview.net/forum?id=5ft8vd9rwc}
}
We are very grateful to the authors of Light-A-Video, AnimateDiff, CogVideoX, and IC-Light for the foundation of our code from their open-source code.