Figure 1: Schematic overview of proposed subspace control framework, SIFT. |
This is the official code repository for SIFT Subspace Control: Turning Constrained Model Steering into Controllable Spectral Optimization.
📣 Check out our Arxiv version on subspace control for constrained model training!
Foundation models, such as large language models (LLMs), are powerful but often require customization before deployment to satisfy practical constraints such as safety, privacy, and task-specific requirements, leading to "constrained" optimization problems for model steering and adaptation. However, solving such problems remains largely underexplored and is particularly challenging due to interference between the primary objective and constraint objectives during optimization. In this paper, we propose a subspace control framework for constrained model training. Specifically, (i) we first analyze, from a model merging perspective, how spectral cross-task interference arises and show that it can be resolved via a one-shot solution that orthogonalizes the merged subspace; (ii) we establish a connection between this solution and gradient orthogonalization in the spectral optimizer Muon; and (iii) building on these insights, we introduce SIFT (spectral interference-free training), which leverages a localization scheme to selectively intervene during optimization, enabling controllable updates that mitigate objective–constraint conflicts. We evaluate SIFT across four representative applications: (a) machine unlearning, (b) safety alignment, (c) text-to-speech adaptation, and (d) hallucination mitigation. Compared to both control-based and control-free baselines, SIFT consistently achieves substantial and robust performance improvements across all tasks.
This repository is organized into four task-specific modules:
unlearningsafetyspeechhallucination
cd unlearning
conda env create -f environment.yml
conda activate <env_name>bash unlearn_sift_run.shcd safety
conda env create -f environment.yml
conda activate <env_name>bash safety_sift_run.shcd speech
conda env create -f environment.yml
conda activate <env_name>bash speech_sift_run.shcd hallucination
conda env create -f environment.yml
conda activate <env_name>bash hallucination_sift_run.shTo directly using our unlearned model, please refer to our HuggingFace Collection:
Published Soon !
Any problem about the code please contact the wangc168@msu.edu directly!