feat(passthrough): Introduce noise injection

[TheLocalDrummer]

Note

Adds configurable noise injection to passthrough (with noise_scale, noise_seed, noise_variance) and updates documentation.

• Passthrough merge method:
  • Add optional noise injection in mergekit/merge_methods/passthrough.py controlled by per-model parameters:
    • noise_scale: scales Gaussian noise added to the tensor
    • noise_seed: seeds the noise generator for reproducibility
    • noise_variance: scales noise by tensor.std() * noise_variance
  • Extend tensor_parameters() to expose new options alongside existing scale.
• Docs:
  • Update docs/merge_methods.md for passthrough to document noise_scale, noise_seed, and noise_variance parameters.

^{Written by Cursor Bugbot for commit 783e3ff. This will update automatically on new commits. Configure here.}

Thanks Cursor! Please note that I haven't revisited this since a month or two ago. Will take another serious look before finalizing things!

Sample config
Base with noise: https://huggingface.co/BeaverAI/Signal-24B-Base-v1a-GGUF
Finetuned: https://huggingface.co/BeaverAI/Signal-24B-v1a-GGUF

merge_method: passthrough
slices:
- sources:
  - layer_range: [0, 24]
    model: anthracite-core/Mistral-Small-3.2-24B-Instruct-2506-Text-Only
- sources:
  - layer_range: [24, 36]
    model: anthracite-core/Mistral-Small-3.2-24B-Instruct-2506-Text-Only
    parameters:
      noise_seed: 69420
      noise_variance: False
      noise_scale:
      - filter: post_attention_layernorm
        value: 0.005
      - filter: q_proj
        value: 0.005
      - filter: k_proj
        value: 0.005
      - value: 0.0
- sources:
  - layer_range: [36, 40]
    model: anthracite-core/Mistral-Small-3.2-24B-Instruct-2506-Text-Only

TODO

• Introduce noise_seed_increment to shuffle random values while maintaining replicability. (Alternatively, the user can just do it manually via the config?)
• Maybe convert noise_variance into a number (for scaling) instead of a flag
• Cache the initialized noise_generator for a significant speed increase (more-so when introducing noise_seed_increment
• Check if passthrough can be run alongside other merge methods

Noise can rewire the LLM to consider new novel concepts. Noise can also lobotomize a model, so make sure to apply it properly! Ask your local LLM for advice.

Credits to DavidAU for inspiration 🤣

[github-actions]

Thank you for your submission, we really appreciate it. Like many open-source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution. You can sign the CLA by just posting a Pull Request Comment same as the below format.

---

I have read the CLA Document and I hereby sign the CLA

---

TheLocalDrummer seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
_{You can retrigger this bot by commenting recheck in this Pull Request. Posted by the CLA Assistant Lite bot.}

[cursor]

This PR is being reviewed by Cursor Bugbot

Details

Your team is on the Bugbot Free tier. On this plan, Bugbot will review limited PRs each billing cycle for each member of your team.

To receive Bugbot reviews on all of your PRs, visit the Cursor dashboard to activate Pro and start your 14-day free trial.

[cursor]

Bug: Silence Debugging Noise

Multiple print() statements for debugging noise injection were left in the code. These will clutter output during production merges and should be removed or replaced with proper logging.

 

[TheLocalDrummer]

To remove before merging

[cursor]

Bug: Default Seed Value Not Honored

The noise_seed parameter has default_value=None in its definition, but line 36 expects it to default to 42. When not provided by the user, the parameter will be None in the dict, so .get("noise_seed", 42) returns None instead of 42, causing the seed to be None rather than the intended default of 42.

 

[TheLocalDrummer]

I remember getting confused by this one. ConfigParameterDef seems to cast the default value into a string?

[TheLocalDrummer]

I have read the CLA Document and I hereby sign the CLA