[Mirror] New quantization type: Q3_HIFI#65

Open

ngxson wants to merge 295 commits intongxson:masterfrom

geoffmunn:master

This pull request is big! We're only showing the most recent 250 commits

Commits on Dec 21, 2025

Old files removed
Geoff Munn
committed
Cross-model documentation added
Geoff Munn
committed
Validation errors fixed
Geoff Munn
committed
Whitespace fixed
Geoff Munn
committed
Whitespace fixes
Geoff Munn
committed
Whitespace fixes
Geoff Munn
committed
Whitespace fixes
Geoff Munn
committed
Whitespace changes
Geoff Munn
committed
Whitespace fixes
Geoff Munn
committed
Whitespace fixes
Geoff Munn
committed
Whitespace fixes
Geoff Munn
committed
Whitespace fixes
Geoff Munn
committed
Whitespace fixes
Geoff Munn
committed
Whitespace fixes
Geoff Munn
committed
print statements changed to logging()
Geoff Munn
committed
Extra blank line removed
Geoff Munn
committed
Merge pull request #9 from geoffmunn/Q3_HIFI
geoffmunn
authored
Documentation moved
Geoff Munn
committed
GGML_TYPE_Q3_HIFI now value 12
Geoff Munn
committed
GGML_TYPE_Q3_HIFI moved to end, numbers re-ordered
Geoff Munn
committed

Commits on Dec 27, 2025

Commits on Dec 30, 2025

Commits on Dec 31, 2025

Commits on Jan 3, 2026

Merge pull request #12 from GeoffApples/Q4_HIFI_v3
geoffmunn
authored

Commits on Jan 4, 2026

Commits on Jan 5, 2026

Commits on Jan 8, 2026

q3_hifi renamed
GeoffApples
committed

Commits on Jan 10, 2026

Commits on Jan 13, 2026

Commits on Jan 14, 2026

Merge pull request #23 from GeoffApples/Q5_K_HIFI
geoffmunn
authored

Commits on Jan 15, 2026

Commits on Jan 16, 2026

Refine Q3_HIFI enhancement thresholds for model size categories in ggml-quants-hifi.c. Align with llama-quant.cpp for consistent tensor handling across model types, improving performance and addres…
GeoffApples
committed

Commits on Jan 17, 2026

Commits on Jan 18, 2026

Commits on Jan 19, 2026

Merge pull request #24 from GeoffApples/Q3_K_HIFI_v4
geoffmunn
authored

Commits on Jan 20, 2026

Commits on Jan 22, 2026

Merge branch 'ggml-org:master' into master
geoffmunn
authored

Commits on Jan 25, 2026

Files not needed
Geoff Munn
committed
Enhance Q3_K_HIFI structure and dequantization logic
Geoff Munn
committed
Add support for Q3_K_HIFI quantization type in model loader.
Geoff Munn
committed
Enhance Q3_K_HIFI quantization and dequantization with debug logging
Geoff Munn
committed
Refactor Q3_K_HIFI quantization logic to always use Q3_K_HIFI type when requested
Geoff Munn
committed
Update debug logging in Q3_K_HIFI quantization and dequantization functions
Geoff Munn
committed
Refine GCC diagnostic handling in repack.cpp
Geoff Munn
committed
Remove unnecessary preprocessor directive for Clang in repack.cpp
Geoff Munn
committed
Enhance Q3_K_HIFI tensor upgrade logic in llama_tensor_get_type function
Geoff Munn
committed
Implement dedicated kernel name for Q3_K_HIFI and optimize outlier handling in kernel_mul_mv_q3_k_hifi_f32_impl
Geoff Munn
committed
Refactor outlier correction logic in kernel_mul_mv_q3_k_hifi_f32_impl for improved thread efficiency
Geoff Munn
committed
Refine Q3_K_HIFI tensor upgrade logic in llama_tensor_get_type function
Geoff Munn
committed
Refine Q3_K_HIFI upgrade conditions in llama_tensor_get_type function
Geoff Munn
committed
Enhance Q3_K_HIFI upgrade logic in llama_tensor_get_type function
Geoff Munn
committed

Commits on Jan 26, 2026

Enhance Q3_K_HIFI quantization with model-size-aware outlier handling
Geoff Munn
committed
Enhance Q3_K_HIFI quantization with true outlier extraction
Geoff Munn
committed
Refactor Q3_K_HIFI quantization to utilize residual corrections for outlier handling
Geoff Munn
committed
Fix attempt for bad PPL readings
Geoff Munn
committed
Refactor Q3_K_HIFI quantization to implement true outlier extraction
Geoff Munn
committed
Refactor Q3_K_HIFI quantization to implement residual-based outlier corrections
Geoff Munn
committed
There is no way this will work, but it's a new change of direction
Geoff Munn
committed
Build errors fixed
Geoff Munn
committed
New SPARSE layout approach
Geoff Munn
committed
Build errors fixed
Geoff Munn
committed
Bad perplexity values fixed
Geoff Munn
committed
Refactor Q3_K_HIFI quantization for sparse layout optimization
Geoff Munn
committed
Garbage perplexity hopefully fixed
Geoff Munn
committed
Refactor Q3_K_HIFI quantization to improve outlier handling
Geoff Munn
committed
Build warnings fixed
Geoff Munn
committed
Add q_offset calculation to kernel_mul_mv_q3_k_hifi_f32_impl
Geoff Munn
committed

Commits on Feb 2, 2026

CUDA should match Metal
Geoff Munn
committed

Commits on Feb 3, 2026

ix Q3_K_HIFI struct alignment for CUDA builds
Geoff Munn
committed
Fix Q3_K_HIFI CUDA warnings and struct size
Geoff Munn
committed
Fix Q3_K_HIFI vec_dot kernel - use simple addition instead of residual calculation
Geoff Munn
committed
Refactor vec_dot_q3_k_hifi_q8_1 kernel to improve outlier processing.
Geoff Munn
committed
New Metal fixes and CUDA updates
Geoff Munn
committed
Fast and high precision
Geoff Munn
committed
Build errors in CUDA fixed
Geoff Munn
committed

Commits on Feb 4, 2026

Commits on Feb 8, 2026

Commits on Feb 9, 2026

Coder imatrix script
Geoff Munn
committed
Faster version
Geoff Munn
committed
Changes reverted
Geoff Munn
committed

Commits on Feb 10, 2026

Q4_K_HIFI build changes
Geoff Munn
committed
Build errors fixed
Geoff Munn
committed
Add validation for Q4_K_HIFI quantization data.
Geoff Munn
committed

Commits on Feb 12, 2026

Commits on Feb 13, 2026

Commits on Feb 17, 2026

Imatrix reverted back to latest version
geoffmunn
committed

Commits on Feb 19, 2026

Commits on Feb 20, 2026

Commits on Feb 23, 2026

Commits on Feb 24, 2026

Commits on Mar 1, 2026

Commits on Mar 2, 2026

Commits on Mar 4, 2026

Phase 1 of the TURBO plan completed
geoffmunn
committed

Commits on Mar 7, 2026

Commits on Mar 9, 2026

Commits on Mar 14, 2026

Fix kernel mapping for Q3_K TURBO implementation in Metal backend and add whitespace for clarity in CPU quantization functions
geoffmunn
committed

Commits on Mar 20, 2026

Merge branch 'ggml-org:master' into master
geoffmunn
authored

Commits on Apr 11, 2026

Commits on Apr 12, 2026