feat: PIC cache reuse with mistralrs (local) backend

[starpit]

Summary

Wire spnl Plus operators to mistral.rs for position-independent KV cache reuse (PIC) across requests. Plus blocks in queries are tagged with in-band markers, which the engine uses to identify cacheable segments via text-based content hashing — enabling automatic KV cache hits when the same document appears across different requests, regardless of position.

What's included

• PIC integration: add_messages_from_query tags Plus blocks with \0PIC_PLUS\0 in-band markers; the mistral.rs engine strips them before chat template processing and uses token subsequence matching to resolve message-to-token boundaries for cache lookup
• Benchmark (spnl bench pic): measures TTFT speedup and accuracy from cross-request PIC cache reuse with shuffled documents
  • -o/--output controls output (comma-separated): speedup, iqr, hitrate, latency, json, accuracy — combinable, e.g. -o speedup,accuracy
  • --full sweeps across doc sizes (xs through xxl) and models
  • -l/--length sets doc words (TTFT) or max tokens (accuracy)
  • --grading-model enables LLM-judge semantic equivalence scoring alongside token F1
  • Progress bars with silent model loading
• Tests: unit tests for PIC helpers (token_f1, normalize_tokens, parse_score, resolve_spectrum, compute_hit_rate, percentile) and add_messages_from_query_inner message builder (Plus tagging, Cross/Plus nesting, unsupported query types). Ignored integration test verifies PIC speedup > 1x with a real model.
• Docs: README_PIC.md (user-facing: query syntax, CLI, results) and engine internals in mistral.rs/PIC.md
• Deps: mistralrs/mistralrs-core pointed to starpit/mistral.rs fork branch pic-cache-reuse

Benchmark Results

Output of spnl bench pic --full:

Model         xs 10w  sm 50w  m 200w  lg 500w  xl 1000w  xxl 2000w
 ──────────── ─────── ─────── ─────── ──────── ───────── ──────────
 llama3.2:1b    2.02x   4.28x  13.35x   29.85x    52.91x     80.55x
 llama3.2:3b    2.12x   4.56x  15.25x   29.84x    48.71x     83.94x
 llama3.1:8b    2.38x   5.48x  17.71x   36.17x    65.57x     91.84x
 qwen2.5:0.5b   1.74x   3.38x  10.10x   22.51x    46.70x     87.87x
 qwen2.5:14b    3.00x   7.41x  24.74x   48.96x    97.98x    124.84x

Test plan

• Unit tests pass: cargo test -p spnl-cli --features bench --bin spnl -- bench::pic::tests (22 tests)
• Message builder tests pass: cargo test -p spnl --features local --lib -- generate::backend::mistralrs::tests (5 tests)
• Clippy clean: both spnl and spnl-cli pass clippy -- -D warnings
• cargo build --release -F bench,metal succeeds
• spnl bench pic runs end-to-end against a local model
• Integration test with GPU: cargo test -p spnl-cli --features bench,metal --bin spnl -- bench::pic::tests::pic_benchmark_shows_speedup --ignored
• Accuracy mode (-o accuracy) produces Plus vs flat comparison output
• Review README_PIC.md for correctness