Can llama_state_* save/restore be used across different n_ctx? Which params must match?

[PopFlamingo]

Hi! I’m would like to use the state APIs and wanted to clarify the compatibility contract.

APIs involved

• Save: llama_state_get_size(ctx), llama_state_get_data(ctx, buf, size) (or llama_state_save_file(path, ctx, …))
• Restore: llama_state_set_data(ctx, buf, size) (or llama_state_load_file(path, ctx, …))

Questions

1. If a state was saved from a context created with llama_context_params where n_ctx = A, can it be restored into a context created with n_ctx = B where A != B?

   • Is this supported when B > A, B < A, or only when B == A?

2. Beyond n_ctx, which fields in llama_context_params must match for llama_state_set_data to succeed and reproduce the same continuation?
   For example:

   • type_k / type_v (KV precision)
   • Unified KV / n_seq_max-related behavior
   • RoPE/scaling fields (e.g., rope scaling type, freq base/scale, YaRN settings)
   • Backend flags like flash attention, etc.

3. Sizing on restore: is the intended pattern to pass the serialized blob’s byte length to llama_state_set_data(ctx, buf, saved_size) rather than calling llama_state_get_size(dst_ctx) on the destination?

Any authoritative guidance (or doc pointers) on which parameters must match for a valid restore would be super helpful. Thanks!

[mahabot]

The state blob has no compatibility metadata. nothing is validated before restore.

n_ctx: restore works if destination n_ctx ≥ source. If smaller, the blob overflows the buffer and assert fail (nread <= state_size, same crash as #20473).

Params that must match: n_embd, n_layer, n_head_kv, type_k/type_v, rope_freq_base, n_vocab. Any mismatch = crash or silent garbage. Watch out for type_k/type_v and enabling -fa can change KV quantization implicitly, so same model + same n_ctx produces a different blob size.

Size to pass to llama_state_set_data(): the actual byte length of the saved blob, not llama_state_get_size() on the destination (which reflects the destination's capacity, not the blob's).

Related: #21145