How do you handle invalid JSON / schema mismatch — retry or hard stop?

[Zi-Ling]

I’m experimenting with local LLM function calling
and noticed that invalid JSON or partial schema matches
are pretty common failure modes.

One thing we found in practice is that retrying after
a schema / validation failure often makes things worse,
because corrupted arguments leak into subsequent steps.

I’m curious about the intended boundary here in this project:

• Are schema mismatches considered retryable?
• Or should they be treated as hard stop conditions?

Would love to understand the design philosophy.

[m13v]

in practice we do a single retry with the validation error included in the prompt, and if it fails again we hard stop and return a graceful error to the user. retrying more than once just wastes tokens since the model is usually confused about the schema at that point.

the thing that actually fixed most of our JSON issues was constraining the output format more aggressively upfront. instead of letting the model generate free-form JSON, we use strict tool schemas with explicit enum values and required fields. Claude rarely produces invalid JSON with well-defined tool schemas, but local models still struggle. for those we added a JSON repair step that fixes common issues like trailing commas and unquoted keys before validation.

[m13v]

our tool execution layer handles JSON validation, schema enforcement, and retry logic for function calling: https://github.com/m13v/fazm/blob/main/Desktop/Sources/Providers/ChatToolExecutor.swift

and the provider abstraction that defines the tool schemas with strict typing: https://github.com/m13v/fazm/blob/main/Desktop/Sources/Providers/ChatProvider.swift