Schema Validity, JSON Validity, and Stringified-Argument Elimination
Field-failure metrics before and after PEFT LoRA / QLoRA tuning: stringified typed arguments drop from 17.5% to 0.0% on Llama 3.2 3B and from 8.3% to 0.0% on Llama 3.2 1B; schema validity rises from 67.5% to 90.8% on 3B and 34.2% to 75.0% on 1B; JSON validity reaches 100.0% across all three tuned models including Muse Glimmer 30B.
Eliminating Field Tool-Call Failures (Unseen Holdout Splits, n = 150)
Strict MCP servers reject calls when numbers or objects are quoted as strings (e.g. "fencing_token": "51")
Stringified Typed Values (Lower = Better)
Share of tool calls quoting numbers or JSON objects
Llama 3.2 3B Base
17.5%
Llama 3.2 3B LoRA
0.0% (eliminated)
Llama 3.2 1B Base
8.3%
Llama 3.2 1B LoRA
0.0% (eliminated)
63 owned synthetic examples removed stringification on both bases.
JSON Validity & Refusal Recall (Higher = Better)
Valid JSON output and proper refusal of unsupported calls
1B JSON valid
85.3% ->
100.0%
3B Schema valid
67.5% ->
90.8%
Glimmer 30B JSON
96.7% ->
100.0%
Glimmer Refusal (dev)
53.3% ->
100.0%
Glimmer Refusal (fresh)
84.6% ->
100.0%
Sources: tessera/docs/continuous-tuning.md and agora/evidence/glimmer-30b-2026-10-29/fresh-2b-decision.json (sha256: 553354e3...)