Structured output: grammar is not truth

Picture it

Picture a bowling lane with the bumpers raised, physically preventing the ball from landing in the gutter. Constrained generation applies similar bumpers to the model's output, forcing it to step only on valid tokens that fit a rigid blueprint, like a perfectly formatted JSON. But remember, the bumpers only guarantee the ball hits a pin, not the right pin—the output will have the correct shape, but the facts inside might still be entirely made up.

What to understand

• Token masks, grammars, schema complexity, and compilation
• Validation, semantic checks, parser versions, and retry budgets

Follow the references

1. 1 · Documentation

   SGLang · Serving documentation ↗

   Find structured outputs and run a small schema example; inspect both validity and latency.

Make it concrete

Benchmark constrained generation versus generate-and-retry on your schemas. Count parse failures, semantic errors, compilation overhead, and total tokens per successful result.