Practical Guide to Enforcing Structured JSON Output (JSON Mode / Structured Outputs)
Is your LLM's output format constantly all over the place and causing code parsing errors? Learn how to use response_mime_type and response_schema to enforce a 100% strict JSON data structure.
1. The Fragility of Previous Prompt Constraints
Even if you write "Please be sure to return valid JSON" a hundred times in the prompt, the LLM will still occasionally add extra code block backticks (```json) or verbose explanations before and after, causing the backend program's json.loads to crash.
2. Enforcing Strict Schema Alignment in Code
import typing_extensions as typing
class Recipe(typing.TypedDict):
recipe_name: str
ingredients: list[str]
model = genai.GenerativeModel(
"gemini-1.5-flash",
generation_config={
"response_mime_type": "application/json",
"response_schema": Recipe
}
)
response = model.generate_content("Teach me how to make a classic tomato and egg stir-fry")
print(response.text)
Once configured, the model is constrained at the underlying decoding level by a grammar state machine, and its output is 100% pure JSON that can be parsed directly and seamlessly by machines!