JSON Output

Getting reliable JSON out of an LLM starts with clear prompting — explicit format instructions, a shown example, and (where the provider supports it) a dedicated JSON output mode rather than relying on instructions alone.

Prompting for JSON — Before/After

Weak:
"Give me the order info as JSON."

Better:
"Return ONLY valid JSON matching this exact structure, with no
additional text before or after:
{
  \"order_id\": string,
  \"status\": string,
  \"total\": number
}

Order: [order details]"

Showing the exact expected structure (not just saying "as JSON") substantially improves output consistency — the model has a concrete pattern to match rather than inferring structure from a vague instruction.

A Common Failure: Extra Text Around the JSON

Model output (unwanted):
"Sure! Here's the JSON you requested:

{"order_id": "4521", "status": "shipped", "total": 1025}

Let me know if you need anything else!"

Explicitly instructing "no additional text, only the JSON object" reduces this, though isn't always perfectly reliable through prompting alone — see Structured Generation for provider features that enforce this more strictly.

Parsing Defensively

import json

def safe_parse_json(raw_output):
    # Strip common wrapping patterns before attempting to parse
    cleaned = raw_output.strip()
    if cleaned.startswith("```json"):
        cleaned = cleaned.removeprefix("```json").removesuffix("```").strip()

    try:
        return json.loads(cleaned)
    except json.JSONDecodeError:
        return None  # caller must handle this explicitly — never
                      # assume parsing always succeeds

Practical Use Case

Any feature extracting structured data from unstructured text (support tickets, resumes, invoices) relies on this pattern — clear format instructions, a shown example, and defensive parsing that treats a malformed response as an expected, handled case rather than a rare exception.

Common Mistakes

• Assuming the model will always return clean JSON with no wrapping text or formatting artifacts, and not handling the case where it doesn't
• Not showing the exact expected structure, relying on a vague "return as JSON" instruction
• Using regex to try to extract JSON from free text instead of proper parsing with clear error handling

Interview Relevance

"How would you reliably get JSON output from an LLM in production?" — explicit structure in the prompt, defensive parsing, and validation are the three pillars of a solid answer.

Practice Question

Write a prompt (with shown structure) for extracting name, email, and requested action from a customer email, and the defensive parsing code to handle the response.

Related Notes

• JSON Schema
• Schema Validation
• Structured Generation