Structured output isn't always the right choice — forcing a genuinely nuanced, exploratory, or conversational response into a rigid schema can lose exactly the qualities that made free-form generation useful in the first place.
The Deciding Question
Will this output be consumed by code (needs structure) or read directly by a human as prose (structure often gets in the way)?
Side-by-Side
| Use Case | Better Fit | Why |
|---|---|---|
| Extracting order details from an email | Structured | Downstream code needs specific fields, not a sentence to parse |
| Drafting a customer support reply | Free text | A human reads it directly; natural, empathetic prose matters |
| Classifying a ticket's category + urgency | Structured | Feeds into routing logic that needs discrete values |
| Summarizing a long document for a human reader | Free text | Nuance and natural flow matter more than rigid fields |
| A chatbot's conversational response | Free text (usually) | Users expect natural conversation, not a JSON blob |
A Middle Ground: Structured Metadata + Free Text Together
{
"response_text": "I'm sorry to hear about the delay! I've checked
and your order is now out for delivery, expected
today by 6pm.",
"sentiment_detected": "frustrated",
"action_taken": "status_lookup",
"requires_escalation": false
}This pattern — a free-text field for the human-facing content, alongside structured metadata for application logic — is extremely common in real systems, capturing the benefit of both: natural language where it matters, structure where it's needed.
Practical Use Case
A customer-facing chatbot typically wants free-text responses for the actual conversation, but structured metadata (intent detected, sentiment, whether to escalate) for the application's own routing and logging logic — often generated together in a single structured response containing both.
Common Mistakes
- Forcing a nuanced explanation or conversational reply into rigid structured fields, producing awkward, robotic-feeling output
- Using pure free text for data that's actually going to be parsed downstream, leading to fragile, error-prone extraction logic
- Not considering the hybrid pattern (structured metadata + free-text field) when both a human-facing response and application logic are genuinely needed
Interview Relevance
"When would you NOT use structured output, even though it's generally more reliable for applications?" — anywhere the output is primarily meant to be read by a human as natural prose, where rigid structure would hurt the actual user experience.
Practice Question
Design a response format (structured, free-text, or hybrid) for a feature that both shows a customer a helpful reply AND logs whether the interaction needs manager review.