​
Overview

Dify’s structured outputs ensures LLMs return data in predictable JSON formats, making the output easier to process and integrate into applications.

​
Benefits

  • Consistent Data Formatting: All LLM-generated content follows predefined formats, eliminating data inconsistencies.

  • Seamless Integration: : Databases, APIs, or frontend applications can directly parse JSON Schema without additional data cleaning.

  • Simplified Development: Developers can skip writing complex validation code by letting JSON Schema handle output constraints.

​
Implementation Methods

Dify offers two ways to create structured outputs:

​
Method 1: Tool Parameters

Define your output structure directly in tool parameters. See Tool > Data Structures > Return Variable Definitions.

​
Method 2: JSON Schema Editor

Use the built-in editor in LLM nodes. See LLM > Advanced Features > Structured Outputs and LLM > Use Cases > Structured Outputs.

​
Error Handling

What Can Go Wrong

When you are working with the JSON Schema Editor, you might run into a few common challenges:

  • Limited Model Performance: Smaller LLMs (like GPT-3.5 Turbo) sometimes have trouble following JSON Schema instructions correctly.

  • Format Issues: Some LLMs only understand basic JSON, not full JSON Schema.

  • Error Messages: You might see Failed to parse structured output: output is not a valid json str when the model fails to generate proper JSON.

How to Fix These Issues

  1. Choose the Right Model. These models work best with JSON Schema:

    • Gemini 2.0 Flash/Flash-Lite
    • Gemini 1.5 Flash 8B (versions 0827/0924)
    • Gemini-1.5 pro
    • GPT-4o and GPT-4o-mini
    • o1-mini/o3-mini series

  2. Write Clear Instructions. Make sure your prompts match what you are trying to do. For example, if you need mathematical formulas, don’t ask for legal analysis—this helps the AI understand exactly what you need.

  3. Have a Backup Plan.

    1. Enable Retry on Failure with custom retry attempts and intervals.

    2. Set up Failure Branch to handle any errors.

See Error Handling for more details.

Edit this page | Report an issue