JSON output validation

JSON Output Validation in gollm

gollm provides robust support for generating and validating structured JSON outputs from LLMs. This feature ensures that LLM responses match your expected data structure, making it easier to process and use the generated content in your applications.

Here's how you can use JSON output validation in gollm:

1. Define your struct with JSON tags and validation rules:

type PersonInfo struct {
    Name       string   `json:"name" validate:"required"`
    Age        int      `json:"age" validate:"required,gte=0,lte=150"`
    Occupation string   `json:"occupation" validate:"required"`
    Hobbies    []string `json:"hobbies" validate:"required,min=1,max=5"`
}

2. Generate a JSON schema for your struct:

schemaJSON, err := gollm.GenerateJSONSchema(&PersonInfo{}, gollm.WithExpandedStruct(true))
if err != nil {
    log.Fatalf("Failed to generate JSON schema: %v", err)
}

3. Create a prompt that includes the JSON schema:

prompt := gollm.NewPrompt(
    "Generate information about a fictional person",
    gollm.WithDirectives(
        "Create a fictional person with a name, age, occupation, and hobbies",
        "Ensure the age is realistic for the occupation",
        "Include 1 to 5 hobbies",
        "Return ONLY the JSON data for the person, not the schema",
    ),
    gollm.WithOutput(fmt.Sprintf("Generate a JSON object that adheres to this schema:\n%s\nDo not include the schema in your response, only the generated data.", string(schemaJSON))),
)

4. Generate the response using the WithJSONSchemaValidation option:

response, err := llm.Generate(ctx, prompt, gollm.WithJSONSchemaValidation())
if err != nil {
    log.Fatalf("Failed to generate text: %v", err)
}

5. Parse and validate the response:

var person PersonInfo
err = json.Unmarshal([]byte(response), &person)
if err != nil {
    log.Fatalf("Failed to parse response as JSON: %v", err)
}

if err := gollm.Validate(&person); err != nil {
    log.Fatalf("Generated data does not match schema: %v", err)
}

This process ensures that the LLM generates data that adheres to your specified structure and validation rules. The WithJSONSchemaValidation option tells the LLM to validate its output against the provided schema before returning it.

For more complex validation scenarios, you can define custom validation functions:

validateComplexPerson := func(person ComplexPerson) error {
    if person.Age < 0 || person.Age > 150 {
        return fmt.Errorf("age must be between 0 and 150")
    }
    if len(person.Hobbies) < 1 || len(person.Hobbies) > 5 {
        return fmt.Errorf("number of hobbies must be between 1 and 5")
    }
    if person.LuckyNumber < 1 || person.LuckyNumber > 100 {
        return fmt.Errorf("lucky number must be between 1 and 100")
    }
    return nil
}

You can then use this custom validation function in conjunction with the CompareModels function to validate outputs across different models:

results, err := gollm.CompareModels(ctx, promptText, validateComplexPerson, configs...)
if err != nil {
    log.Fatalf("Error comparing models: %v", err)
}

By leveraging these JSON output validation features, you can ensure that your LLM-generated content consistently meets your application's data structure and quality requirements.