7. Structured output comparison

1. Defining the Complex Structure: The example starts by defining a complex struct for a person:

   type ComplexPerson struct {
       Name          string   `json:"name" validate:"required"`
       Age           int      `json:"age" validate:"required,gte=0,lte=150"`
       Occupation    string   `json:"occupation" validate:"required"`
       City          string   `json:"city" validate:"required"`
       Country       string   `json:"country" validate:"required"`
       FavoriteColor string   `json:"favoriteColor" validate:"required"`
       Hobbies       []string `json:"hobbies" validate:"required,min=1,max=5"`
       Education     string   `json:"education" validate:"required"`
       PetName       string   `json:"petName" validate:"required"`
       LuckyNumber   int      `json:"luckyNumber" validate:"required,gte=1,lte=100"`
   }

2. Debug Logging Function: A debug logging function is defined to help with troubleshooting:

   func debugLog(level gollm.LogLevel, format string, args ...interface{}) {
       if level == gollm.LogLevelDebug {
           fmt.Printf("[DEBUG] "+format+"\n", args...)
       }
   }

3. Defining Models to Compare: The example defines a list of models from different providers to compare:

   models := []struct {
       provider string
       model    string
   }{
       {"openai", "gpt-4o-mini"},
       {"openai", "gpt-4o"},
       {"anthropic", "claude-3-haiku-20240307"},
       {"anthropic", "claude-3-5-sonnet-20240620"},
   }

4. Creating Configurations: For each model, a configuration is created using environment variables for API keys:

   configs := make([]*gollm.Config, 0, len(models))
   for _, m := range models {
       apiKeyEnv := fmt.Sprintf("%s_API_KEY", strings.ToUpper(m.provider))
       apiKey := os.Getenv(apiKeyEnv)
       if apiKey == "" {
           fmt.Printf("Skipping %s %s: API key not set. Please set %s environment variable.\n", m.provider, m.model, apiKeyEnv)
           continue
       }
   
       config := &gollm.Config{}
       gollm.SetProvider(m.provider)(config)
       gollm.SetModel(m.model)(config)
       gollm.SetAPIKey(apiKey)(config)
       // ... (other configuration options)
   
       configs = append(configs, config)
   }

5. Generating JSON Schema: A JSON schema is generated based on the ComplexPerson struct:

   schema, err := gollm.GenerateJSONSchema(&ComplexPerson{})

6. Creating the Prompt: A prompt is created that includes the generated JSON schema:

   promptText := fmt.Sprintf(`Generate information about a fictional person.
   Create a fictional person with the following attributes: name, age, occupation, city, country, favorite color, hobbies (1-5), education, pet name, and lucky number (1-100).
   Ensure all fields are filled and adhere to the specified constraints.
   Return the data as a JSON object that adheres to this schema:
   %s`, string(schema))

7. Defining a Validation Function: A custom validation function is defined for the ComplexPerson struct:

   validateComplexPerson := func(person ComplexPerson) error {
       // Add any additional validation logic here
       if person.Age < 0 || person.Age > 150 {
           return fmt.Errorf("age must be between 0 and 150")
       }
       // ... (other validation checks)
       return nil
   }

8. Comparing Model Outputs: The CompareModels function is used to generate and compare outputs from all configured models:

   results, err := gollm.CompareModels(ctx, promptText, validateComplexPerson, configs...)

9. Processing and Displaying Results: The results are processed and displayed for each model:

   for _, result := range results {
       fmt.Printf("\n%s\n", strings.Repeat("=", 50))
       fmt.Printf("Results for %s %s\n", result.Provider, result.Model)
       fmt.Printf("Attempts: %d\n", result.Attempts)
   
       if result.Error != nil {
           fmt.Printf("Error: %v\n", result.Error)
           debugLog(debugLevel, "Raw response:\n%s", result.Response)
           continue
       }
   
       fmt.Println("Valid output generated:")
       prettyJSON, err := json.MarshalIndent(result.Data, "", "  ")
       if err != nil {
           fmt.Printf("Error prettifying JSON: %v\n", err)
           debugLog(debugLevel, "Raw response:\n%s", result.Response)
       } else {
           fmt.Printf("%s\n", string(prettyJSON))
       }
   }

10. Final Analysis: The example concludes with an analysis of the comparison results:

    fmt.Println("\nFinal Analysis of Results:")
    analysis := gollm.AnalyzeComparisonResults(results)
    fmt.Println(analysis)

In summary, this example demonstrates:

• Defining a complex struct for structured output

• Configuring multiple LLM models from different providers

• Generating a JSON schema from a Go struct

• Creating a prompt that includes the JSON schema

• Implementing custom validation for the structured output

• Using gollm's CompareModels function to generate and compare outputs from multiple models

• Processing and displaying the results, including error handling and debug logging

• Analyzing the comparison results

This example is particularly useful for developers who need to compare the performance and output quality of different LLM models when generating complex, structured data. It showcases gollm's capabilities in handling multiple providers, structured output validation, and model comparison.