Models

Choosing the right models

Coherence's flexibility allows it to work with various Large Language Models (LLMs) across different generation stages. When you create your task, the configuration system automatically selects appropriate models for you. One technique Coherence uses is model ensembling, where it leverages multiple models in parallel to improve generation quality and reliability.

You can experiment with different models for each stage by changing them in the Models tab of the configuration UI. Keep in mind that the best model for one stage might not be optimal for another, and sometimes reducing the number of models can actually improve performance. This varies significantly between tasks, so experimentation is key.

Generation Stages

The generation process consists of four main stages:

1. Description: Creating human-readable scenario descriptions
2. Input Generation: Creating realistic input data
3. Output Generation: Creating expected outputs
4. Assessment: Evaluating accuracy and quality and improving the match between input and output

When selecting models for your task, consider using stronger models for more complex tasks, but keep in mind the balance between cost and quality. For critical tasks, model ensembles often provide the best results.

Working with Model Ensembles

Coherence automatically configures an initial ensemble based on your task's requirements, but you may want to customize this setup for optimal results. A good starting point is to use 2-3 models in your ensemble, focusing on stronger models for critical stages like output generation and assessment.

The key to effective model ensembles is matching model strengths to specific tasks. For example, you might want to use models with strong reasoning capabilities for assessment, while using models optimized for creativity in the description stage. Some models also excel in specific domains - consider using domain-specialized models when available.

As you refine your ensemble configuration, monitor the performance metrics closely. You may find that adding more models doesn't always improve results, and sometimes a simpler configuration performs better. The goal is to find the sweet spot between reliability, quality, and cost for your specific use case.

Remember that model ensembles typically increase costs due to multiple model calls. However, the improved reliability and quality often justify the investment for critical tasks where accuracy is paramount.