What is fine-tuning?

Training vs. fine-tuning

Training a large language model costs tens of millions of euros. Fine-tuning takes an already-trained base model and adapts it for a specific application with a relatively small dataset.

When is fine-tuning useful?

• Training on sector-specific jargon or style
• Consistent behavior: always the same format and tone
• Processing sensitive data without an external API
• Better performance on a specific task

How does fine-tuning work?

1. Compile training data
2. Choose a base model
3. Train
4. Evaluate
5. Deploy

Fine-tuning vs. RAG

Fine-tuning is suitable for style and behavior; RAG is suitable for knowledge and facts that change regularly.