Skip to main content
Version: 0.0.0

Supported models for fine-tuning

The following table provides a list of models available for fine-tuning. Each model has a context length, or, the maximum number of tokens the model can process in a single input. The context length depends on whether you're running inference on the base model, fine-tuning the base model, or running inference on the fine-tuned model.

ModelSizeBase context lengthFine-tuning context lengthInferencing context length
meta-llama/Llama-2-13b-chat-hf13B4096 tokensUp to 32768 tokensThe greater of the base context length and fine-tuning context length used; up to 32768 tokens.
meta-llama/Llama-2-70b-chat-hf70B4096 tokensUp to 16384 tokensThe greater of the base context length and fine-tuning context length used; up to 16384 tokens.
mistralai/Mistral-7B-Instruct-v0.17B16384 tokensUp to 4096 tokensThe greater of the base context length and fine-tuning context length used; up to 16384 tokens.
mistralai/Mixtral-8x7B-Instruct-v0.18x7B32768 tokensUp to 4096 tokensThe greater of the base context length and fine-tuning context length used; up to 32768 tokens.
Context length considerations

Fine-tuning a model with a context length longer than its base context length may increase training time and result in reduced model quality.

Deprecation of meta-llama/Llama-2-7b-chat-hf

meta-llama/Llama-2-7b-chat-hf has transitioned to the legacy models list. Moving forward, for fine-tuning a similar model, use mistralai/Mistral-7B-Instruct-v0.1.

While current fine-tuned models based on meta-llama/Llama-2-7b-chat-hf remain accessible, this support is temporary.