Craft Your AI Vision: Fine-Tuning Magic with Mistral on Mistral Server

Ankush k Singal
AI Artistry
Published in
10 min readJun 12, 2024

Ankush k Singal

Source: Image created by Author using MidJourney

Introduction

Imagine a culinary maestro who can whip up anything from delicate pastries to hearty stews. Now, imagine giving them your grandma’s secret recipe for the most amazing pie you’ve ever tasted. That’s the essence of fine-tuning in AI — taking a pre-trained model, already a master of many skills, and specializing it for your specific project.

French AI startup Mistral is making waves by giving developers and businesses this exact power: the ability to fine-tune their powerful AI models. This exciting development opens a world of possibilities for anyone who wants to leverage the power of AI in a way that perfectly aligns with their unique needs.

Source: Finetuned-Mistral

Fine-Tuning: The Secret Sauce

So, how does this fine-tuning work? Picture Mistral’s pre-trained models as those culinary masters. They can understand and generate text in a variety of ways, but fine-tuning lets you give them a specific focus. Let’s say you need a model that can craft all sorts of creative content, translate languages specific to your industry jargon, or even generate code tailored to your programming language. Fine-tuning takes Mistral’s base model and trains it on your data, like that secret family recipe, so it excels in your desired area.

Benefits You Can Savor

There are several reasons to get excited about Mistral’s fine-tuning options:

  • Faster Development: Ditch the idea of starting from scratch! By using a pre-trained model as your foundation, you can significantly reduce the time and resources needed to train a model for your specific task. It’s like having a head start in the kitchen — you don’t need to learn all the basics before creating a masterpiece.
  • Performance with a Zing: Fine-tuning a model on your own data leads to better accuracy and performance compared to a generic model. It’s like the chef perfecting grandma’s recipe — the results are going to be truly outstanding!
  • Flexibility on Your Plate: Mistral understands that a cookie-cutter approach doesn’t work in the world of AI. They offer a range of customization options, from self-service tools to fully managed solutions, allowing you to choose the approach that best suits your needs and expertise.

Code Implementation

Lets delve into the code implementation of Mistral Finetune on Mistral Server.

Step I: Download Libraries

pip install mistralai pandas pyarrow rich gradio
export MISTRAL_API_KEY=xxxxxx
export WANDB_API_KEY=xxxxxxxx
wget https://raw.githubusercontent.com/mistralai/mistral-finetune/main/utils/reformat_data.py

Step II: Download Data

import pandas as pd
from rich import print
df = pd.read_parquet('https://huggingface.co/datasets/smangrul/ultrachat-10k-chatml/resolve/main/data/test-00000-of-00001.parquet')

df_train = df.sample(frac=0.995, random_state=200)
df_eval = df.drop(df_train.index)

df_train.to_json("ultrachat_chunk_train.jsonl", orient="records", lines=True)
df_eval.to_json("ultrachat_chunk_eval.jsonl", orient="records", lines=True)

print(df_train.iloc[100]['messages'])

Step III: Refactor and Finetune

# Refactor 
python reformat_data.py ultrachat_chunk_eval.jsonl
python reformat_data.py ultrachat_chunk_train.jsonl

# Finetune
import os
import json
import time
from rich import print
from mistralai.client import MistralClient
from mistralai.models.jobs import TrainingParameters
from mistralai.models.chat_completion import ChatMessage
from mistralai.models.jobs import WandbIntegrationIn

api_key = os.environ.get("MISTRAL_API_KEY")
client = MistralClient(api_key=api_key)

def pprint(obj):
print(json.dumps(obj.dict(), indent=4))

# 1. Upload the dataset
with open("ultrachat_chunk_train.jsonl", "rb") as f:
ultrachat_chunk_train = client.files.create(file=("ultrachat_chunk_train.jsonl", f))
with open("ultrachat_chunk_eval.jsonl", "rb") as f:
ultrachat_chunk_eval = client.files.create(file=("ultrachat_chunk_eval.jsonl", f))

print("Data:")
pprint(ultrachat_chunk_train)
pprint(ultrachat_chunk_eval)

# 2. Create Fine Tuning Job
created_jobs = client.jobs.create(
model="open-mistral-7b",
training_files=[ultrachat_chunk_train.id],
validation_files=[ultrachat_chunk_eval.id],
hyperparameters=TrainingParameters(
training_steps=10,
learning_rate=0.0001,
),
integrations=[
WandbIntegrationIn(
project="test_ft_api",
run_name="test",
api_key=os.environ.get("WANDB_API_KEY"),
).dict()
],
)
print("\nCreated Jobs:")
pprint(created_jobs)

# 3. Check the Status of the Job
print("\nChecking Job Status:")
retrieved_job = client.jobs.retrieve(created_jobs.id)
while retrieved_job.status in ["RUNNING", "QUEUED"]:
retrieved_job = client.jobs.retrieve(created_jobs.id)
pprint(retrieved_job)
print(f"Job is {retrieved_job.status}, waiting 10 seconds")
time.sleep(10)

jobs = client.jobs.list()
pprint(jobs)

retrieved_jobs = client.jobs.retrieve(created_jobs.id)
pprint(retrieved_jobs)

# 4. Use the Fine Tuned Model
chat_response = client.chat(
model=retrieved_jobs.fine_tuned_model,
messages=[ChatMessage(role='user', content='What is the best French cheese?')]
)
print("\nTesting Fine Tuned Model:")
pprint(chat_response)

# Output
Data:
{
"id": "3d3fcfcc-e16d-4391-99db-87e3ef2a9c24",
"object": "file",
"bytes": 11812415,
"created_at": 1718200942,
"filename": "ultrachat_chunk_train.jsonl",
"purpose": "fine-tune"
}
{
"id": "808e457a-53ba-4464-a4c9-eec7e3581b89",
"object": "file",
"bytes": 59290,
"created_at": 1718200943,
"filename": "ultrachat_chunk_eval.jsonl",
"purpose": "fine-tune"
}
Created Jobs:
{
"id": "a702040d-2091-448f-a272-0b1197772463",
"hyperparameters": {
"training_steps": 10,
"learning_rate": 0.0001
},
"fine_tuned_model": null,
"model": "open-mistral-7b",
"status": "QUEUED",
"job_type": "FT",
"created_at": 1718200943,
"modified_at": 1718200943,
"training_files": [
"3d3fcfcc-e16d-4391-99db-87e3ef2a9c24"
],
"validation_files": [
"808e457a-53ba-4464-a4c9-eec7e3581b89"
],
"object": "job",
"integrations": [
{
"type": "wandb",
"project": "test_ft_api",
"name": null,
"run_name": "test"
}
]
}
Checking Job Status:
{
"id": "a702040d-2091-448f-a272-0b1197772463",
"hyperparameters": {
"training_steps": 10,
"learning_rate": 0.0001
},
"fine_tuned_model": null,
"model": "open-mistral-7b",
"status": "QUEUED",
"job_type": "FT",
"created_at": 1718200943,
"modified_at": 1718200943,
"training_files": [
"3d3fcfcc-e16d-4391-99db-87e3ef2a9c24"
],
"validation_files": [
"808e457a-53ba-4464-a4c9-eec7e3581b89"
],
"object": "job",
"integrations": [
{
"type": "wandb",
"project": "test_ft_api",
"name": null,
"run_name": "test"
}
],
"events": [
{
"name": "status-updated",
"data": {
"status": "QUEUED"
},
"created_at": 1718200943
}
],
"checkpoints": [],
"estimated_start_time": null
}
Job is QUEUED, waiting 10 seconds
{
"id": "a702040d-2091-448f-a272-0b1197772463",
"hyperparameters": {
"training_steps": 10,
"learning_rate": 0.0001
},
"fine_tuned_model": null,
"model": "open-mistral-7b",
"status": "RUNNING",
"job_type": "FT",
"created_at": 1718200943,
"modified_at": 1718200945,
"training_files": [
"3d3fcfcc-e16d-4391-99db-87e3ef2a9c24"
],
"validation_files": [
"808e457a-53ba-4464-a4c9-eec7e3581b89"
],
"object": "job",
"integrations": [
{
"type": "wandb",
"project": "test_ft_api",
"name": null,
"run_name": "test"
}
],
"events": [
{
"name": "status-updated",
"data": {
"status": "RUNNING"
},
"created_at": 1718200945
},
{
"name": "status-updated",
"data": {
"status": "QUEUED"
},
"created_at": 1718200943
}
],
"checkpoints": [],
"estimated_start_time": null
}
Job is RUNNING, waiting 10 seconds
{
"id": "a702040d-2091-448f-a272-0b1197772463",
"hyperparameters": {
"training_steps": 10,
"learning_rate": 0.0001
},
"fine_tuned_model": null,
"model": "open-mistral-7b",
"status": "RUNNING",
"job_type": "FT",
"created_at": 1718200943,
"modified_at": 1718200945,
"training_files": [
"3d3fcfcc-e16d-4391-99db-87e3ef2a9c24"
],
"validation_files": [
"808e457a-53ba-4464-a4c9-eec7e3581b89"
],
"object": "job",
"integrations": [
{
"type": "wandb",
"project": "test_ft_api",
"name": null,
"run_name": "test"
}
],
"events": [
{
"name": "status-updated",
"data": {
"status": "RUNNING"
},
"created_at": 1718200945
},
{
"name": "status-updated",
"data": {
"status": "QUEUED"
},
"created_at": 1718200943
}
],
"checkpoints": [],
"estimated_start_time": null
}
Job is RUNNING, waiting 10 seconds
{
"id": "a702040d-2091-448f-a272-0b1197772463",
"hyperparameters": {
"training_steps": 10,
"learning_rate": 0.0001
},
"fine_tuned_model": null,
"model": "open-mistral-7b",
"status": "RUNNING",
"job_type": "FT",
"created_at": 1718200943,
"modified_at": 1718200945,
"training_files": [
"3d3fcfcc-e16d-4391-99db-87e3ef2a9c24"
],
"validation_files": [
"808e457a-53ba-4464-a4c9-eec7e3581b89"
],
"object": "job",
"integrations": [
{
"type": "wandb",
"project": "test_ft_api",
"name": null,
"run_name": "test"
}
],
"events": [
{
"name": "status-updated",
"data": {
"status": "RUNNING"
},
"created_at": 1718200945
},
{
"name": "status-updated",
"data": {
"status": "QUEUED"
},
"created_at": 1718200943
}
],
"checkpoints": [],
"estimated_start_time": null
}
Job is RUNNING, waiting 10 seconds
{
"id": "a702040d-2091-448f-a272-0b1197772463",
"hyperparameters": {
"training_steps": 10,
"learning_rate": 0.0001
},
"fine_tuned_model": null,
"model": "open-mistral-7b",
"status": "RUNNING",
"job_type": "FT",
"created_at": 1718200943,
"modified_at": 1718200945,
"training_files": [
"3d3fcfcc-e16d-4391-99db-87e3ef2a9c24"
],
"validation_files": [
"808e457a-53ba-4464-a4c9-eec7e3581b89"
],
"object": "job",
"integrations": [
{
"type": "wandb",
"project": "test_ft_api",
"name": null,
"run_name": "test"
}
],
"events": [
{
"name": "status-updated",
"data": {
"status": "RUNNING"
},
"created_at": 1718200945
},
{
"name": "status-updated",
"data": {
"status": "QUEUED"
},
"created_at": 1718200943
}
],
"checkpoints": [],
"estimated_start_time": null
}
Job is RUNNING, waiting 10 seconds
{
"id": "a702040d-2091-448f-a272-0b1197772463",
"hyperparameters": {
"training_steps": 10,
"learning_rate": 0.0001
},
"fine_tuned_model": null,
"model": "open-mistral-7b",
"status": "RUNNING",
"job_type": "FT",
"created_at": 1718200943,
"modified_at": 1718200945,
"training_files": [
"3d3fcfcc-e16d-4391-99db-87e3ef2a9c24"
],
"validation_files": [
"808e457a-53ba-4464-a4c9-eec7e3581b89"
],
"object": "job",
"integrations": [
{
"type": "wandb",
"project": "test_ft_api",
"name": null,
"run_name": "test"
}
],
"events": [
{
"name": "status-updated",
"data": {
"status": "RUNNING"
},
"created_at": 1718200945
},
{
"name": "status-updated",
"data": {
"status": "QUEUED"
},
"created_at": 1718200943
}
],
"checkpoints": [],
"estimated_start_time": null
}
Job is RUNNING, waiting 10 seconds
{
"id": "a702040d-2091-448f-a272-0b1197772463",
"hyperparameters": {
"training_steps": 10,
"learning_rate": 0.0001
},
"fine_tuned_model": null,
"model": "open-mistral-7b",
"status": "RUNNING",
"job_type": "FT",
"created_at": 1718200943,
"modified_at": 1718200945,
"training_files": [
"3d3fcfcc-e16d-4391-99db-87e3ef2a9c24"
],
"validation_files": [
"808e457a-53ba-4464-a4c9-eec7e3581b89"
],
"object": "job",
"integrations": [
{
"type": "wandb",
"project": "test_ft_api",
"name": null,
"run_name": "test"
}
],
"events": [
{
"name": "status-updated",
"data": {
"status": "RUNNING"
},
"created_at": 1718200945
},
{
"name": "status-updated",
"data": {
"status": "QUEUED"
},
"created_at": 1718200943
}
],
"checkpoints": [],
"estimated_start_time": null
}
Job is RUNNING, waiting 10 seconds
{
"id": "a702040d-2091-448f-a272-0b1197772463",
"hyperparameters": {
"training_steps": 10,
"learning_rate": 0.0001
},
"fine_tuned_model": null,
"model": "open-mistral-7b",
"status": "RUNNING",
"job_type": "FT",
"created_at": 1718200943,
"modified_at": 1718200945,
"training_files": [
"3d3fcfcc-e16d-4391-99db-87e3ef2a9c24"
],
"validation_files": [
"808e457a-53ba-4464-a4c9-eec7e3581b89"
],
"object": "job",
"integrations": [
{
"type": "wandb",
"project": "test_ft_api",
"name": null,
"run_name": "test"
}
],
"events": [
{
"name": "status-updated",
"data": {
"status": "RUNNING"
},
"created_at": 1718200945
},
{
"name": "status-updated",
"data": {
"status": "QUEUED"
},
"created_at": 1718200943
}
],
"checkpoints": [],
"estimated_start_time": null
}
Job is RUNNING, waiting 10 seconds
{
"id": "a702040d-2091-448f-a272-0b1197772463",
"hyperparameters": {
"training_steps": 10,
"learning_rate": 0.0001
},
"fine_tuned_model": null,
"model": "open-mistral-7b",
"status": "RUNNING",
"job_type": "FT",
"created_at": 1718200943,
"modified_at": 1718200945,
"training_files": [
"3d3fcfcc-e16d-4391-99db-87e3ef2a9c24"
],
"validation_files": [
"808e457a-53ba-4464-a4c9-eec7e3581b89"
],
"object": "job",
"integrations": [
{
"type": "wandb",
"project": "test_ft_api",
"name": null,
"run_name": "test"
}
],
"events": [
{
"name": "status-updated",
"data": {
"status": "RUNNING"
},
"created_at": 1718200945
},
{
"name": "status-updated",
"data": {
"status": "QUEUED"
},
"created_at": 1718200943
}
],
"checkpoints": [],
"estimated_start_time": null
}
Job is RUNNING, waiting 10 seconds
{
"id": "a702040d-2091-448f-a272-0b1197772463",
"hyperparameters": {
"training_steps": 10,
"learning_rate": 0.0001
},
"fine_tuned_model": null,
"model": "open-mistral-7b",
"status": "RUNNING",
"job_type": "FT",
"created_at": 1718200943,
"modified_at": 1718200945,
"training_files": [
"3d3fcfcc-e16d-4391-99db-87e3ef2a9c24"
],
"validation_files": [
"808e457a-53ba-4464-a4c9-eec7e3581b89"
],
"object": "job",
"integrations": [
{
"type": "wandb",
"project": "test_ft_api",
"name": null,
"run_name": "test"
}
],
"events": [
{
"name": "status-updated",
"data": {
"status": "RUNNING"
},
"created_at": 1718200945
},
{
"name": "status-updated",
"data": {
"status": "QUEUED"
},
"created_at": 1718200943
}
],
"checkpoints": [],
"estimated_start_time": null
}
Job is RUNNING, waiting 10 seconds
{
"id": "a702040d-2091-448f-a272-0b1197772463",
"hyperparameters": {
"training_steps": 10,
"learning_rate": 0.0001
},
"fine_tuned_model": null,
"model": "open-mistral-7b",
"status": "RUNNING",
"job_type": "FT",
"created_at": 1718200943,
"modified_at": 1718200945,
"training_files": [
"3d3fcfcc-e16d-4391-99db-87e3ef2a9c24"
],
"validation_files": [
"808e457a-53ba-4464-a4c9-eec7e3581b89"
],
"object": "job",
"integrations": [
{
"type": "wandb",
"project": "test_ft_api",
"name": null,
"run_name": "test"
}
],
"events": [
{
"name": "status-updated",
"data": {
"status": "RUNNING"
},
"created_at": 1718200945
},
{
"name": "status-updated",
"data": {
"status": "QUEUED"
},
"created_at": 1718200943
}
],
"checkpoints": [
{
"metrics": {
"train_loss": 0.829197,
"valid_loss": null,
"valid_mean_token_accuracy": null
},
"step_number": 10,
"created_at": 1718201037
}
],
"estimated_start_time": null
}
Job is RUNNING, waiting 10 seconds
{
"id": "a702040d-2091-448f-a272-0b1197772463",
"hyperparameters": {
"training_steps": 10,
"learning_rate": 0.0001
},
"fine_tuned_model": "ft:open-mistral-7b:82cbcaaa:20240612:a702040d",
"model": "open-mistral-7b",
"status": "SUCCESS",
"job_type": "FT",
"created_at": 1718200943,
"modified_at": 1718201052,
"training_files": [
"3d3fcfcc-e16d-4391-99db-87e3ef2a9c24"
],
"validation_files": [
"808e457a-53ba-4464-a4c9-eec7e3581b89"
],
"object": "job",
"integrations": [
{
"type": "wandb",
"project": "test_ft_api",
"name": null,
"run_name": "test"
}
],
"events": [
{
"name": "status-updated",
"data": {
"status": "SUCCESS"
},
"created_at": 1718201052
},
{
"name": "status-updated",
"data": {
"status": "RUNNING"
},
"created_at": 1718200945
},
{
"name": "status-updated",
"data": {
"status": "QUEUED"
},
"created_at": 1718200943
}
],
"checkpoints": [
{
"metrics": {
"train_loss": 0.829197,
"valid_loss": null,
"valid_mean_token_accuracy": null
},
"step_number": 10,
"created_at": 1718201037
}
],
"estimated_start_time": null
}
Job is SUCCESS, waiting 10 seconds
{
"data": [
{
"id": "a702040d-2091-448f-a272-0b1197772463",
"hyperparameters": {
"training_steps": 10,
"learning_rate": 0.0001
},
"fine_tuned_model": "ft:open-mistral-7b:82cbcaaa:20240612:a702040d",
"model": "open-mistral-7b",
"status": "SUCCESS",
"job_type": "FT",
"created_at": 1718200943,
"modified_at": 1718201052,
"training_files": [
"3d3fcfcc-e16d-4391-99db-87e3ef2a9c24"
],
"validation_files": [
"808e457a-53ba-4464-a4c9-eec7e3581b89"
],
"object": "job",
"integrations": [
{
"type": "wandb",
"project": "test_ft_api",
"name": null,
"run_name": "test"
}
]
}
],
"object": "list"
}
{
"id": "a702040d-2091-448f-a272-0b1197772463",
"hyperparameters": {
"training_steps": 10,
"learning_rate": 0.0001
},
"fine_tuned_model": "ft:open-mistral-7b:82cbcaaa:20240612:a702040d",
"model": "open-mistral-7b",
"status": "SUCCESS",
"job_type": "FT",
"created_at": 1718200943,
"modified_at": 1718201052,
"training_files": [
"3d3fcfcc-e16d-4391-99db-87e3ef2a9c24"
],
"validation_files": [
"808e457a-53ba-4464-a4c9-eec7e3581b89"
],
"object": "job",
"integrations": [
{
"type": "wandb",
"project": "test_ft_api",
"name": null,
"run_name": "test"
}
],
"events": [
{
"name": "status-updated",
"data": {
"status": "SUCCESS"
},
"created_at": 1718201052
},
{
"name": "status-updated",
"data": {
"status": "RUNNING"
},
"created_at": 1718200945
},
{
"name": "status-updated",
"data": {
"status": "QUEUED"
},
"created_at": 1718200943
}
],
"checkpoints": [
{
"metrics": {
"train_loss": 0.829197,
"valid_loss": null,
"valid_mean_token_accuracy": null
},
"step_number": 10,
"created_at": 1718201037
}
],
"estimated_start_time": null
}
Testing Fine Tuned Model:
{
"id": "8b21421d3fcd43cb973a1369bcf3493f",
"object": "chat.completion",
"created": 1718201068,
"model": "ft:open-mistral-7b:82cbcaaa:20240612:a702040d",
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": "The best French cheese can be quite subjective, but some popular choices include
Roquefort, Brie de Meaux, Camembert de Normandie, and Comt\u00e9. Roquefort is a strong blue-veined sheep's milk
cheese, Brie de Meaux is a soft, creamy cheese, Camembert de Normandie is a soft, creamy cow's milk cheese, and
Comt\u00e9 is a semi-hard, nutty cow's milk cheese. The choice of French cheese depends on personal preference.",
"name": null,
"tool_calls": null,
"tool_call_id": null
},
"finish_reason": "stop"
}
],
"usage": {
"prompt_tokens": 10,
"total_tokens": 124,
"completion_tokens": 114
}
}

Conclusion

Mistral’s fine-tuning options are a transformative development in the world of AI. By offering accessible tools and customization, they empower developers and businesses to leverage the power of AI in a way that perfectly aligns with their specific goals. This opens doors for innovation across various industries, allowing anyone to tap into the potential of AI and create groundbreaking solutions. With Mistral’s approach, the future of AI is no longer a one-size-fits-all proposition, but rather a personalized culinary experience, where you can use your unique recipe to craft the perfect dish for your needs.

Resources

Stay connected and support my work through various platforms:

Github Patreon Kaggle Hugging-Face YouTube GumRoad Calendly

Like my content? Feel free to Buy Me a Coffee ☕ !

Requests and questions: If you have a project in mind that you’d like me to work on or if you have any questions about the concepts I’ve explained, don’t hesitate to let me know. I’m always looking for new ideas for future Notebooks and I love helping to resolve any doubts you might have.

Remember, each “Like”, “Share”, and “Star” greatly contributes to my work and motivates me to continue producing more quality content. Thank you for your support!

If you enjoyed this story, feel free to subscribe to Medium, and you will get notifications when my new articles will be published, as well as full access to thousands of stories from other authors.

--

--

Ankush k Singal
AI Artistry

My name is Ankush Singal and I am a traveller, photographer and Data Science enthusiast .