Codestral [Part 1]: Empowering Developers Through the Power of AI

What is Codestral?

Codestral, the first-ever code model, is an open-weight generative AI model explicitly designed for code generation tasks. It enables developers to write and interact with code through a shared instruction and completion API endpoint. By mastering both code and English, Codestral opens up exciting possibilities for designing advanced AI applications tailored to software developers.

Fluent in 80+ Programming Languages

Codestral is trained on a diverse dataset covering 80+ programming languages. From Python and Java to C++, JavaScript, and Bash, it excels across a wide range of languages. Even more specialized languages like Swift and Fortran pose no challenge for Codestral. This broad language base ensures that developers can rely on Codestral in various coding environments and projects.

Time-Saving Features

Codestral streamlines the development process by offering several time-saving features:

1. Code Completion: Codestral can complete coding functions, write tests, and fill in partial code using a sophisticated fill-in-the-middle mechanism.
2. Error Reduction: Interacting with Codestral helps level up your coding game and reduces the risk of errors and bugs.

Setting New Standards for Code Generation Performance

As a 22B model, Codestral sets a new bar for performance and latency in code generation compared to previous models. Let’s dive into the details:

Detailed Benchmarks

1. Context Window: Codestral’s larger context window (32k compared to competitors’ 4k, 8k, or 16k) contributes to its superior performance in RepoBench, a long-range evaluation for code generation.
2. Python Evaluation: Codestral’s Python code generation ability is assessed using four benchmarks: HumanEval pass@1, MBPP sanitized pass@1, CruxEval (Python output prediction), and RepoBench EM (Long-Range Repository-Level Code Completion).
3. SQL Performance: Codestral’s performance in SQL is evaluated using the Spider benchmark.
4. Additional Languages: Codestral’s performance in C++, bash, Java, PHP, Typescript, and C# is evaluated, with results averaged across multiple HumanEval pass@1 evaluations.
5. Fill-in-the-Middle (FIM): Codestral’s FIM performance is compared to DeepSeek Coder 33B in Python, JavaScript, and Java.

Getting Started with Codestral

1. Download and Test: Codestral is a 22B open-weight model available under the new Mistral AI Non-Production License. You can download it from HuggingFace for research and testing purposes.
2. Dedicated Endpoint: Use codestral.mistral.ai for Instruct or Fill-In-the-Middle routes within your IDE. The API Key for this endpoint is personal and not bound by organization rate limits.

Conclusion

Codestral is an open-weight generative AI model designed for code generation tasks. It excels in over 80 programming languages, streamlining development with features like code completion and error reduction. As a 22B model, Codestral sets new standards for performance and latency.