Mistral AI sets code generation benchmark with Codestral 25.01

Mistral AI has been a leader in the code generation field and its latest model, Codestral 25.01, continues to push boundaries. This upgraded iteration promises even greater speed and capability, setting a new benchmark for coding models in its class.

Described as lightweight, fast, and proficient across more than 80 programming languages, the original Codestral proved invaluable for developers tackling tasks like fill-in-the-middle (FIM), code correction, and test generation. The model quickly gained recognition for its ability to enhance productivity, serving as a reliable coding companion for developers worldwide.

With a refined architecture and an upgraded tokeniser, Codestral 25.01 delivers code generation and completion abilities that are double the speed of its predecessor. For FIM use cases, Codestral 25.01 now leads the pack and is the definitive SOTA model in its weight class.  

Codestral 25.01 benchmarks  

To grasp just how capable Codestral 25.01 is, Mistral AI has benchmarked its performance against top sub-100 billion parameter coding models that are widely considered to excel in FIM tasks.  

Key coding metrics speak volumes about the advancements made with the new model:  

• In Python-focused HumanEval tests: Codestral 25.01 scored an impressive 86.6%, outclassing its 22-billion parameter predecessor, Codestral 2405, and even competitors like Codellama 70B Instruct and DeepSeek Coder 33B Instruct.  
• For JavaScript: Codestral 25.01 achieved an extraordinary HumanEvalFIM score of 87.96%, maintaining a clear lead over rivals.  
• Across FIM pass@1 tasks: Codestral 25.01 earned a stellar average score of 95.3%, surpassing industry benchmarks and APIs like OpenAI’s GPT-3.5 Turbo.  

These results reflect Codestral 25.01’s efficiency and precision, qualities that are crucial to developers navigating increasingly complex coding tasks.  

Designed for developers

The advancements in Codestral 25.01 reflect a strong focus on practical applications and developer feedback. Whether deployed via integrated development environment (IDE) partners or embedded in enterprise systems, the model is engineered for real-world usability.

Developers using IDEs with Codestral 25.01 support will immediately notice enhancements in code completion speed and accuracy. This capability, paired with its adept handling of FIM and other coding challenges, cements Codestral’s reputation as a go-to solution for agile software development.

“For AI code assistants, code completion constitutes a large portion of the work, which requires models that are great at fill-in-the-middle (FIM),” explains Ty Dunn, co-founder of Continue, a company providing enterprise-grade deployments of Mistral’s models.

“Mistral AI’s new model is capable of providing more precise suggestions, much faster—a critical component of accurate, efficient software development. This is why Codestral is our recommended autocomplete model for developers.”

Accessible across platforms  

Mistral AI is bringing Codestral 25.01 to developers through multiple avenues. It is available on leading APIs, including Google Cloud’s Vertex AI, and is in private preview on Azure AI Foundry, with availability on Amazon Bedrock coming soon. The flexibility ensures that developers and enterprises alike can integrate Codestral 25.01 into their preferred workflows.

Additionally, Mistral offers open access to its Codestral documentation and API on its la Plateforme portal. Developers can experiment with integrations, ensuring the model’s capabilities align with their specific needs.

For those interested in trying out Codestral 25.01, the model is accessible through IDE plugins for platforms like VS Code and JetBrains.  

With Codestral 25.01, Mistral AI reaffirms its position among the frontrunners of the coding model revolution. Its blend of efficiency, accuracy, and scalability sets the stage for further innovations that promise to reshape software development as we know it.

See also: Software development trends and predictions for 2025

Want to learn more about AI and big data from industry leaders? Check out AI & Big Data Expo taking place in Amsterdam, California, and London. The comprehensive event is co-located with other leading events including Intelligent Automation Conference, BlockX, Digital Transformation Week, and Cyber Security & Cloud Expo.

Explore other upcoming enterprise technology events and webinars powered by TechForge here.

Tags: AI, artificial intelligence, code generation, codestral, coding, development, mistral ai, programming, tools