Mistral Inference

Mistral Inference is the official Python library from Mistral AI for running their open-weight language models locally. It provides a streamlined way to download, load, and run inference on Mistral's entire model family. With Mistral Inference, developers can run models like Mistral 7B, Mixtral 8x7B, Mixtral 8x22B, Codestral 22B, Mathstral 7B, Mistral Nemo, Mistral Large 2, Pixtral 12B, and Mistral Small 3.1 on their own hardware. The library supports both CLI-based demo commands for quick testing and Python APIs for programmatic inference. Installation is straightforward via pip (\`pip install mistral-inference\`) or from source using Poetry. Models can be downloaded from Mistral's CDN or Hugging Face Hub. The library uses xformers for optimized attention and supports multi-GPU setups through torchrun for larger models like the 8x7B and 8x22B Mixtral variants. All Mistral models support function calling capabilities, enabling structured output and tool use. The library also works with Hugging Face's safetensors format for easy model weight management. Several model variants use custom licenses (MNPL for Codestral, MRL for Mistral Large 2), so users should review license terms before deployment. Mistral Inference is ideal for developers who need private, local AI inference without API dependency. It is actively maintained by Mistral AI with 10,800+ GitHub stars and 1,000+ forks, Apache 2.0 licensed for the library code itself (model weights have their own licenses). The repository includes tutorials in Jupyter notebook format via Google Colab for getting started quickly.