Mistral AI · Mistral · based on mistralai/Mistral-Small-24B-Base-2501
Mistral Small 3 (24B)
A fast 24B Apache-2.0 model tuned for low latency; a single 48 GB GPU serves moderate traffic at INT4.
48,741 downloads · 970 likes · 10,825 GitHub stars · Model card · GitHub · synced 2026-09-19
Specifications
- Parameters
- 23.6B
- Architecture
- dense · mistral
- Layers
- 40
- Hidden size
- 5,120
- KV heads · head dim
- 8 · 128
- Vocabulary
- 131,072
- Native dtype
- bfloat16
- Context window
- 32,768 tokens
- Released
- 2025-01-28
- Last modified on Hub
- 2025-07-28
Features & licensing
- License
- Apache-2.0
- Commercial use
- Yes
- Modalities
- text
- Tool / function calling
- No
- Reasoning mode
- No
- Languages
- EN, FR, DE, ES, IT, PT, ZH, JA, RU, KO
- Quantised variants on Hub
- AWQ, EXL2, FP8, GGUF, bitsandbytes
- Pipeline
- —
Task fit
Editorial scores (0–100) used by the recommendation engine.
- Chat82
- RAG / Q&A80
- Code78
- Summarisation80
- Extraction82
- Agents80
Hardware to run Mistral Small 3 (24B)
Weights need about 52 GB at FP16, 27 GB at INT8 and 16 GB at INT4. The KV cache adds roughly 164 MB per 1,000 tokens per request. For a reference workload of 5 requests per second with 1,500 input and 300 output tokens, total GPU memory is around 22.9 GB, which fits on 1 × NVIDIA A100 (80 GB).
| Configuration | VRAM | Utilisation | Est. first token | Cloud / month |
|---|---|---|---|---|
| 2 × NVIDIA L4 (24 GB) | 48 GB | 48 % | ~0.5 s | $1,168 |
| 1 × NVIDIA L40S (48 GB) | 48 GB | 48 % | ~0.2 s | $1,387 |
| 1 × NVIDIA A100 (80 GB)recommended | 80 GB | 29 % | ~0.1 s | $2,336 |
| 1 × NVIDIA H100 (80 GB) | 80 GB | 29 % | ~0.0 s | $3,285 |
Estimates only. Run an assessment for your own traffic.
Mistral Small 3 is designed for latency-sensitive deployments and function calling. Its 32K window suits chat and short-document workloads.