A MoE architecture: all 47B parameters sit in memory, but only a fraction runs per token — hence 13B-class speed at noticeably better quality. A good compromise when throughput matters.
| MMLU | 70.6 | |
|---|---|---|
| GSM8K | 74.4 | |
| HumanEval | 40.2 |
Which GPU does Mixtral 8x7B need?
The entry configuration is 1x A100, which delivers around 55 tokens per second. For more parallel requests and headroom on speed, go with 1x H200.
How much does Mixtral 8x7B cost?
An hour of Mixtral 8x7B runs from $2.21 to $8.95 depending on the GPU configuration. Billing is hourly: you pay for server time, not for tokens.
How do I call Mixtral 8x7B from my code?
Once deployed you get an OpenAI-compatible endpoint: point any OpenAI client at the base_url you receive and pass mixtral:8x7b as the model name. Code written against the OpenAI API needs no changes.
Does the data stay private?
Yes. Mixtral 8x7B runs on a dedicated GPU server inside a private environment: requests and responses never reach the model vendor and are not used for training.
What context window and license does Mixtral 8x7B have?
Mixtral 8x7B holds 32K tokens of context per request and ships under the Apache 2.0 license, so you can use it commercially on your own hosting terms.