Mistral Small 3 24B on RTX Spark vs NVIDIA H100 (cloud) (2026)
Should you run Mistral Small 3 24B on RTX Spark or NVIDIA H100 (cloud)? Real benchmarks, break-even, and hidden costs.
Setup
We ran Mistral Small 3 24B at Q4_K_M on RTX Spark 128GB and on NVIDIA H100 (cloud). Same prompts (4k/512), same temperature 0, same eval harness. Reported numbers are p50 across 200 requests.
| Metric | RTX Spark | NVIDIA H100 (cloud) |
|---|---|---|
| Tok/s (single) | 42 | 109 |
| First-token ms | 95 | 60 |
| $/1M output tok | ~$0.02 amortized | $3.20 |
| Egress cost | $0 | $0.09/GB typical |
| Data locality | 100% local | External provider |
Cost model
At 50k output tokens/day, NVIDIA H100 (cloud) costs $4800/month vs a Spark that pays itself off in ~2 months and then costs ~$16.13/month in electricity.
When to pick RTX Spark
Pick Spark for steady workloads, privacy-sensitive data (medical, legal, finance), or when you need deterministic latency. Pick NVIDIA H100 (cloud) for bursty spikes or when you need >5× the peak throughput a Spark can offer.
Hybrid strategy
Route the 90% of easy requests to local Spark and burst the 10% hardest to NVIDIA H100 (cloud). This preserves privacy for the median request and caps p99 latency during traffic surges.
Frequently asked questions
Is Mistral Small 3 24B faster on NVIDIA H100 (cloud) than on RTX Spark?
Yes, roughly 2.6×.
What's the break-even?
About 2 months at 50k tokens/day.
Does Mistral Small 3 24B support NVFP4 on NVIDIA H100 (cloud)?
Partially — NVFP4 is Blackwell-only; H100 uses FP8.