Serving Qwen3-Coder 32B with Triton Inference Server on RTX Spark
Production Triton Inference Server deployment of Qwen3-Coder 32B on RTX Spark. Configs, systemd, TLS, observability.
Install
Install Triton Inference Server via pip/docker/apt as appropriate. Verify GPU visibility with nvidia-smi.
- # Triton Inference Server install
- # see Triton Inference Server docs
Config
Optimized Triton Inference Server config for Qwen3-Coder 32B at Q4_K_M: batch, KV cache dtype, and parallelism knobs.
Systemd unit
[Unit] Description=Triton Inference Server for Qwen3-Coder 32B — [Service] ExecStart=/usr/bin/triton serve qwen3-coder-32b --port 8000 — Restart=always.
nginx + TLS
Terminate TLS at nginx with a Let's Encrypt cert, rate-limit at 20 req/s per IP, and proxy to the local Triton Inference Server port.
Observability
Scrape Triton Inference Server Prometheus metrics into Grafana; watch tokens_per_second, batch_size, and kv_cache_usage.
Frequently asked questions
Is Triton Inference Server the fastest for Qwen3-Coder 32B?
It's the easiest, not always the fastest.
Can I run multiple models with one instance?
One process per model is standard.