How-toUpdated 2026-07-01 · 9 min read · by RTXsparks Lab
Serving DeepSeek Coder V2 236B with SGLang on RTX Spark
Production SGLang deployment of DeepSeek Coder V2 236B on RTX Spark. Configs, systemd, TLS, observability.
Install
Install SGLang via pip/docker/apt as appropriate. Verify GPU visibility with nvidia-smi.
- # SGLang install
- # see SGLang docs
Config
Optimized SGLang config for DeepSeek Coder V2 236B at Q3_K_M: batch, KV cache dtype, and parallelism knobs.
Systemd unit
[Unit] Description=SGLang for DeepSeek Coder V2 236B — [Service] ExecStart=/usr/bin/sglang serve deepseek-coder-v2-236b --port 8000 — Restart=always.
nginx + TLS
Terminate TLS at nginx with a Let's Encrypt cert, rate-limit at 20 req/s per IP, and proxy to the local SGLang port.
Observability
Scrape SGLang Prometheus metrics into Grafana; watch tokens_per_second, batch_size, and kv_cache_usage.
Frequently asked questions
Is SGLang the fastest for DeepSeek Coder V2 236B?
It's the easiest, not always the fastest.
Can I run multiple models with one instance?
One process per model is standard.