Customer support agent with LlamaIndex on RTX Spark
Build a customer support agent on RTX Spark using LlamaIndex. Stack, models, code, and production checklist.
Architecture
LlamaIndex coordinates tool calls, memory, and retrieval. vLLM serves the model locally at http://localhost:8000/v1. Data never leaves the Spark.
Setup
Install LlamaIndex, vLLM, and a vector store (Qdrant recommended). Wire LlamaIndex's LLM adapter to the local endpoint.
- pip install llama-index
- vllm serve qwen3-32b --quantization q4_k_m
- docker run -p 6333:6333 qdrant/qdrant
Reference implementation
~150 lines of Python: ingestion, retrieval, tool schemas, and the LlamaIndex graph/crew.
Production checklist
Add tracing (Langfuse or Phoenix), rate limiting, and a fallback route to a smaller model when p95 breaches SLO.
Cost
~$0.02 per 1M tokens amortized. Compare to cloud GPT-class equivalents at $3–15 per 1M.
Frequently asked questions
Why LlamaIndex instead of LangGraph?
LlamaIndex favors role-based agent teams with less boilerplate.
Can I swap the model later?
Yes — the adapter is OpenAI-compatible; swap by env var.