Classification microservice with CrewAI on RTX Spark
Build a classification microservice on RTX Spark using CrewAI. Stack, models, code, and production checklist.
Architecture
CrewAI coordinates tool calls, memory, and retrieval. vLLM serves the model locally at http://localhost:8000/v1. Data never leaves the Spark.
Setup
Install CrewAI, vLLM, and a vector store (Qdrant recommended). Wire CrewAI's LLM adapter to the local endpoint.
- pip install crewai
- vllm serve qwen3-32b --quantization q4_k_m
- docker run -p 6333:6333 qdrant/qdrant
Reference implementation
~150 lines of Python: ingestion, retrieval, tool schemas, and the CrewAI graph/crew.
Production checklist
Add tracing (Langfuse or Phoenix), rate limiting, and a fallback route to a smaller model when p95 breaches SLO.
Cost
~$0.02 per 1M tokens amortized. Compare to cloud GPT-class equivalents at $3–15 per 1M.
Frequently asked questions
Why CrewAI instead of LangGraph?
CrewAI favors role-based agent teams with less boilerplate.
Can I swap the model later?
Yes — the adapter is OpenAI-compatible; swap by env var.