How-toUpdated 2026-06-30 · 10 min read · by RTXsparks Lab
ORPO alignment for Llama 3.2 90B Vision on RTX Spark
ORPO alignment recipe for Llama 3.2 90B Vision on RTX Spark: prerequisites, script, and evaluation.
Prerequisites
Ubuntu 24.04, driver 585+, CUDA 12.6+, PyTorch 2.6, and the Llama 3.2 90B Vision weights at Q4_K_S.
Recipe
Config file, run script, and expected timing for orpo alignment on Llama 3.2 90B Vision.
Evaluation
Run the standard eval harness pre/post to quantify quality change. Report perplexity, MMLU delta, and task-specific F1.
Deploy the tuned model
Merge adapters, re-quantize, and serve via vLLM. Verify tokenizer parity and JSON-mode.
Frequently asked questions
Does ORPO alignment fit on Spark 128GB?
Yes with gradient checkpointing and CPU offload for optimizer states.
Is it worth it vs base?
For narrow domains: yes; for general chat: usually not.