LOCAL LLM FIELD GUIDE
Choose Local LLM hardware from the job, not a GPU name
Choose from required memory, bandwidth, runtime, concurrency, and operating model. Macs suit single-device local workflows. consumer GPUs suit personal or experimental use. workstation and datacenter GPUs serve larger memory or scale requirements.
CHOOSE BY CONSTRAINT
A tier is an operating model, not a ranking
USE IT ON REAL WORK
Choose hardware from the deployment path
Start with the runtime and artifact you must use. They determine the accelerator path before price. Then compare reserved memory, bandwidth, and operating effort.
More VRAM does not automatically make a workflow easier. Compare the model, context, and concurrent requests with a real path to run them.
| Option | Start with | Verify |
|---|---|---|
| Mac Apple Silicon | MLX or GGUF on Metal | Unified memory, conversion path, local workflow |
| NVIDIA | A supported CUDA runtime and kernel | VRAM, bandwidth, driver, serving topology |
| AMD | ROCm and HIP support in the runtime | GPU model, kernel path, artifact support |
Questions before buying or renting
- Which model and quantization must run
- How many working-context tokens and concurrent sequences
- Whether TTFT, per-user tok/s, or aggregate throughput matters
- How much Linux, CUDA, or ROCm operation the team can support
- Whether the plan is one device, replicas, or tensor parallel
hardware = <device>
framework = <runtime>
model = <checkpoint>
working_context = <tokens>
batch = <concurrent sequences>Mac Apple Silicon
Unified memory can fit large models on one device through MLX or GGUF/Metal. It is strong for quiet interactive local work, but CUDA-specific quantization and some serving paths are not direct fits.
NVIDIA and AMD
Start from the runtime and kernel your team will use. NVIDIA has a broad server-inference ecosystem. on AMD, verify HIP/ROCm and runtime/quant support for the exact device. VRAM alone is not enough.
Consumer, workstation, datacenter
Consumer GPUs offer an accessible starting cost but limited memory per card. Workstations emphasize memory and reliability. Datacenter GPUs fit capacity, fabric, isolation, and serving operations at higher cost and complexity.
Questions before buying
Which model and quantization must run, at what context and sequence count, with which latency or throughput goal, and can the team operate Linux/CUDA/ROCm? Put those answers into the calculator first.
Sources
Try your configuration
Put the intended model, quantization, context, framework, and hardware into the calculator, then validate with a production-like benchmark.
Open calculator