KNOWLEDGE BASE

Local LLM Guides: hardware, models, quantization, and serving

A practical guide library for choosing local LLM hardware, models, quantization, KV cache, serving, and performance paths.

Quantization: what is reduced, and how to chooseChoose local LLM weight and KV-cache quantization from the runtime, memory budget, and quality risk.Why LLM tok/s often hits memory bandwidthWhy LLM decode tok/s is often memory-bandwidth-limited, and how to read calculator estimates responsibly.Choose Local LLM hardware from the job, not a GPU nameChoose local LLM hardware across NVIDIA, AMD, and Mac plus consumer, workstation, and datacenter GPUs by workload and memory budget.Serving frameworks: choose by serving shapeSelect a local LLM serving framework with a decision tree: when to use vLLM, SGLang, llama.cpp, MLX, Ollama, and TensorRT-LLM.Choose Local LLMs by work: Thai, chatbot, agent, coding, workflowSelect local LLMs for Thai, chatbots, agents, coding, and structured workflows by workload profile instead of ungrounded rankings.Speculative decoding and attention backends: faster only when the conditions fitUnderstand speculative decoding, acceptance rate, and attention backends for local LLMs, including when they improve or hurt performance.