Vérificateur VRAM LLM

This calculator estimates VRAM requirements using the fundamental relationship between model parameters and memory. At full precision (FP32), each parameter requires 4 bytes; at half-precision (FP16/BF16), 2 bytes per parameter. Quantized formats reduce this further — Q8_0 uses approximately 1 byte per parameter, Q4_K_M uses roughly 0.5 bytes, and Q2_K approximately 0.25 bytes. The KV-cache memory is estimated based on the model's architecture: number of layers, attention heads, head dimension, and the configured context length. The formula accounts for both key and value tensors stored in FP16 format. Our GPU database includes specifications from NVIDIA (GeForce, Quadro, Tesla, A100, H100 series), AMD (Radeon, Instinct series), and Apple Silicon (M1-M3 with unified memory). Specifications are sourced from official manufacturer data sheets. Limitations: Actual VRAM usage varies by inference framework (llama.cpp, vLLM, TGI, Ollama), operating system overhead, and specific model architecture details. Our estimates include a 10% overhead buffer but real-world usage may differ by 5-15%. Multi-GPU setups using tensor parallelism may have additional communication overhead not captured in these estimates.