Local LLM VRAM & Memory Calculator

• Apple Silicon: macOS shares RAM dynamically between CPU and GPU. A 16GB Mac gives ~12GB to Metal, fitting 8B models comfortably.

• Windows & Linux (NVIDIA): If your model exceeds VRAM, Ollama will automatically split layers between GPU and CPU, but tokens/sec will drop by 70–80%.

• KV Cache Memory: At 64k+ context, the KV cache can consume more memory than the model itself. Use Q8 or Q4 KV cache in llama.cpp to cut this overhead by half.