Local AI Hardware Matrix & Performance Estimates
Find out exactly which models run on your machine. Compare estimated tokens/second, VRAM requirements, and runner performance across Apple Silicon, Qualcomm Snapdragon, and NVIDIA RTX GPUs.
Models Directory
Hardware requirements, VRAM matrices, and 1-click Ollama commands for Qwen3.8, Gemma 4, gpt-oss, DeepSeek and more.
Browse Models →Hardware Matrix
Faceted database of 75 Apple Silicon, Snapdragon X, and RTX systems profiled for 7B/8B/70B local LLM inference (estimates).
Browse 75 Devices →API Pricing & Break-Even
Compare token prices across Claude, GPT-5.6, Gemini, DeepSeek and Groq. Calculate the exact month local hardware breaks even.
Compare API Costs →VRAM & KV Calculator
Calculate model weights, KV cache buffers, and context scaling up to 256k tokens. Includes custom model sandbox.
Calculate Memory →What LLM Can I Run?
Start with your VRAM. Eight tiers from 8GB to 128GB, each scoring every model and the best quantization it can use.
Pick Your VRAM →Image Model Directory
FLUX.2 and FLUX.1, SDXL, SD 3.5 and Qwen-Image. Component sizes from the real safetensors files, plus which ones you can ship.
Browse Image Models →Image Gen VRAM Calculator
FLUX.2 and FLUX.1, SDXL, SD 3.5 and Qwen-Image. Peak VRAM by resolution, batch size, precision and offload strategy.
Size a Diffusion Model →Model Licence Checker
Can you actually ship it? Apache 2.0 vs Llama Community vs Gemma vs CC-BY-NC, and the obligations teams miss.
Check a Licence →GGUF Size Calculator
How big is a 70B at Q4_K_M? File size at every quantization, from bits-per-weight measured on real GGUF files.
Size a Model →Model Recommender
Input your RAM and primary task (Coding, Reasoning, Chat, RAG) to find your ideal open-source model.
Get Recommendations →Side-by-Side Compare
Direct head-to-head comparison of memory bandwidth, token generation rates, and runner support across popular systems.
Compare Matchups →Mac VRAM Unlocker
Safely raise the GPU memory limit on Apple Silicon (M1 to M6). Generates the sysctl command for your machine.
Optimize Mac VRAM →Hardware Profiles
45 Platforms
Apple M-Series & Snapdragon
Discrete GPUs
30 GPUs
NVIDIA RTX & AMD ROCm
AI Models
30 Models
Qwen3.8, Gemma 4, gpt-oss
Software Runners
4 Engines
Ollama, LM Studio, vLLM
Featured Compatibility Estimates
Estimated tokens-per-second and memory needs for popular 2026 setups. Estimates from memory bandwidth, not lab measurements.
Apple M5 Max (128GB)
Estimated from 614 GB/s bandwidth and ~5.1B active parameters (MoE).
AMD Ryzen AI Max+ 395 (128GB)
Estimated from 256 GB/s; MoE models suit Strix Halo well.
RTX 3090 (24GB)
Estimated from 936 GB/s with the whole model in VRAM.
Apple M5 Pro (48GB)
Estimated; only ~3B parameters are active per token.
RTX 5060 Ti (16GB)
Estimated; fits 16GB with modest context.
Apple M6 (16GB)
Estimated from 153 GB/s, the 16GB configuration's unified memory bandwidth.
Supported Local Inference Engines
Running local models depends heavily on matching your hardware accelerator (Metal, CUDA, DirectML, ROCm) to the right runner. Each engine has its own page with per-model memory costs and the commands to load them.
Ollama
Backends: Metal, CUDA, ROCm, CPU
OS: macOS, Linux, Windows
LM Studio
Backends: Metal, CUDA, DirectML, CPU
OS: macOS, Windows, Linux
vLLM
Backends: CUDA, ROCm
OS: Linux
llama.cpp
Backends: Metal, CUDA, Vulkan, SYCL, CPU
OS: macOS, Linux, Windows
Interfaces and image generation
An engine answers requests; these are what you put in front of it, and what to read before you pick either.
Start with these guides
The reads that answer most local-AI questions before you touch a calculator.
- How Much RAM and VRAM Do You Need to Run Local LLMs?Model weights, KV cache and OS overhead, tier by tier.Read the guide
- Best Local LLM Runners: Ollama vs LM Studio vs Jan vs llama.cpp vs vLLMWhich runner suits your hardware and workflow.Read the guide
- Tokens per Second ExplainedWhat sets local LLM speed, and the formula behind our estimates.Read the guide
- LLM Context Windows: What They Cost You in VRAMThe KV-cache arithmetic behind the VRAM calculator.Read the guide
- Best Budget GPUs for Local LLMs16GB cards, used RTX 3090s and current street prices.Read the guide
- Best Vector Database for Local RAGWhat the index really costs in RAM, computed from published formulas.Read the guide