The short answer LoRA training memory is decided by three things, and network rank is not one of them. What matters is the precision you hold the frozen base model at, whether you cache latents and text embeddings before training starts, and the training resolution. ModelBF16FP8NF4Smallest card (FP8) SD 1.5 (512×512)2.9 GB——6GB SDXL (1024×1024)6.9 GB4.3 GB—8GB SD 3.5 … [Read more...] about LoRA Training VRAM Requirements: Flux, SDXL and SD 3.5 (2026)
Best GPU for AI in 2026: VRAM Tiers, Bandwidth and What Actually Fits
The short answer For local AI the ranking that matters is not the one on gaming benchmark charts. VRAM capacity decides which models you can run at all. Memory bandwidth decides how fast they run. Those are two different specifications, and the second one is worthless if the first one fails. If you want a single recommendation: a used RTX 3090 with 24GB remains the best value … [Read more...] about Best GPU for AI in 2026: VRAM Tiers, Bandwidth and What Actually Fits
How Much VRAM Do You Need to Fine-Tune an LLM? (2026 Requirements)
The short answer Fine-tuning needs far more memory than running the same model, and the gap is not small. Llama 3.1 8B runs comfortably on an 8GB card at Q4_K_M. Fine-tuning every parameter of it with 32-bit AdamW needs roughly 130GB. QLoRA brings that back down to about 5.9GB — which is why almost nobody does full fine-tuning on their own hardware. Here is what each method … [Read more...] about How Much VRAM Do You Need to Fine-Tune an LLM? (2026 Requirements)
Best Local Embedding Models for RAG in 2026: Sizes, Dimensions & Licences
If you are building a local RAG system in 2026, the embedding model matters more than the LLM you plug it into. A weak retriever hands your model the wrong chunks, and no amount of parameters downstream will fix that. The good news is that the best open embedding models are small — most run comfortably in under 1 GB, alongside your main model, on hardware you already … [Read more...] about Best Local Embedding Models for RAG in 2026: Sizes, Dimensions & Licences
How to Run Qwen3.8 27B Locally: Hardware, VRAM & Setup Guide (2026)
Qwen3.8 27B is Alibaba's newest open-weight model, released in August 2026. It's one of the strongest models you can realistically run on your own computer. The short version: you need about 18GB for the model at 4-bit, plus room for context, so a 24GB graphics card or a Mac with 32GB or more of unified memory. This guide covers exact memory needs by quantization and context … [Read more...] about How to Run Qwen3.8 27B Locally: Hardware, VRAM & Setup Guide (2026)
AMD Strix Halo vs Apple M5 Max: The 128GB Unified Memory Local AI Showdown
If you want to run big AI models locally without buying several expensive GPUs, two platforms offer up to 128GB of fast unified memory in a small box: AMD's Ryzen AI Max+ 395 ("Strix Halo") and Apple's M5 Max, in the 2026 Mac Studio and the MacBook Pro. Both can hold 70B-class models and ~120B Mixture-of-Experts models that won't fit on a consumer GPU. This guide compares them … [Read more...] about AMD Strix Halo vs Apple M5 Max: The 128GB Unified Memory Local AI Showdown
Best Local LLM Runners in 2026: Ollama vs LM Studio vs Jan vs llama.cpp vs vLLM
To run an AI model on your own computer you need a runner: the software that loads the model and serves it to you or your apps. The five most popular are Ollama, LM Studio, Jan, llama.cpp and vLLM. They overlap a lot (four of them are built on or around llama.cpp), but each suits a different kind of user. This guide compares them as of September 2026 and helps you pick. If your … [Read more...] about Best Local LLM Runners in 2026: Ollama vs LM Studio vs Jan vs llama.cpp vs vLLM
Running Giant Open Models Locally: DeepSeek V4, Qwen3.8 & Llama 4 Hardware Sizing Guide
The biggest open-weight models are now Mixture-of-Experts (MoE) giants with hundreds of billions, or even trillions, of parameters. Can you run them at home? Sometimes, if you understand one key idea: MoE models need memory for all their parameters, but only compute a small "active" slice for each token. This guide explains how to size hardware for today's largest open models, … [Read more...] about Running Giant Open Models Locally: DeepSeek V4, Qwen3.8 & Llama 4 Hardware Sizing Guide
Quantization Guide for Local LLMs: Q4_K_M vs Q8_0 vs FP8 (DeepSeek R1 Examples)
When you download a model for Ollama or LM Studio, you'll see labels like Q4_K_M, Q8_0 or FP8. These are quantization levels: ways of compressing a model's weights so it takes less memory and runs faster, at some cost to accuracy. This guide uses the DeepSeek R1 distills as examples, but the same rules apply to Qwen, Gemma, gpt-oss and other models. It explains what each format … [Read more...] about Quantization Guide for Local LLMs: Q4_K_M vs Q8_0 vs FP8 (DeepSeek R1 Examples)
NVIDIA RTX 5080 vs RTX 4090 for Local LLMs: Why 16GB VRAM Is the Limit
On paper, NVIDIA's RTX 5080 is the newer, more efficient card. For gaming it's roughly in the RTX 4090's league. For local AI, though, the comparison comes down to one number: 16GB of VRAM on the 5080 versus 24GB on the 4090. That 8GB gap decides which models you can run entirely on the GPU. This guide explains the practical difference and what it means for buyers in 2026, when … [Read more...] about NVIDIA RTX 5080 vs RTX 4090 for Local LLMs: Why 16GB VRAM Is the Limit