Ollama is already a server. The moment it is running, it answers HTTP requests on 127.0.0.1:11434 — the desktop app, the command line and every tool that "connects to Ollama" are all just clients of that API. So the real questions are narrower: how to keep it running as a proper service, how to run it in Docker, and how to let other machines reach it without opening a model … [Read more...] about How to Run Ollama as a Server (and in Docker): Network Access, Security and Tuning
Tutorial
Open WebUI in 2026: Setup, Backends, RAG and the Licence Catch
Open WebUI is the chat interface most people put in front of a local model. It looks like ChatGPT, runs on your own machine, and talks to whatever engine you already have. The single most common misunderstanding about it is worth settling first: Open WebUI does not run the model, and it uses no VRAM of its own. It is a web app. Every gigabyte of memory in a local AI setup is … [Read more...] about Open WebUI in 2026: Setup, Backends, RAG and the Licence Catch
LoRA Training VRAM Requirements: Flux, SDXL and SD 3.5 (2026)
The short answer LoRA training memory is decided by three things, and network rank is not one of them. What matters is the precision you hold the frozen base model at, whether you cache latents and text embeddings before training starts, and the training resolution. ModelBF16FP8NF4Smallest card (FP8) SD 1.5 (512×512)2.9 GB——6GB SDXL (1024×1024)6.9 GB4.3 GB—8GB SD 3.5 Large … [Read more...] about LoRA Training VRAM Requirements: Flux, SDXL and SD 3.5 (2026)
How Much VRAM Do You Need to Fine-Tune an LLM? (2026 Requirements)
The short answer Fine-tuning needs far more memory than running the same model, and the gap is not small. Llama 3.1 8B runs comfortably on an 8GB card at Q4_K_M. Fine-tuning every parameter of it with 32-bit AdamW needs roughly 130GB. QLoRA brings that back down to about 5.9GB — which is why almost nobody does full fine-tuning on their own hardware. Here is what each method … [Read more...] about How Much VRAM Do You Need to Fine-Tune an LLM? (2026 Requirements)
Best Local Embedding Models for RAG in 2026: Sizes, Dimensions & Licences
If you are building a local RAG system in 2026, the embedding model matters more than the LLM you plug it into. A weak retriever hands your model the wrong chunks, and no amount of parameters downstream will fix that. The good news is that the best open embedding models are small — most run comfortably in under 1 GB, alongside your main model, on hardware you already … [Read more...] about Best Local Embedding Models for RAG in 2026: Sizes, Dimensions & Licences
How to Run Qwen3.8 27B Locally: Hardware, VRAM & Setup Guide (2026)
Qwen3.8 27B is Alibaba's newest open-weight model, released in August 2026. It's one of the strongest models you can realistically run on your own computer. The short version: you need about 18GB for the model at 4-bit, plus room for context, so a 24GB graphics card or a Mac with 32GB or more of unified memory. This guide covers exact memory needs by quantization and context … [Read more...] about How to Run Qwen3.8 27B Locally: Hardware, VRAM & Setup Guide (2026)
AMD Strix Halo vs Apple M5 Max: The 128GB Unified Memory Local AI Showdown
If you want to run big AI models locally without buying several expensive GPUs, two platforms offer up to 128GB of fast unified memory in a small box: AMD's Ryzen AI Max+ 395 ("Strix Halo") and Apple's Max-class chips, now the M5 Max in the 2026 Mac Studio. Both can hold 70B-class models and ~120B Mixture-of-Experts models that won't fit on a consumer GPU. This guide compares … [Read more...] about AMD Strix Halo vs Apple M5 Max: The 128GB Unified Memory Local AI Showdown
Best Local LLM Runners in 2026: Ollama vs LM Studio vs Jan vs llama.cpp vs vLLM
To run an AI model on your own computer you need a runner: the software that loads the model and serves it to you or your apps. The five most popular are Ollama, LM Studio, Jan, llama.cpp and vLLM. They overlap a lot (four of them are built on or around llama.cpp), but each suits a different kind of user. This guide compares them as of September 2026 and helps you … [Read more...] about Best Local LLM Runners in 2026: Ollama vs LM Studio vs Jan vs llama.cpp vs vLLM
Running Giant Open Models Locally: DeepSeek V4, Qwen3.8 & Llama 4 Hardware Sizing Guide
The biggest open-weight models are now Mixture-of-Experts (MoE) giants with hundreds of billions, or even trillions, of parameters. Can you run them at home? Sometimes, if you understand one key idea: MoE models need memory for all their parameters, but only compute a small "active" slice for each token. This guide explains how to size hardware for today's largest open models, … [Read more...] about Running Giant Open Models Locally: DeepSeek V4, Qwen3.8 & Llama 4 Hardware Sizing Guide
Quantization Guide for Local LLMs: Q4_K_M vs Q8_0 vs FP8 (DeepSeek R1 Examples)
When you download a model for Ollama or LM Studio, you'll see labels like Q4_K_M, Q8_0 or FP8. These are quantization levels: ways of compressing a model's weights so it takes less memory and runs faster, at some cost to accuracy. This guide uses the DeepSeek R1 distills as examples, but the same rules apply to Qwen, Gemma, gpt-oss and other models. It explains what each format … [Read more...] about Quantization Guide for Local LLMs: Q4_K_M vs Q8_0 vs FP8 (DeepSeek R1 Examples)