Ollama is already a server. The moment it is running, it answers HTTP requests on 127.0.0.1:11434 — the desktop app, the command line and every tool that "connects to Ollama" are all just clients of that API. So the real questions are narrower: how to keep it running as a proper service, how to run it in Docker, and how to let other machines reach it without opening a model … [Read more...] about How to Run Ollama as a Server (and in Docker): Network Access, Security and Tuning
Local AI & Tech Guides
Practical guides to running AI on your own hardware. We cover local LLMs end to end — how much RAM and VRAM you actually need, which runner to use, how quantization changes the maths, and which GPUs, Macs and handhelds are worth buying. Every speed figure here is a clearly labelled estimate, never a lab claim.
Pair these guides with our free AI tools: the VRAM calculator, the hardware matrix and the model directory.
Open WebUI in 2026: Setup, Backends, RAG and the Licence Catch
Open WebUI is the chat interface most people put in front of a local model. It looks like ChatGPT, runs on your own machine, and talks to whatever engine you already have. The single most common misunderstanding about it is worth settling first: Open WebUI does not run the model, and it uses no VRAM of its own. It is a web app. Every gigabyte of memory in a local AI setup is … [Read more...] about Open WebUI in 2026: Setup, Backends, RAG and the Licence Catch
LoRA Training VRAM Requirements: Flux, SDXL and SD 3.5 (2026)
The short answer LoRA training memory is decided by three things, and network rank is not one of them. What matters is the precision you hold the frozen base model at, whether you cache latents and text embeddings before training starts, and the training resolution. ModelBF16FP8NF4Smallest card (FP8) SD 1.5 (512×512)2.9 GB——6GB SDXL (1024×1024)6.9 GB4.3 GB—8GB SD 3.5 Large … [Read more...] about LoRA Training VRAM Requirements: Flux, SDXL and SD 3.5 (2026)
Best GPU for AI in 2026: VRAM Tiers, Bandwidth and What Actually Fits
The short answer For local AI the ranking that matters is not the one on gaming benchmark charts. VRAM capacity decides which models you can run at all. Memory bandwidth decides how fast they run. Those are two different specifications, and the second one is worthless if the first one fails. If you want a single recommendation: a used RTX 3090 with 24GB remains the best value … [Read more...] about Best GPU for AI in 2026: VRAM Tiers, Bandwidth and What Actually Fits
How Much VRAM Do You Need to Fine-Tune an LLM? (2026 Requirements)
The short answer Fine-tuning needs far more memory than running the same model, and the gap is not small. Llama 3.1 8B runs comfortably on an 8GB card at Q4_K_M. Fine-tuning every parameter of it with 32-bit AdamW needs roughly 130GB. QLoRA brings that back down to about 5.9GB — which is why almost nobody does full fine-tuning on their own hardware. Here is what each method … [Read more...] about How Much VRAM Do You Need to Fine-Tune an LLM? (2026 Requirements)
Best Local Embedding Models for RAG in 2026: Sizes, Dimensions & Licences
If you are building a local RAG system in 2026, the embedding model matters more than the LLM you plug it into. A weak retriever hands your model the wrong chunks, and no amount of parameters downstream will fix that. The good news is that the best open embedding models are small — most run comfortably in under 1 GB, alongside your main model, on hardware you already … [Read more...] about Best Local Embedding Models for RAG in 2026: Sizes, Dimensions & Licences
How to Run Qwen3.8 27B Locally: Hardware, VRAM & Setup Guide (2026)
Qwen3.8 27B is Alibaba's newest open-weight model, released in August 2026. It's one of the strongest models you can realistically run on your own computer. The short version: you need about 18GB for the model at 4-bit, plus room for context, so a 24GB graphics card or a Mac with 32GB or more of unified memory. This guide covers exact memory needs by quantization and context … [Read more...] about How to Run Qwen3.8 27B Locally: Hardware, VRAM & Setup Guide (2026)
AMD Strix Halo vs Apple M5 Max: The 128GB Unified Memory Local AI Showdown
If you want to run big AI models locally without buying several expensive GPUs, two platforms offer up to 128GB of fast unified memory in a small box: AMD's Ryzen AI Max+ 395 ("Strix Halo") and Apple's Max-class chips, now the M5 Max in the 2026 Mac Studio. Both can hold 70B-class models and ~120B Mixture-of-Experts models that won't fit on a consumer GPU. This guide compares … [Read more...] about AMD Strix Halo vs Apple M5 Max: The 128GB Unified Memory Local AI Showdown
Best Local LLM Runners in 2026: Ollama vs LM Studio vs Jan vs llama.cpp vs vLLM
To run an AI model on your own computer you need a runner: the software that loads the model and serves it to you or your apps. The five most popular are Ollama, LM Studio, Jan, llama.cpp and vLLM. They overlap a lot (four of them are built on or around llama.cpp), but each suits a different kind of user. This guide compares them as of September 2026 and helps you … [Read more...] about Best Local LLM Runners in 2026: Ollama vs LM Studio vs Jan vs llama.cpp vs vLLM
Running Giant Open Models Locally: DeepSeek V4, Qwen3.8 & Llama 4 Hardware Sizing Guide
The biggest open-weight models are now Mixture-of-Experts (MoE) giants with hundreds of billions, or even trillions, of parameters. Can you run them at home? Sometimes, if you understand one key idea: MoE models need memory for all their parameters, but only compute a small "active" slice for each token. This guide explains how to size hardware for today's largest open models, … [Read more...] about Running Giant Open Models Locally: DeepSeek V4, Qwen3.8 & Llama 4 Hardware Sizing Guide