Tutorials
Install Ollama in 10 Minutes: Run Any LLM Locally Free
A no-fluff walkthrough for installing Ollama on Mac, Windows, or Linux and running open models like Llama 3.3 and DeepSeek V3 completely offline.
September 2, 2026
Running a capable large language model on your own laptop used to sound like a weekend project for GPU nerds. Not anymore. Ollama has turned the whole thing into a two-command setup, and as of early 2026 it's the easiest way to get a real LLM running on your machine without paying a cent to OpenAI, Anthropic, or anyone else.
This tutorial walks through how to install Ollama on Mac, Windows, and Linux, pull your first model, chat with it, hook it up to an API, and pick the right model for your hardware. No cloud account. No credit card. No telemetry going anywhere unless you turn it on yourself.
What you'll learn
Worth flagging: by the end of this guide, you'll have:
- Ollama installed and running as a background service
- At least one open-weights model downloaded locally
- A working chat session in your terminal
- A local API endpoint at
http://localhost:11434you can hit from any app - A shortlist of models matched to your RAM and GPU
The whole thing takes roughly 10 minutes if your internet isn't terrible. Model downloads take longer (some are 40GB+), but the actual setup is fast.
Prerequisites: what you actually need
Ollama is surprisingly forgiving, but the model you pick matters more than the tool itself. Rough guidance based on the official Ollama hardware notes:
| Your RAM | Realistic model size | Example models |
|---|---|---|
| 8GB | 3B parameters (Q4) | Llama 3.2 3B, Phi-4 Mini |
| 16GB | 7-8B parameters (Q4) | Llama 3.1 8B, Mistral 7B, Qwen 2.5 7B |
| 32GB | 13-14B parameters | Qwen 2.5 14B, Phi-4 14B |
| 64GB+ | 30-70B parameters | Llama 3.3 70B (Q4), Qwen 2.5 32B |
| 128GB+ | 100B+ or MoE models | DeepSeek V3 (partial offload) |
A GPU helps a lot but isn't required. On Apple Silicon (M1 and newer), Ollama uses Metal automatically and the unified memory architecture is genuinely great for this. On Windows and Linux, an NVIDIA card with 8GB+ VRAM will run circles around CPU inference. AMD ROCm support has been available in official Ollama builds for a while now and works well on supported cards.
And yes, you can absolutely run these on CPU only. It's just slower. A Llama 3.1 8B model on a modern laptop CPU pushes roughly 5-10 tokens per second, which feels usable for chat but painful for anything longer.
Step 1: Install Ollama
Go to ollama.com/download and grab the installer for your OS. It's the same URL for everyone.
macOS: Download the .dmg, drag Ollama to Applications, launch it once. That's it. A tiny llama icon appears in your menu bar and the background service starts.
Windows: Download Ollama Setup.exe and run it. Windows 10 (22H2) or Windows 11 required. The installer sets up Ollama as a service that starts with Windows.
Linux: One command:
Bash
curl -fsSL https://ollama.com/install.sh | sh
Yes, you should read scripts before piping them to shell. The install script is open source on GitHub if you want to inspect it first (recommended).
On Linux, the installer registers a systemd service. Verify it's running:
Bash
systemctl status ollama
On any platform, confirm the CLI works:
Bash
ollama --version
You should see something like ollama version is 0.11.x or higher. If the command isn't found on macOS, restart your terminal so the PATH refreshes.
Step 2: Pull your first model
The ollama pull command downloads a model. The ollama run command downloads it (if needed) and drops you into a chat session. Most people just use run.
For a first test, grab something small so you're not waiting 20 minutes to see if things work:
Bash
ollama run llama3.2:3b
This pulls Meta's Llama 3.2 3B model, which is about 2GB and runs on basically any modern machine. Once the download finishes, you'll see a >>> prompt. Type a question, hit enter, watch the tokens stream in.
Exit the chat with /bye or Ctrl+D.
A few other solid starter models:
Bash
ollama run llama3.1:8b # Meta's 8B, great general-purpose model (4.9GB)
ollama run qwen2.5:7b # Alibaba's Qwen, strong at coding and reasoning (4.7GB)
ollama run mistral:7b # Mistral 7B v0.3, fast and multilingual (4.4GB)
ollama run phi4 # Microsoft's Phi-4 14B, punches above its weight (9.1GB)
Browse the full catalog at ollama.com/library. Each model page lists the exact size, quantization options, and any special tags. If you want to build a real app on top of one of these, our Llama API tutorial walks through it end-to-end.
Understanding quantization tags
When you see something like llama3.1:8b-instruct-q4_K_M, that's a quantization tag. Ollama's default (usually q4_K_M) is a 4-bit quantization that cuts memory use by roughly 75% versus full precision, with fairly minor quality loss. If you have plenty of RAM and want the best output, try q8_0 or fp16 variants. If you're squeezed for memory, q3_K_M shaves off more.
Rule of thumb: 4-bit quantized models need roughly params × 0.6GB of RAM. So an 8B model wants about 5GB just for weights, plus context window overhead.
Step 3: Chat with your model
Inside a running session, a few slash commands are useful:
/set parameter temperature 0.3— lower creativity, more deterministic answers/set system "You're a Rust expert. Give concise answers."— set a system prompt/show info— see the current model's metadata/save mysessionand/load mysession— checkpoint conversations/clear— reset context without leaving
Context windows depend on the model. Llama 3.1 supports 128K tokens; Qwen 2.5 supports 128K; Phi-4 is 16K. Ollama defaults to a 4096-token context to save memory. Bump it with:
Bash
OLLAMA_CONTEXT_LENGTH=32768 ollama run llama3.1:8b
Or set it persistently in the model's Modelfile (more on that below).
Step 4: Use the local API
This is the part that matters. Ollama exposes an HTTP API on localhost:11434 the moment it's running. You can hit it from Python, Node, curl, whatever.
Quick curl test:
Bash
curl http://localhost:11434/api/generate -d '{
"model": "llama3.1:8b",
"prompt": "Explain HTTP/2 in one paragraph.",
"stream": false
}'
Python with the official ollama package:
Python
from ollama import chat
response = chat(model='llama3.1:8b', messages=[
{'role': 'user', 'content': 'Write a Python function to reverse a linked list.'}
])
print(response['message']['content'])
And if your existing code targets the OpenAI SDK, Ollama has an OpenAI-compatible endpoint at /v1. So this works out of the box:
Python
from openai import OpenAI
client = OpenAI(base_url='http://localhost:11434/v1', api_key='ollama')
resp = client.chat.completions.create(
model='llama3.1:8b',
messages=[{'role': 'user', 'content': 'Hello!'}]
)
That compatibility layer is genuinely great. Any tool built for OpenAI (LangChain, LlamaIndex, Continue.dev, most chat frontends) will happily point at your local Ollama instance with zero code changes beyond the base URL.
Step 5: Give it a GUI (optional but nice)
The terminal is fine. A proper chat UI is nicer. Two solid options:
Open WebUI is the most popular front-end. It looks and feels like ChatGPT, supports multiple models, RAG over uploaded documents, and image input for multimodal models. Install with Docker:
Bash
docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway \
-v open-webui:/app/backend/data --name open-webui --restart always \
ghcr.io/open-webui/open-webui:main
Visit http://localhost:3000, create a local account, and it'll auto-detect your Ollama models.
Enchanted is a native macOS app if you want something lighter and prettier. Free on the App Store.
For coding, install the Continue.dev extension in VS Code and point it at Ollama. You get inline completions and chat backed by your local model, no GitHub Copilot subscription needed.
Common pitfalls
Models eating all your disk. Downloaded models live in ~/.ollama/models on Mac/Linux and C:\Users\<you>\.ollama\models on Windows. Run ollama list to see what you have and ollama rm <model> to delete. It adds up fast, some models are 40GB+.
GPU not being used. On Windows/Linux, check ollama ps while a model is loaded. If it says 100% CPU, your NVIDIA drivers might be out of date or your VRAM is too small for the model. Try a smaller quant.
Slow first response. The first prompt after loading a model is always slower because the model is being loaded into memory. Subsequent prompts are fast. Ollama unloads models after 5 minutes of inactivity by default, tune with OLLAMA_KEEP_ALIVE=30m.
Random gibberish output. Almost always a context window issue. Longer conversations overflow the default 4096-token window and the model starts hallucinating. Bump OLLAMA_CONTEXT_LENGTH.
Port 11434 already in use. Set OLLAMA_HOST=0.0.0.0:11500 before starting the service. Same variable also exposes Ollama on your network, which you probably don't want unless you specifically do.
Testing and verification
Quick sanity check that everything is wired up:
Bash
ollama list # shows installed models
ollama ps # shows loaded/running models
curl http://localhost:11434/api/tags # API endpoint responding
If all three return sensible output, you're done.
For a more meaningful test, ask your model something that requires reasoning: "There are 3 killers in a room. Someone enters and kills one of them. How many killers are in the room?" A good 7B+ model gets this right (three, because the new person is also a killer). A weak one confidently says two.
Which model should you actually run?
Honest opinion, based on what the open-source community has been running through 2026:
- Best 3B for weak hardware: Llama 3.2 3B. Not brilliant, but usable for summarization and simple Q&A.
- Best 7-8B all-rounder: Qwen 2.5 7B. Beats Llama 3.1 8B on most coding and math benchmarks per Hugging Face's Open LLM Leaderboard.
- Best 14B model: Phi-4. Microsoft's 14B model that punches way above its weight, particularly on reasoning tasks.
- Best local coding model: Qwen 2.5 Coder 32B (if you have 24GB+ VRAM). Legitimately competitive with cloud models on HumanEval.
- Frontier local model: Llama 3.3 70B at 4-bit needs about 40GB of memory but is the closest thing to a hosted API you'll get locally. If you want even more options in this range, see our roundup of the best uncensored GGUF models to run locally.
DeepSeek V3 keeps getting mentioned as a local option. Be careful: the full model is 671B parameters. Even quantized, you need serious hardware (multi-GPU rigs or a top-spec Mac Studio with 512GB unified memory). Fun to try, not practical for most people.
Next steps
Once the basics are working, a few directions worth exploring:
- Custom Modelfiles. Create your own model variants with baked-in system prompts, temperatures, and context sizes. Docs at github.com/ollama/ollama.
- Local RAG. Combine Ollama with a vector database (Chroma, Qdrant) to query your own documents. Open WebUI has this built in if you don't want to code it.
- Agentic workflows. Point Aider, Cline, or Continue at your local model for hands-off coding tasks. Works best with Qwen 2.5 Coder or Llama 3.3.
- Fine-tuning. Ollama itself doesn't fine-tune, but you can train with Unsloth or Llama Factory and import the resulting GGUF file with a Modelfile. See our guide to fine-tuning an LLM on your own data for a full walkthrough.
Running models locally isn't going to replace Claude or GPT-5 for hard problems anytime soon. But for privacy-sensitive work, offline coding, or just avoiding a monthly API bill, it's a genuinely great option in 2026. And the fact that it now takes 10 minutes to set up is kind of remarkable.
