Ollama Basics

Privacy: Your prompts stay on your laptop. No OpenAI, no Anthropic, no third party. Your data is yours.

Cost: Free. Download once, run forever. No $20/month subscriptions, no pay-per-token API fees.

Speed: Local inference is fast. No network latency waiting for cloud API responses. Your laptop responds instantly.

Offline: Internet down? Your LLM still works. Perfect for training, experimentation, or just using AI when connectivity is spotty.

• ✓ Install Ollama on Linux, Mac, or Windows
• ✓ Understand which models fit your hardware (spoiler: lots of them)
• ✓ Run your first LLM and generate text
• ✓ Use Ollama's REST API for programmatic access
• ✓ Monitor performance and understand what's happening under the hood
• ✓ Manage multiple models simultaneously

This tutorial is deliberately beginner-focused. We're not covering:

• Model fine-tuning or training (that's Tutorial 2+)
• Deep learning or transformer internals
• Advanced optimization or quantization
• Production deployment or scaling

What we ARE covering: Getting Ollama running and understanding it works. That's the goal.

If you have these, you can run Ollama. The key variable is speed - which depends on your CPU cores and RAM bandwidth.

The model landscape has moved fast. Llama 2 and Neural Chat are gone from the recommended list - there are better, faster, smarter options that run just as well on consumer hardware. Here are the three best starting points:

When you browse ollama.com/library, you'll see tags like q4_0, q4_K_M, and q8_0 next to model names. These are quantization levels - how much the model has been compressed to fit in memory. Here's what they mean:

When you pull a model (after installing Ollama in the next section), you can specify a quantization tag or just use the default:

# :latest usually resolves to q4_K_M (recommended)
ollama pull qwen3:8b

# Explicitly request q8 for better quality (more RAM)
ollama pull qwen3:8b-q8_0

# 1B model - tiny and fast for simple tasks
ollama pull llama3.2:1b

For most users, :latest is the right call. Ollama picks a good default. Only specify a quantization tag if you're optimizing for a specific RAM budget or quality ceiling. You'll pull your first model in Section 04.

Some models need more headroom than others. If you're on 8GB total RAM, stick to 3B models:

If you're on 8GB and a 7B model feels sluggish, try llama3.2:3b instead - it's genuinely impressive for its size and will be noticeably more responsive.

Speed varies dramatically based on your CPU. Here's what to expect running a 7B model (q4_K_M):

Budget / Older CPU (4 cores, 8 threads):   3–6 tokens/sec (usable, slower)
Mid-Range CPU (8 cores, 16 threads):      8–15 tokens/sec (very good!)
High-End CPU (12+ cores, 24+ threads):    15–25+ tokens/sec (excellent)
Apple Silicon M2/M3:                      30–50+ tokens/sec (fast as GPU)

Note: Tested on systems with 16GB+ RAM and NVMe SSD storage

To put it in perspective: 10 tokens/second means a 500-token response (a long paragraph) takes about 50 seconds. That's fast for CPU-only inference. You'll be pleasantly surprised.

Models pile up fast - Qwen3:8b is 5.2GB, DeepSeek-R1:7b is 4.7GB, and once you start experimenting you'll want 6-10 models on hand. That's 30-50GB before you know it. Here's the gear that makes the experience better, from a genuine "we ran these tutorials on this hardware" perspective:

Amazon links above are affiliate links - they cost you nothing extra and help keep these tutorials free and updated. Full hardware buying guide with specific model recommendations: Mini PC Setup - Hardware Guide.

• ✓ Linux OS (Ubuntu 24.04+, Fedora, Debian, etc.)
• ✓ 8GB+ RAM (16GB+ recommended for multiple models)
• ✓ 20GB+ free disk space (models are ~4GB each)
• ✓ Stable internet connection (for downloading models once)
• ✓ Ability to run sudo commands (or use your user password)

Open a terminal and run:

curl -fsSL https://ollama.ai/install.sh | sh

This script:

• Downloads the Ollama binary for your system
• Places it in `/usr/local/bin/` (in your PATH)
• Sets up a systemd service to auto-start on boot
• Creates the ollama user and group

The installation takes 1–2 minutes. You'll see output as it progresses. When it finishes, you're done.

Check that Ollama is installed and in your PATH:

ollama --version

You should see something like:

ollama version is 0.1.45 (or newer)

During installation, the Ollama daemon started automatically in the background. It's listening on http://127.0.0.1:11434.

Do not run ollama serve manually - the port is already in use by the running daemon. You can verify it's active by testing the API:

curl -s http://127.0.0.1:11434/api/tags

If it responds with JSON (like {"models":[]}), the daemon is running. Good to go!

Now that the daemon is running, here's how you can use it:

The daemon is running and the API is responding. Installation is complete.

You're now ready to download and run your first model. Head to the next section to pull Llama 3.2.

Ollama is installed and running as a background service on http://127.0.0.1:11434. No configuration needed. No manual daemon management. Just pull a model and start using it.

You can now:

• Pull models with ollama pull <model>
• Run interactive chat with ollama run <model>
• Make API calls to http://127.0.0.1:11434 from scripts
• Start using local AI immediately - no API keys, no cloud, no costs

"Pulling" a model means downloading it from Ollama's registry and storing it locally. Run:

ollama pull llama3.2

You'll see output like:

pulling manifest
pulling 418956b73c34... (downloading layer 1)
pulling e1cd8f6a5d4a... (downloading layer 2)
verifying sha256 digest
writing manifest
success

The model is about 2GB. On a typical internet connection, this takes 1-2 minutes. Quick.

Once the download completes, start an interactive chat session:

ollama run llama3.2

You'll see the prompt appear with the model ready:

>>> 

(Run ollama list anytime to see all installed models and their exact versions.)

Now type a question or statement. Let's try something simple:

>>> What is Ollama?

Watch as the model generates a response in real-time. You'll see tokens appearing one by one. This is your local LLM doing inference right now, on your CPU, with no API calls, no cloud, no tracking.

Keep the chat session open and try different questions. Here are some good ones to test:

>>> Explain machine learning in simple terms
>>> Write a Python function that checks if a number is prime
>>> What's a good name for a Discord bot?
>>> Why is the sky blue?
>>> Tell me a joke

Notice:

• Response speed (you'll see "14 tokens/sec" or similar at the end)
• Response quality (is it coherent? Accurate?)
• CPU usage (all your cores working hard)
• No waiting for external APIs

To exit the interactive session, type:

>>> /bye

Or press Ctrl+D. Either works.

Let's be concrete about the workflow:

1. Pull: Download the model (~2GB) to ~/.ollama/models
2. Load: When you run Llama 3.2, Ollama loads it into RAM (~3GB used)
3. Inference: Your prompt goes to the model, which generates tokens one at a time
4. Display: Each token appears on your screen as it's generated
5. Repeat: You type, the model responds, until you exit

The interactive mode doesn't display detailed timing information directly. However, Ollama's REST API provides complete timing metrics including generation speed, which you'll explore in the next section.

For now, what you can observe from the interactive experience:

• The response appears token-by-token in real-time
• You can estimate speed by watching how fast tokens appear (roughly 10–15 tokens/sec on mid-range CPUs is typical)
• The model loads into memory the first time you run it (notice a slight delay before responses start)
• Subsequent responses should be faster since the model stays loaded

In the next section, you'll use the REST API to make requests and see exact timing metrics (tokens/sec, total duration, load time, etc.) in JSON responses. That's where you get precise performance data.

Ollama runs a simple HTTP server on localhost:11434. You send JSON requests, you get JSON responses. No authentication, no setup. Just HTTP.

This is how you'll integrate Ollama with OpenClaw later. For now, let's test it with curl.

Open a terminal and run:

curl http://localhost:11434/api/generate \
  -d '{
    "model": "llama3.2",
    "prompt": "Explain quantum computing in 2 sentences",
    "stream": false
  }'

The -d flag sends JSON data in the request body. The stream: false means "wait for the full response before returning."

You'll get a JSON response that looks like:

{
  "model": "llama3.2",
  "created_at": "2026-02-22T10:30:45.123456Z",
  "response": "Quantum computing exploits quantum mechanics (superposition and entanglement) to process data in fundamentally different ways than classical computers. A quantum computer can explore many solutions simultaneously, making certain problems exponentially faster to solve.",
  "done": true,
  "total_duration": 2500000000,
  "load_duration": 300000000,
  "prompt_eval_count": 12,
  "prompt_eval_duration": 800000000,
  "eval_count": 30,
  "eval_duration": 1400000000
}

Key fields:

• response: The generated text (what you want)
• done: Whether generation is complete
• eval_count: Number of tokens generated
• eval_duration: Time spent generating (nanoseconds)

The full JSON response includes several fields you may not need, including a context array (used for multi-turn conversations). If you have jq installed, you can extract just what you want:

curl -s http://localhost:11434/api/generate \
  -d '{
    "model": "llama3.2",
    "prompt": "Tell me a joke",
    "stream": false
  }' | jq -r '.response'

The -r flag means "raw output" (no quotes around the text). You'll just see:

Why don't scientists trust atoms? Because they make up everything!

Or extract key fields without the context clutter:

curl -s http://localhost:11434/api/generate \
  -d '{
    "model": "llama3.2",
    "prompt": "Tell me a joke",
    "stream": false
  }' | jq '{response, eval_count, eval_duration}'

Much cleaner. Install jq if you don't have it: sudo apt install jq

Note on the context field: The actual API response includes a context array containing token IDs from your prompt and response. This is used for multi-turn conversations (sending context back to maintain conversation history). For single requests, you can safely ignore it or filter it out with jq as shown above.

For longer responses, you might want tokens to stream in real-time (like in interactive chat). Set stream: true:

curl http://localhost:11434/api/generate \
  -d '{
    "model": "llama3.2",
    "prompt": "Write a haiku about programming",
    "stream": true
  }'

With streaming, you get multiple JSON objects (one per token), streamed line-by-line:

{"model":"llama3.2","created_at":"...","response":"Code","done":false}
{"model":"llama3.2","created_at":"...","response":" flows","done":false}
{"model":"llama3.2","created_at":"...","response":" like","done":false}
...
{"model":"llama3.2","created_at":"...","response":"","done":true}

Parse each line and print the response field to see tokens appear in real-time. This is how chat interfaces work.

You can pass additional parameters to control generation behavior:

curl http://localhost:11434/api/generate \
  -d '{
    "model": "llama3.2",
    "prompt": "Complete this: The future of AI is...",
    "stream": false,
    "temperature": 0.7,
    "top_p": 0.9,
    "num_predict": 100
  }'

Parameters explained (you'll dive deeper in Tutorial 2):

• temperature: Randomness (0.0=deterministic, 1.0=balanced, >1.5=creative)
• top_p: Diversity control (lower=more focused)
• num_predict: Max tokens to generate (prevents runaway responses)

For now, the defaults (no parameters) are fine. We'll explore these in Advanced.

When you integrate Ollama with OpenClaw, this is what happens under the hood:

1. OpenClaw receives a user message from Discord
2. It constructs a JSON request to your local Ollama API
3. Ollama generates a response
4. OpenClaw parses the response and sends it back to Discord

All locally. All on your laptop. All in milliseconds. That's powerful.

While Ollama is generating text, open another terminal and watch resource usage:

watch -n 1 'free -h'

This updates every second showing your RAM usage. A typical 7B model uses about 5–6GB.

              total        used        free      shared  buff/cache   available
Mem:           31Gi       6.2Gi        22Gi       1.3Gi      3.4Gi        23Gi

This example shows Ollama using ~6GB out of 31GB total-plenty of breathing room. Your system will vary.

For CPU usage, in another terminal:

top -n 1 -o %CPU

Or use htop for a nicer interface:

htop

During inference, you'll see all your CPU cores at high utilization (70–95%). This is normal and expected. Your CPU is working hard, which is why you get good performance.

Here's what to expect for a 7B-8B class model (like Qwen3 8B, which you'll pull in the next section) on different hardware. All numbers assume 16GB+ RAM and SSD storage. Your current Llama 3.2 3B runs roughly 2x faster and uses about half the RAM - these numbers represent the all-rounder tier:

Budget / Older CPU (4 cores):
  Tokens/Sec:         3–6 tokens/sec (slow but usable)
  Time to First Token: 1–2 seconds
  RAM Used:           ~6GB

Mid-Range CPU (8 cores):
  Tokens/Sec:         8–15 tokens/sec (very good)
  Time to First Token: 500ms–1 second
  RAM Used:           ~6GB

High-End CPU (12+ cores):
  Tokens/Sec:         15–25+ tokens/sec (excellent)
  Time to First Token: 300–500ms
  RAM Used:           ~6GB

Universal:
  Model Size on Disk: ~5.2GB
  Model Load Time:    1–3 seconds (SSD-dependent)
  CPU Utilization:    70–95% during generation

Where you land on this spectrum depends on your CPU core count and clock speed. The good news: even budget CPUs generate text at usable speeds.

Token generation speed varies based on several factors:

Let's put it in perspective:

Short Answer (50 tokens):       ~3–5 seconds
Medium Answer (200 tokens):     ~13–20 seconds
Long Answer (500 tokens):       ~33–50 seconds
Full Essay (1000 tokens):       ~65–100 seconds

For comparison:

• OpenAI's API: Also 10–20 tokens/sec, but costs money and requires internet
• Claude/GPT directly: Similar speed with 100x the cost
• Instant messaging apps: Slower (typing speed is 40–60 words/min = ~10 tokens/sec)

So yes, 10–15 tokens/sec is genuinely fast. You're getting good performance locally, for free.

Most hardware can run Ollama continuously without issues:

• Thermals: CPU inference doesn't generate excessive heat. Modern cooling handles it fine.
• Battery Drain: On laptops with battery, expect 1–2 hours per charge during heavy continuous use.
• Reliability: Modern CPUs are designed for sustained loads. No degradation over time.
• Memory Leaks: Ollama is stable. No memory creep after hours of running.

For occasional or development use on a laptop/desktop, your current hardware is fine. If you want to run Ollama 24/7 at scale with multiple concurrent requests, consider a dedicated server or device with more consistent power delivery.

Let's pull two more models to build out your library. The first is Qwen3:8b - the current all-rounder pick with exceptional reasoning, code ability, and 100+ language support:

ollama pull qwen3:8b

And the tiny-but-capable Llama 3.2 1B - useful when you want an instant response and don't need heavy reasoning:

ollama pull llama3.2:1b

The 1B model is only ~1.3GB and loads in seconds. It's your speed tier - great for quick lookups and simple tasks when you don't want to wait for a 7B model to warm up.

See everything you've downloaded:

ollama list

Output:

NAME                 ID              SIZE    MODIFIED
llama3.2:latest      a4f39e04031c    2.0 GB  3 hours ago
qwen3:8b             9a3f5f67c0c9    5.2 GB  2 hours ago
llama3.2:1b          baf6a787fdff    1.3 GB  1 hour ago

Three models, ~8.5GB total. They cover different use cases - speed (1B), everyday tasks (3B), and serious reasoning (8B).

ollama list shows what's downloaded. ollama ps shows what's currently loaded in memory - this is the command you want when troubleshooting performance or wondering why your RAM is full:

ollama ps

NAME             ID              SIZE      PROCESSOR    UNTIL
qwen3:8b         9a3f5f67c0c9    5.2 GB    100% CPU     4 minutes from now
llama3.2:latest  a4f39e04031c    2.0 GB    100% CPU     Expires in 2 minutes

By default, Ollama unloads a model from RAM 5 minutes after it was last used. This is good for shared machines and 8GB systems. But if you're the only user and have the RAM, keeping models loaded means instant responses with no cold-start delay.

# Keep models loaded for 30 minutes of idle time
export OLLAMA_KEEP_ALIVE=30m

# Keep models loaded indefinitely (until you restart Ollama)
export OLLAMA_KEEP_ALIVE=-1

# Use the default 5-minute unload (default behavior)
export OLLAMA_KEEP_ALIVE=5m

# Apply it permanently (add to ~/.bashrc or ~/.zshrc)
echo 'export OLLAMA_KEEP_ALIVE=30m' >> ~/.bashrc
source ~/.bashrc

To apply OLLAMA_KEEP_ALIVE to the Ollama systemd service (so it persists across reboots), add it to the service's environment:

sudo systemctl edit ollama

[Service]
Environment="OLLAMA_KEEP_ALIVE=30m"

sudo systemctl daemon-reload
sudo systemctl restart ollama

Switch is simple - just run a different model name:

ollama run qwen3:8b

You're now in Qwen3's interactive session. When done:

>>> /bye

Then switch to the fast tier:

ollama run llama3.2:1b

With the REST API, you can specify which model to use per request. This is how OpenClaw routes different task types to different models:

curl http://localhost:11434/api/generate \
  -d '{
    "model": "qwen3:8b",
    "prompt": "Analyze this error and suggest a fix...",
    "stream": false
  }'

curl http://localhost:11434/api/generate \
  -d '{
    "model": "llama3.2:1b",
    "prompt": "What does HTTP 429 mean?",
    "stream": false
  }'

Ollama queues requests and processes them in order. On 16GB+ RAM with keep-alive set, both models can stay loaded and the switch between them is instantaneous - no unload/reload delay.

All of these are 4-bit quantized by default and pull directly from Ollama's library:

ollama pull llama3.2:1b       # 1.3GB - instant speed, simple tasks
ollama pull llama3.2:3b       # 2GB   - small and surprisingly capable
ollama pull qwen3:4b          # 2.5GB - tiny but rivals much larger models
ollama pull deepseek-r1:7b    # 4.7GB - reasoning model, shows its work
ollama pull qwen3:8b          # 5.2GB - best all-rounder, 100+ languages
ollama pull qwen3:14b         # 9.3GB - step up in quality, needs 16GB RAM
ollama pull deepseek-r1:14b   # 9.0GB - serious reasoning, needs 16GB RAM

Start with llama3.2:3b and qwen3:8b. That covers 90% of use cases. Add qwen3:14b or deepseek-r1:14b if you have 16GB+ and want stronger reasoning.

Models live in ~/.ollama/models. Check your usage:

du -sh ~/.ollama/models

To remove a model and free disk space:

ollama rm llama3.2:1b

Re-pull anytime - Ollama uses layer caching, so re-downloading a model you've had before is faster than the initial pull. Models that share architecture (like Llama 3.2 1B and 3B) share base layers, so the second model is much smaller to download.

Ollama loads models on-demand into RAM. Here's what you need to know:

• One 7B model (q4_K_M): ~5–6GB RAM (plus OS overhead, so ~8GB total consumed)
• One 3B model: ~2.5GB RAM - very comfortable on 8GB systems
• Two 7B models loaded: ~12GB - comfortable on 16GB systems
• One 14B model (phi4): ~9GB - needs 16GB, leaves headroom

Ollama gracefully unloads models you're not using when memory gets tight. With OLLAMA_KEEP_ALIVE set, models stay warm as long as you have the RAM to support them.

Book III has a main road and side roads. The main road is:

• Ollama Basics (you just finished this) → install, run models, learn the API
• Best Local AI Models 2026 → which model to actually pull for your hardware and use case
• Ollama Advanced → tune parameters, benchmark, and integrate with OpenClaw

Once your OpenClaw bot is running on local models, the rest of Book III is optional extras you can add whenever you want.

These tutorials are standalone and don't depend on each other. Pick whichever sounds useful:

• Open WebUI - a ChatGPT-style web interface for your local models. Great for non-terminal users or sharing with family.
• Document Q&A with RAG - upload PDFs, notes, and manuals, then ask questions about them. Uses AnythingLLM.
• Local Coding Assistant - Copilot-style autocomplete and AI chat in VS Code, powered by Ollama. Your code never leaves your machine.

Q: Can I run this in the background permanently?
A: Yes! Ollama already auto-starts as a systemd service. It'll run on boot and keep running.

Q: Can I access Ollama from other machines (not localhost)?
A: Not by default (security). You'd need to expose the API with an nginx proxy or SSH tunnel (advanced).

Q: What if I want to upgrade to a bigger model later?
A: You can, but 13B+ models are significantly slower on CPU. Consider upgrading hardware to GPU.

Q: Is there a web UI for Ollama?
A: Not built-in, but there are excellent options. Our Open WebUI tutorial walks you through setting up a polished, ChatGPT-style interface that connects directly to your local Ollama instance.

Q: Can I use Ollama on Windows/Mac?
A: Yes! Ollama has installers for all platforms. Same concepts apply.

You've completed Ollama Basics. You understand:

• ✓ What Ollama is and why it matters
• ✓ How to install and verify it works
• ✓ How to download and run models
• ✓ How to use the REST API programmatically
• ✓ What performance to expect from modern CPUs
• ✓ How to manage multiple models

That's a solid foundation. From here, you can:

• → Ollama Advanced - optimization, parameter tuning, and OpenClaw integration
• → Open WebUI - give your local models a real chat interface
• → Local Coding Assistant - use Ollama to power VS Code autocomplete and chat
• → Document Q&A with RAG - search and ask questions about your own documents

Whatever you choose, you're now part of the local AI revolution. Your data is yours. Your LLM is yours. No cloud, no subscriptions, just you and your hardware. That's powerful. Enjoy!