Ollama Advanced

You have:

• ✓ Ollama installed and running as a service
• ✓ At least one model downloaded (Llama 3.2 or Qwen3)
• ✓ REST API working (tested with curl)
• ✓ Baseline performance metrics (10-15 tokens/sec on the Ryzen)

If you don't have these, go back to Ollama Basics first. Not sure which model to use? The Best Local AI Models 2026 comparison covers every model worth knowing with pull commands ready to go.

In this 90-minute tutorial:

• OpenClaw Integration: Wire your bot to local Ollama - the whole reason you're here
• Parameters: Temperature, top_p, context windows - the knobs that control your model's behavior
• Use-Case Tuning: Different settings for different tasks (structured output, creative writing, code)
• Benchmarking: Objective methods to measure quality and speed
• GPU Detection: Understand when GPUs help (spoiler: not for you)
• Multiple Models: Run models concurrently, queue requests, instant switching
• Optimization: Memory tuning, CPU optimization, inference speed tuning
• Monitoring: Health checks, logs, continuous operation
• Troubleshooting: Common issues and how to fix them

This tutorial works on any modern hardware capable of running Ollama. Here's what to keep in mind:

Performance numbers in this tutorial are based on the Ryzen 7 6800H mini PC with 32GB RAM - the same hardware recommended throughout these tutorials. If you're on different hardware, your numbers will vary but the tuning principles are the same.

For OpenClaw: A well-tuned local LLM can power a bot that feels just as smart as cloud-based solutions, but faster, cheaper, and completely private.

For Learning: Understanding parameters teaches you how language models actually work. You'll develop intuition for LLM behavior.

For Optimization: Squeezing another 20% performance from your hardware means faster responses and better user experience.

This tutorial assumes you're comfortable with terminal commands and basic system administration. You don't need to be an expert, but "comfortable with Linux" is the baseline.

If something is unclear, go back to Ollama Basics or skip to the sections that interest you. You don't have to do everything in order.

Verify Ollama is ready before touching OpenClaw config:

curl http://localhost:11434/api/tags
# Should return JSON with your pulled models listed
# If you get "connection refused" - start Ollama first: ollama serve

OpenClaw has a built-in onboarding command that handles Ollama configuration automatically. This is the quickest way to get wired up:

# Interactive mode (recommended for first time)
openclaw onboard

# Non-interactive with your chosen model
openclaw onboard --non-interactive \
  --auth-choice ollama \
  --custom-model-id "qwen3:8b" \
  --accept-risk

The --auth-choice ollama flag tells OpenClaw to use local Ollama instead of a cloud provider. It auto-discovers models from http://127.0.0.1:11434 and sets all costs to $0.

After onboarding, open the Control UI at http://localhost:18789 to verify the agent is running and connected.

If you want to specify exact models, a custom host, or tune parameters, edit the config file directly. OpenClaw stores its settings at:

~/.openclaw/openclaw.json

Here is a working Ollama config block. Add or merge this into your existing openclaw.json:

{
  "models": {
    "providers": {
      "ollama": {
        "baseUrl": "http://127.0.0.1:11434",
        "apiKey": "ollama-local",
        "api": "ollama",
        "models": [
          {
            "id": "qwen3:8b",
            "name": "Qwen3 8B",
            "reasoning": false,
            "input": ["text"],
            "cost": { "input": 0, "output": 0, "cacheRead": 0, "cacheWrite": 0 },
            "contextWindow": 4096,  // Qwen3 native default is 40,960 - override for speed
            "maxTokens": 4096
          },
          {
            "id": "llama3.2",
            "name": "Llama 3.2 3B",
            "reasoning": false,
            "input": ["text"],
            "cost": { "input": 0, "output": 0, "cacheRead": 0, "cacheWrite": 0 },
            "contextWindow": 4096,
            "maxTokens": 2048
          }
        ]
      }
    }
  },
  "agents": {
    "defaults": {
      "model": {
        "primary": "ollama/qwen3:8b",
        "fallbacks": ["ollama/llama3.2"]
      }
    }
  }
}

If you just want OpenClaw to pick up whatever models you have installed without listing them manually, set one environment variable and restart OpenClaw:

export OLLAMA_API_KEY="ollama-local"

# Add to your shell profile so it persists
echo 'export OLLAMA_API_KEY="ollama-local"' >> ~/.bashrc
source ~/.bashrc

With this set, OpenClaw automatically queries /api/tags to find all your models, reads their context windows, and marks anything with "r1", "reasoning", or "think" in the name as a reasoning-capable model. All costs are set to $0.

Once configured, you can verify OpenClaw sees your Ollama models and swap between them:

# See what models OpenClaw has available
openclaw models list

# Set qwen3:8b as primary
openclaw models set ollama/qwen3:8b

# Switch to the fast 3B model for lighter tasks
openclaw models set ollama/llama3.2

The ollama/ prefix tells OpenClaw which provider the model lives on. You'll use this prefix any time you reference a model in config or CLI commands.

Before you start routing Discord messages through it, confirm the full chain is working:

# 1. Confirm Ollama is responding
curl http://localhost:11434/api/tags

# 2. Confirm OpenClaw sees the models
openclaw models list

# 3. Check provider connectivity
openclaw models status

# 4. Open the Control UI and send a test prompt
# http://localhost:18789

If openclaw models list shows your Ollama models and the Control UI returns a response to a test prompt, you are fully wired up.

What to expect on a Ryzen 7 6800H mini PC (CPU-only inference):

Short factual question (e.g. "What is HTTP 429?")
  → llama3.2 (3B):   3-5 seconds
  → qwen3:8b:      6-10 seconds

Medium response (paragraph explanation)
  → llama3.2 (3B):   8-12 seconds
  → qwen3:8b:      12-20 seconds

Multiple users queued simultaneously
  → Ollama processes one at a time
  → Each user waits for the one ahead of them
  → Fine for personal bots and small communities

For a personal Discord server this is completely usable. Users perceive the "typing..." indicator as the bot thinking, which feels natural. If you need faster responses, use llama3.2 as the primary and save qwen3:8b for when quality matters more than speed.

Range: 0.0 to 2.0 (or higher)
Default: Usually 0.7 (balanced)

Temperature controls how "creative" or "random" the model gets:

Try it:

# Cold (deterministic)
curl http://localhost:11434/api/generate -d '{
  "model": "qwen3:8b",
  "prompt": "Complete: The future of AI is",
  "temperature": 0.0,
  "stream": false
}' | jq -r '.response'

# Hot (creative)
curl http://localhost:11434/api/generate -d '{
  "model": "qwen3:8b",
  "prompt": "Complete: The future of AI is",
  "temperature": 1.5,
  "stream": false
}' | jq -r '.response'

Run both and notice the difference. You'll see the same semantic answer at 0.0, wildly creative answers at 1.5.

Range: 0.0 to 1.0
Default: 0.9

Top-P is a more sophisticated diversity control than temperature. It works by:

1. Model predicts probabilities for the next token
2. Sort tokens by likelihood (highest first)
3. Keep only the top tokens that sum to p% probability
4. Randomly sample from that restricted set

In practice:

• 0.1: Ultra-focused (only the top 10% of likely tokens)
• 0.5: Focused (top 50% of likely tokens)
• 0.9: Balanced (top 90% of likely tokens, allows more creativity)

Works best with temperature: Top-P controls diversity among the remaining candidates after temperature does its work. Usually keep at 0.9 unless you're experimenting.

Range: 1 to infinity
Default: 40 (varies by Ollama version)

Top-K is the simpler cousin of top-P. It limits the model to only consider the K most likely tokens:

• K=1: Only consider the most likely token (extremely constrained)
• K=10: Consider top 10 most likely tokens
• K=40: Consider top 40 (good balance)
• K=100+: Very permissive (almost all tokens allowed)

Practical advice: Leave top-K at default and use temperature+top-P instead. Top-K is older and less intuitive than top-P.

Range: 0.0 to 2.0
Default: 1.1

Repeat penalty prevents the model from repeating the same phrase over and over (common LLM failure mode):

• 1.0: No penalty (tokens can repeat freely)
• 1.1: Mild penalty (slight discouragement to repeat)
• 1.5: Strong penalty (heavily discourage repetition)

Use when: Model generates repetitive text ("the the the..." or "and and and...").
Default is good: Usually leave at 1.1.

The context window is how many tokens the model can "see" at once - your prompt, conversation history, and the response being generated. Bigger window means more context retained, but slower responses and higher RAM usage.

Check what your model is actually using:

ollama show qwen3:8b
# Look for num_ctx under the Parameters section

Speed tradeoffs on a typical 8-core CPU with a 7B model:

num_ctx  1024:   ~20 t/s   - fast, fine for short chats
num_ctx  2048:   ~15 t/s   - solid default for most use cases
num_ctx  4096:   ~10 t/s   - longer docs and conversations
num_ctx  8192:   ~7 t/s    - only use when you actually need it
num_ctx  16384+: ~3-5 t/s  - significantly slower on CPU
num_ctx  40960:  ~1-2 t/s  - Qwen3's default - too slow for most

Your hardware will differ - run the benchmark in the next section to get your real baseline.

Recommended values by use case:

• Quick chat / Q&A: 2048 - handles most conversations comfortably
• Longer back-and-forth: 4096 - retains several thousand words of history
• Document analysis / summarization: 8192 - enough for most articles and reports
• Large docs or full codebases: 16384+ - only when you're actually pasting long content
• Qwen3:8b general use: Override to 4096 - don't leave it at the 40,960 default

How to change num_ctx - three ways:

Per API request (applies to that one call only):

curl http://localhost:11434/api/generate -d '{
  "model": "qwen3:8b",
  "prompt": "Your prompt here",
  "options": { "num_ctx": 4096 }
}'

Per CLI session (applies until you close ollama run):

/set parameter num_ctx 4096

Permanently, via Modelfile (bakes the setting into a named custom model):

# Create the Modelfile
echo 'FROM qwen3:8b
PARAMETER num_ctx 4096
PARAMETER temperature 0.3' > Modelfile

# Build it as a named model
ollama create qwen3-tuned -f Modelfile

# Run it - your settings are baked in
ollama run qwen3-tuned

Context is the conversation history the model carries into each response. Clearing it gives you a fresh start without restarting Ollama or waiting for the model to reload.

In the CLI (ollama run):

/clear    - wipes conversation history, keeps model loaded
/bye      - exits the session entirely
/?        - shows all available commands

Other useful commands while inside an ollama run session:

/set parameter num_ctx 4096     - change context window on the fly
/set parameter temperature 0.1  - change any parameter mid-session
/show                           - show current model info
/load modelname                 - switch to a different model
/save mysession                 - save the current conversation
Ctrl+G                          - open editor for multi-line prompts
Ctrl+C                          - stop a response mid-generation

Via the API:

The Ollama API is stateless by default. Each /api/generate request is independent - no conversation history carries over unless you explicitly pass it. To start fresh, just send a new request.

For multi-turn conversations, use /api/chat with a messages array. To clear history, start a new array without the previous turns. The context field on /api/generate is deprecated and being removed from Ollama - don't build new integrations around it.

curl http://localhost:11434/api/chat -d '{
  "model": "qwen3:8b",
  "messages": [
    {"role": "user", "content": "Hello, starting fresh"}
  ]
}'
# To clear: omit previous messages from the array
# Each new messages array is a clean conversation

Range: -1 (unlimited) to any positive number
Default: -1 (unlimited)

Maximum tokens to generate before stopping. Prevents runaway responses:

• -1: Unlimited (model stops when it feels done)
• 128: Stop after 128 tokens (~100 words)
• 512: Stop after 512 tokens (~400 words)
• 2048: Stop after 2048 tokens (full page)

Use when: You want predictable response lengths. Good for APIs where you need bounded latency.

Range: 1 to your CPU core count
Default: Auto-detect (all cores)

How many CPU cores Ollama uses. For example, on an 8-core CPU:

• 1-2: Slow, leaves CPU idle
• 8: All cores active (default, good)
• >8: Hyperthreading counts, can use more (usually 16 total)

Leave it on auto. Ollama detects your CPU and uses all available threads.

Goal: Consistent, deterministic responses. OpenClaw needs predictable JSON or structured text.

Recommended Settings:

curl http://localhost:11434/api/generate \
  -d '{
    "model": "qwen3:8b",
    "prompt": "Your prompt here",
    "temperature": 0.1,
    "top_p": 0.9,
    "num_predict": 2048,
    "num_ctx": 4096,
    "stream": false
  }'

Why these values:

• temperature: 0.1 → Focused, deterministic (reproducible output)
• top_p: 0.9 → Still allows valid variation, not robotic
• num_predict: 2048 → Reasonable max length for API
• num_ctx: 4096 → Enough memory for conversation context

Expected behavior: Responses are consistent. Same prompt = mostly same answer (good for testing and debugging).

Goal: Variety and creativity. You want different results each time, but still coherent.

Recommended Settings:

curl http://localhost:11434/api/generate \
  -d '{
    "model": "qwen3:8b",
    "prompt": "Write a short story about...",
    "temperature": 1.2,
    "top_p": 0.95,
    "num_predict": 1000,
    "num_ctx": 2048,
    "repeat_penalty": 1.2,
    "stream": true
  }'

Why these values:

• temperature: 1.2 → Creative, more varied outputs
• top_p: 0.95 → Very permissive vocabulary
• repeat_penalty: 1.2 → Prevent repetitive phrases
• stream: true → See creativity unfold in real-time

Expected behavior: Each run produces unique, interesting variations on the theme.

Goal: Speed matters. Users are waiting for a response. Trade some context for speed.

Recommended Settings:

curl http://localhost:11434/api/generate \
  -d '{
    "model": "llama3.2",
    "prompt": "What is...",
    "temperature": 0.7,
    "top_p": 0.9,
    "num_predict": 512,
    "num_ctx": 2048,
    "stream": true
  }'

Why these values:

• num_predict: 512 → Shorter responses (faster generation)
• num_ctx: 2048 → Not too large (faster processing)
• stream: true → First token appears faster (perceived speed)
• model: llama3.2 → 3B model, faster than qwen3:8b for simple tasks

Expected behavior: First token appears in <500ms, total response in 5-10 seconds.

Goal: Correct, working code. Logic must be sound.

Recommended Settings:

curl http://localhost:11434/api/generate \
  -d '{
    "model": "qwen3:8b",
    "prompt": "Write a Python function that...",
    "temperature": 0.3,
    "top_p": 0.9,
    "num_predict": 1024,
    "num_ctx": 4096,
    "repeat_penalty": 1.1,
    "stream": false
  }'

Why these values:

• temperature: 0.3 → Focused on correct syntax
• num_ctx: 4096 → Understand full code context
• repeat_penalty: 1.1 → Avoid redundant code

Expected behavior: Code is syntactically correct and logically sound most of the time.

Goal: Understand and analyze large texts. Need big context window.

Recommended Settings:

curl http://localhost:11434/api/generate \
  -d '{
    "model": "qwen3:8b",
    "prompt": "Summarize this document:\n\n[LARGE TEXT HERE]",
    "temperature": 0.5,
    "top_p": 0.9,
    "num_predict": 1024,
    "num_ctx": 8192,
    "stream": false
  }'

Why these values:

• num_ctx: 8192 → Can see entire document (trade-off: slower)
• temperature: 0.5 → Faithful to source material
• stream: false → You're doing analysis, not interactive chat

Expected behavior: Accurate summaries that capture main points. Slower (expect 20–30 seconds), but comprehensive.

Don't just trust recommendations. Test with your own prompts:

#!/bin/bash
PROMPT="Write a haiku about programming"

echo "=== Configuration A ==="
time curl -s http://localhost:11434/api/generate \
  -d '{
    "model": "qwen3:8b",
    "prompt": "'$PROMPT'",
    "temperature": 0.5,
    "stream": false
  }' | jq -r '.response'

echo ""
echo "=== Configuration B ==="
time curl -s http://localhost:11434/api/generate \
  -d '{
    "model": "qwen3:8b",
    "prompt": "'$PROMPT'",
    "temperature": 1.0,
    "stream": false
  }' | jq -r '.response'

Run the same prompt with different configurations. Notice speed and quality differences. Build intuition.

Once you find good settings, save them to files for reuse:

# OpenClaw Config
alias ollama-openclaw='curl -d "{\"model\": \"qwen3:8b\", \"temperature\": 0.1, \"top_p\": 0.9}"'

# Creative Config
alias ollama-creative='curl -d "{\"model\": \"qwen3:8b\", \"temperature\": 1.2, \"top_p\": 0.95}"'

# Fast Config
alias ollama-fast='curl -d "{\"model\": \"llama3.2\", \"temperature\": 0.7, \"num_predict\": 512}"'

Source the file and use aliases. Saves typing and keeps configs consistent.

Four key metrics for LLM inference:

Create a reusable benchmarking script:

#!/bin/bash

MODEL="${1:-qwen3:8b}"
PROMPT="Explain machine learning in 3 paragraphs"

echo "Benchmarking $MODEL..."
echo ""

# Run inference and capture timing
START=$(date +%s%N)
RESPONSE=$(curl -s http://localhost:11434/api/generate \
  -d "{
    \"model\": \"$MODEL\",
    \"prompt\": \"$PROMPT\",
    \"temperature\": 0.5,
    \"stream\": false
  }")
END=$(date +%s%N)

# Extract metrics
TOTAL_TIME=$((($END - $START) / 1000000))  # milliseconds
TOKENS=$(echo "$RESPONSE" | jq -r '.eval_count')
TPS=$(echo "1000 * $TOKENS / $(echo \"$RESPONSE\" | jq -r '.eval_duration') * 1000000" | bc -l)

echo "Model: $MODEL"
echo "Total Time: ${TOTAL_TIME}ms"
echo "Tokens: $TOKENS"
echo "Tokens/Sec: $(echo \"scale=2; $TPS\" | bc)"
echo ""
echo "Response:"
echo "---"
echo "$RESPONSE" | jq -r '.response'
echo "---"

Run it:

bash benchmark.sh qwen3:8b
bash benchmark.sh llama3.2
bash benchmark.sh llama3.2:1b

Test the same prompt across models to find your best balance of speed and quality:

Model            Time    Tokens  TPS     Quality
qwen3:8b       8.2s    82      10.0    Excellent
llama3.2         4.5s    85      18.9    Good
llama3.2:1b      2.1s    78      37.1    Basic

The 1B model is fastest, Qwen3 has the best quality. Your choice depends on your priority.

In another terminal, watch resource usage:

watch -n 0.5 'free -h && echo && top -n 1 -b | head -n 3'

Look for:

• Peak memory usage (should be <8GB)
• CPU utilization (should be 80%+ during inference)
• No thermal throttling (CPU stays stable frequency)

Most CPUs include an integrated GPU, but these are far too weak for LLM acceleration. For reference:

• Typical integrated GPU: ~1–2 TFLOPs
• RTX 4070: ~20 TFLOPs (10–20x more powerful)
• RTX 4090: ~82 TFLOPs (40–80x more powerful)

Integrated GPUs are orders of magnitude weaker than discrete gaming GPUs. For LLMs on integrated graphics, the CPU is actually competitive.

Ollama logs what hardware it's using. Check:

tail -50 ~/.ollama/ollama.log | grep -i gpu

If you see GPU references, Ollama detected it. If not, it's using CPU (default, which is fine for you).

GPU acceleration is worth it if:

• Nvidia RTX GPU: 2080 Ti or newer (30x-60x speedup)
• AMD Radeon: RX 6700 or newer (10x-20x speedup)
• Apple Silicon: Built-in GPU (3x-5x speedup)

An integrated GPU? Not worth the complexity. CPU inference is simpler and nearly as fast.

For local LLMs, consider GPU if:

• You want to run 13B+ models (currently too slow on CPU)
• You need sub-second response times (production use)
• You want to run multiple concurrent inferences

For now, stick with CPU. A modern multi-core CPU is plenty fast for a single Ollama instance running 7B models.

Before thinking about concurrent models, know what's already in memory:

ollama ps

NAME              ID              SIZE      PROCESSOR    UNTIL
qwen3:8b        845dbda0ea48    5.5 GB    100% CPU     28 minutes from now
llama3.2          a80c4f17acd5    2.5 GB    100% CPU     25 minutes from now

PROCESSOR shows whether your GPU is involved. On a CPU-only machine you'll see 100% CPU. On a machine with a compatible GPU (NVIDIA with CUDA, AMD with ROCm, or Apple Silicon), you'll see a split like 45%/55% CPU/GPU - higher GPU % means faster inference. The UNTIL column shows the keep-alive expiry for each loaded model.

On a 32GB system, loading your full stack looks like this:

OS + System:           ~2GB (always used)
Ollama Runtime:        ~1GB
qwen3:8b (q4_K_M):  ~5.5GB  - all-rounder, 128K context
llama3.2 (3B):         ~2.5GB  - speed tier, quick queries
phi4 (14B):            ~9.0GB  - heavy reasoning, complex problems
Reserve Buffer:        ~12GB   (available for gemma4:12b or larger models)
────────────────────────────────
Total Used:            ~20GB
Total Available:       ~12GB

With OLLAMA_KEEP_ALIVE set to -1 or a long duration, all three models stay resident. Switching between them is instant - the model is already warm in memory.

A sensible three-tier stack for most use cases:

Total disk for the first two: ~8GB. Add phi4 and you're at ~17GB. On 32GB RAM, that's comfortable headroom.

With multiple models loaded, route requests to the right model for the task:

# Complex task → Qwen3 for reasoning + 128K context
curl http://localhost:11434/api/generate \
  -d '{"model":"qwen3:8b","prompt":"Analyze this long document..."}'

# Quick factual lookup → Llama 3.2 for speed
curl http://localhost:11434/api/generate \
  -d '{"model":"llama3.2","prompt":"What does HTTP 429 mean?"}'

# Code review → Qwen3 for reliable instruction following
curl http://localhost:11434/api/generate \
  -d '{"model":"qwen3:8b","prompt":"Review this function for bugs..."}'

If all three are loaded in memory (keep-alive), these requests complete without any model loading delay. Ollama queues concurrent requests and processes them in order, switching models transparently.

The key to no-delay model switching is keeping models in memory. Set keep-alive system-wide via systemd:

sudo systemctl edit ollama

[Service]
Environment="OLLAMA_KEEP_ALIVE=-1"

sudo systemctl daemon-reload && sudo systemctl restart ollama

Ollama exposes a set of environment variables that control how it loads and runs models. These are the most useful ones for a Mini PC setup:

sudo systemctl edit ollama

[Service]
Environment="OLLAMA_KEEP_ALIVE=30m"
Environment="OLLAMA_MAX_LOADED_MODELS=3"
Environment="OLLAMA_FLASH_ATTENTION=1"

sudo systemctl daemon-reload && sudo systemctl restart ollama

Pin Ollama to specific cores to reduce OS scheduling overhead and give the model consistent CPU access:

# Add to your [Service] block in systemctl edit ollama:
CPUAffinity=0-7   # Pins to cores 0–7 (adjust to your core count)

On a 12-core machine, you might pin Ollama to 8 cores (0–7) and leave 4 cores (8–11) for the OS and other services. Experiment - the benefit varies by workload.

Context window size directly affects inference speed and RAM usage. Smaller context = faster tokens:

Context 1024:   ~20 tokens/sec (very fast, short conversations)
Context 2048:   ~15 tokens/sec (balanced - good default)
Context 4096:   ~10 tokens/sec (slower, good for most tasks)
Context 8192:   ~7 tokens/sec  (long docs, complex reasoning)
Context 32768+: ~3–5 tokens/sec (Qwen3 long context, only when needed)

You can override context size per request via the API using the options.num_ctx field. For most chat tasks, 2048–4096 is the sweet spot. Only go higher when you're actually sending long inputs.

curl http://localhost:11434/api/generate \
  -d '{
    "model": "qwen3:8b",
    "prompt": "Summarize this document...",
    "options": {
      "num_ctx": 8192
    }
  }'

If you're hitting RAM limits - models are being unexpectedly unloaded, or you see swap activity - try these:

# See current model memory usage
ollama ps

# Check system RAM
free -h

# Check if swap is being used (bad for inference speed)
swapon --show

If you're hitting swap, either reduce OLLAMA_MAX_LOADED_MODELS, switch to smaller model variants (e.g. llama3.2:3b instead of a 7B), or reduce OLLAMA_KEEP_ALIVE so models unload faster.

Logs live at ~/.ollama/ollama.log. Check for issues:

# View recent logs
tail -50 ~/.ollama/ollama.log

# Find errors
grep ERROR ~/.ollama/ollama.log | tail -10

• Weekly: Check disk usage (du -sh ~/.ollama/models)
• Monthly: Update Ollama (sudo apt update && sudo apt install ollama)
• Quarterly: Clean unused models (ollama rm model_name)
• Yearly: Backup models to external drive

Save your working configuration:

# Backup models
cp -r ~/.ollama/models ~/ollama-models-backup

# Document your settings
cat > ~/ollama-config.txt << EOC
Model: qwen3:8b
Temperature: 0.1
Top-P: 0.9
Context: 4096
Num Predict: 2048
EOC

• ☐ Ollama endpoint configured in OpenClaw (localhost:11434)
• ☐ Model name matches what you pulled
• ☐ Test prompt sent and received successfully
• ☐ Response quality acceptable
• ☐ Response time reasonable (5-20 seconds)
• ☐ No memory leaks after sustained use
• ☐ Reboot test: Ollama starts automatically, works after restart

Symptom: Service shows inactive or fails to start

systemctl status ollama
# Read error message

# Try manual start to see error
ollama serve

Common fixes:

• Permission denied: sudo chown ollama:ollama ~/.ollama
• Port conflict: Check sudo lsof -i :11434
• Corrupted model: Delete and re-pull

Symptom: "OOM Killer" in dmesg, processes killed

free -h
# If available < 2GB during inference, you're hitting limits

dmesg | grep -i killed | tail -5
# Shows what got OOM killed

Fixes:

• Reduce context window (num_ctx: 2048 instead of 8192)
• Unload unused models (ollama rm model_name)
• Close other apps consuming memory
• Add swap if permanently needed

Symptom: <5 tokens/sec (should be 10-15)

top -n 1
# CPU usage <80%? Issue might be elsewhere
# Check if CPU is being shared with other apps

iostat -x 1
# High wait time? Disk bottleneck

Fixes:

• Close heavy apps (browser, IDE, etc)
• Reduce num_ctx to 2048
• Use faster model (llama3.2:3b)
• Check for thermal throttling (watch -n 1 'cat /proc/cpuinfo | grep MHz')

Symptom: curl returns Connection refused

systemctl status ollama
# Make sure it's running

netstat -an | grep 11434
# Should show LISTEN on port 11434

curl http://localhost:11434/api/tags
# Should return JSON, not error

Fixes:

• Start service: sudo systemctl start ollama
• Check firewall: sudo ufw allow 11434
• Restart: sudo systemctl restart ollama

Symptom: Responses are nonsensical or repetitive

Fixes:

• Lower temperature (0.3 or less)
• Increase repeat_penalty (1.5)
• Reduce num_predict (limit length)
• Try different model (llama3.2 instead of qwen3:8b)

• ✓ All Ollama parameters and how to tune them
• ✓ Context window settings and the num_ctx tradeoff
• ✓ Benchmarking models against your specific hardware
• ✓ Running multiple models concurrently
• ✓ Environment variable optimization (OLLAMA_KEEP_ALIVE, flash attention, parallel requests)
• ✓ Monitoring, health checks, and troubleshooting

If you haven't already, this is the main thing. OpenClaw is the AI agent layer that runs on top of Ollama - it handles Discord and Telegram automation, hooks, skills, and memory. You already configured the integration in the OpenClaw Integration section above. If that's working, you've completed the core local AI stack.

Not running OpenClaw yet? The Mini PC Setup and EC2 Setup guides cover getting the full stack - Ollama + OpenClaw - running on your own hardware or in the cloud.

Ollama's CLI is great for testing. But if you want a proper chat interface - conversation history, file uploads, model switching from a browser - Open WebUI is the standard choice. It's a Docker container that wraps around your existing Ollama install. Takes about 30 minutes to set up and works on any platform.

Open WebUI Setup Guide - Private ChatGPT on Your Own Machine

The same Ollama instance you just optimized can power tab autocomplete and AI chat inside VS Code. Continue.dev connects directly to http://localhost:11434 and works with any model you've already pulled. You can run coding autocomplete on qwen2.5-coder:1.5b and general chat on qwen3:8b simultaneously - the models are already loaded.

Local Coding Assistant - Replace Copilot for Free

RAG (Retrieval-Augmented Generation) lets you upload PDFs, notes, or any text and ask questions about them. Your local Ollama handles the actual AI responses. AnythingLLM provides the document handling, chunking, and vector storage. Everything stays local - no cloud, no API keys.

Document Q&A with RAG - Chat with Your PDFs Locally

Ollama is the best balance of simplicity and control for most setups. But these are worth knowing:

• LM Studio - GUI-first local model runner. Easier for beginners, less scriptable.
• vLLM - High-throughput inference server. Better for multi-user or high-request loads. Much more complex to set up.
• GPT4All - Lightweight and beginner-friendly. Fewer models, less tuning control.
• llama.cpp - What Ollama is built on under the hood. Direct access if you need features Ollama doesn't expose yet.