OpenClaw + Ollama: Running Local LLMs

+15 Mana ✨
OpenClaw Local Models(DOCUMENTATION)

Introduction

Ollama is a local model runner that lets you run large language models on your own hardware. OpenClaw integrates with Ollama as a model provider, allowing your agents to use locally-hosted models instead of cloud APIs. This eliminates API costs and keeps all data on your infrastructure.

Key Concepts

  • Ollama Provider: A model provider configuration that points OpenClaw to a local Ollama instance.
  • Model Scanning: The openclaw models scan command discovers all models available in your local Ollama installation.
  • Model Specification: Models are referenced using the format ollama:<model-name> in agent configuration.
  • Pull Before Use: Models must be downloaded to Ollama before OpenClaw can use them.

Real World Context

A defense contractor needs an AI coding assistant but cannot send any code or conversations to external APIs due to security regulations. By running Ollama with a capable open-weight model on an internal GPU server, they get a fully air-gapped AI assistant that OpenClaw routes to just like a cloud model.

Deep Dive

First, install and start Ollama, then pull a model:

bash
# Install Ollama
curl -fsSL https://ollama.ai/install.sh | sh

# Pull a model
ollama pull llama3.1:70b

# Verify it's available
ollama list

The ollama pull command downloads the model weights to your machine. The ollama list command confirms the model is ready to use.

Next, configure OpenClaw to use Ollama as a provider:

json
{
  "providers": {
    "ollama": {
      "type": "ollama",
      "baseUrl": "http://localhost:11434"
    }
  },
  "agents": {
    "local-assistant": {
      "model": {
        "primary": "ollama:llama3.1:70b"
      }
    }
  }
}

This tells OpenClaw to connect to the Ollama instance at localhost:11434 and configure an agent to use the llama3.1:70b model. The ollama: prefix tells OpenClaw to route requests through the Ollama provider.

Use model scanning to discover available models:

bash
# Scan for all available local models
openclaw models scan

# Output:
# Found 3 models via ollama provider:
#   ollama:llama3.1:70b (39GB)
#   ollama:codellama:34b (19GB)
#   ollama:mistral:7b (4.1GB)

The scan command checks all configured providers and lists available models with their sizes.

Common Pitfalls

  1. Not pulling the model first — OpenClaw cannot use a model that has not been downloaded to Ollama. Always run ollama pull before configuring the agent.
  2. Insufficient VRAM for large models — A 70B model needs approximately 40GB of VRAM. Running a model that does not fit in memory causes extreme slowness due to CPU offloading.

Best Practices

  1. Match model size to hardware — Use 7B models for machines with 8GB VRAM, 34B for 24GB, and 70B for 48GB+.
  2. Set Ollama as a failover — Configure a cloud model as primary and Ollama as failover for when the cloud API is unavailable.

Summary

  • Ollama runs LLMs locally, and OpenClaw integrates with it as a model provider
  • Models are referenced as ollama:<model-name> in agent configuration
  • Use openclaw models scan to discover available local models
  • Models must be pulled to Ollama before OpenClaw can use them
  • Match model size to available VRAM to avoid performance degradation
✓ Completed