Fully Offline Operation

+15 Mana ✨
OpenClaw Local Models(DOCUMENTATION)

Introduction

OpenClaw can operate completely offline with no external API calls. By combining local models (Ollama or vLLM) with local embeddings (node-llama-cpp with GGUF models), you can run a fully air-gapped AI assistant. This is critical for classified environments, offline field work, and maximum data privacy.

Key Concepts

  • Air-Gapped Operation: Running OpenClaw with zero network connectivity to external services.
  • Local Embeddings: Using node-llama-cpp with GGUF format embedding models for memory search, replacing cloud embedding APIs.
  • GGUF Format: A binary format for quantized model weights, optimized for CPU inference with node-llama-cpp.
  • Offline Checklist: The set of components that must be local for fully offline operation: model provider, embedding provider, and channel adapters.

Real World Context

A security researcher works in a SCIF (Sensitive Compartmented Information Facility) with no internet access. They need an AI coding assistant for analyzing malware samples. By pre-downloading Ollama models and GGUF embedding files, they run a fully offline OpenClaw instance that never attempts to reach the internet.

Deep Dive

Fully offline operation requires three components to be local:

json
{
  "providers": {
    "ollama": {
      "type": "ollama",
      "baseUrl": "http://localhost:11434"
    }
  },
  "embeddings": {
    "provider": "local",
    "model": "nomic-embed-text-v1.5.Q8_0.gguf",
    "dimensions": 768
  },
  "agents": {
    "offline-assistant": {
      "model": {
        "primary": "ollama:llama3.1:70b"
      }
    }
  }
}

This configuration uses Ollama for the language model and a local GGUF file for embeddings. No external API calls are made.

The local embedding provider uses node-llama-cpp to run GGUF embedding models:

bash
# Download embedding model for offline use
wget https://huggingface.co/nomic-ai/nomic-embed-text-v1.5-GGUF/resolve/main/nomic-embed-text-v1.5.Q8_0.gguf

# Move to OpenClaw models directory
mv nomic-embed-text-v1.5.Q8_0.gguf ~/.openclaw/models/

The embedding model file must be downloaded while you still have internet access. Once downloaded, it runs entirely on CPU with no network calls.

The embedding cache (SQLite-backed, up to 50K entries) means frequently-accessed memories do not need to be re-embedded, reducing CPU load:

json
{
  "embeddings": {
    "provider": "local",
    "model": "nomic-embed-text-v1.5.Q8_0.gguf",
    "cache": {
      "enabled": true,
      "maxEntries": 50000
    }
  }
}

With caching enabled, repeated queries for the same content are served from the SQLite cache instantly.

Common Pitfalls

  1. Forgetting to download models before going offline — Both the LLM and the embedding model must be pre-downloaded. There is no way to pull models without internet.
  2. Underestimating CPU requirements for local embeddings — GGUF embedding models run on CPU. On a low-power machine, embedding large documents can be slow.

Best Practices

  1. Pre-warm the embedding cache — Before going offline, run a batch indexing job to pre-embed all important documents into the cache.
  2. Use quantized models — Q4 and Q8 quantized models offer a good balance between quality and resource usage for offline operation.

Summary

  • Fully offline operation requires local models (Ollama/vLLM), local embeddings (GGUF), and local channels
  • GGUF embedding models run on CPU via node-llama-cpp with no network calls
  • The SQLite embedding cache (50K entries) reduces redundant computation
  • All models must be pre-downloaded before going offline
  • Quantized models (Q4, Q8) optimize resource usage for constrained environments
✓ Completed