Running Hermes 3 with Ollama: Local AI for Tool Use & Reasoning

Hermes 3 is a family of open-source language models developed by Nous Research, fine-tuned specifically for instruction following, function calling, and tool use. It excels at structured outputs, JSON generation, and complex reasoning tasks. Unlike general-purpose models, Hermes 3 is optimized for agentic workflows — it can call functions, use external tools, and follow multi-step instructions reliably. When combined with Ollama, you get a production-ready AI backend that runs entirely on your hardware.

Hermes 3 is a language model — a neural network that processes text. It is NOT an agent framework. If you are looking for "Hermes Agent" that requires WSL and complex setup, that is a different tool entirely. This guide covers running the Hermes 3 model via Ollama, which works natively on Windows without WSL. Ollama provides a Windows installer that sets everything up automatically. For the agent framework setup, see our Hermes Agent setup guide.

Hermes 3 models are optimized for agentic workflows. They can call functions, use tools, and follow complex multi-step instructions with high reliability. Running via Ollama gives you: zero API costs after setup, complete data privacy with no data leaving your machine, full offline capability, and total control over model parameters like temperature and context length.

Download the Windows installer from ollama.com/download. Run the .exe installer and follow the setup wizard. Ollama will add itself to your system PATH automatically. Open PowerShell and verify with "ollama --version". You should see v0.5.0 or higher. No WSL is required — Ollama runs natively on Windows with full GPU support for NVIDIA cards.

Ollama hosts Hermes 3 via the Nous Research model page. Run "ollama pull hermes3:latest" for the most capable version. For older hardware with limited VRAM, use "ollama pull hermes3:8b" which requires only 6-8GB of RAM. For production use on powerful hardware, "ollama pull hermes3:70b" offers state-of-the-art reasoning. The download may take 10-30 minutes depending on your connection speed.

By default, Ollama runs in the background on localhost:11434. You can start it explicitly by running "ollama serve" in PowerShell. For API access from other devices on your network, set environment variables: "$env:OLLAMA_HOST="0.0.0.0:11434"" and "$env:OLLAMA_ORIGINS="*"", then restart the server. For persistent environment variables, use System Properties > Advanced > Environment Variables.

Hermes 3 shines at structured outputs. In PowerShell, test it with: "ollama run hermes3" then type a prompt like "Generate a JSON object with name, age, and email fields for a user named John". Hermes 3 will reliably output valid JSON. For function calling, provide a function schema in your prompt and ask the model to call it with specific arguments. This is ideal for building AI agents that interact with APIs and databases.

Use the Ollama JavaScript SDK: "npm install ollama". Then: "import { Ollama } from "ollama"; const ollama = new Ollama({ host: "http://localhost:11434" }); const response = await ollama.generate({ model: "hermes3", prompt: "Your prompt here" });". For Python: "pip install ollama" and use the equivalent Python API. Hermes 3's structured output makes it perfect for backend services that need reliable JSON responses.

If you get CUDA errors: update your NVIDIA drivers to the latest version. For CPU-only mode (slower but works on any PC): set the environment variable OLLAMA_NO_CUDA=1. If the model is slow: close other applications to free RAM, or use a smaller variant like hermes3:8b. For persistent environment variables: use System Properties > Advanced > Environment Variables rather than PowerShell $env: variables.