Bernhard the Builder
BlogAboutAgents

What is NemoClaw doing with semantic search?

NemoClaw memory search on this Spark is still exact words. Here is the problem, and why I need to look further.

What is NemoClaw doing with semantic search?
Investigation
Bernhard Rieder
2026-09-13
6 MIN READ

What is NemoClaw doing with semantic search?

I asked my agent a simple question this afternoon: "What did we decide about the render farm?"
It had the answer. It had written it down itself, two days ago, in its own notebook. And it could not find it. Because the note said "GPU cluster" and I said "render farm". That is NemoClaw memory search as it runs on this Spark today: exact words, or nothing.
The house they built around the agent is serious work. Semantic search is the whole point of a machine that is supposed to remember. I need to find out what NemoClaw is actually doing with it.
NemoClaw 0.0.123OpenShell 0.0.106OpenClaw 2026.7.1Ollama 0.34.0NVIDIA Spark
Reproduced on this Spark //
Every claim below was reproduced on this machine. NemoClaw 0.0.123, OpenShell 0.0.106, OpenClaw 2026.7.1, Ollama 0.34.0. The raw commands and outputs are on the GitHub issues I filed. This is a problem report, not a verdict.

What NemoClaw memory search needs

An agent that really remembers needs two things. A notebook it can write into. And a librarian who turns every page into a fingerprint, so "render farm" and "GPU cluster" land next to each other. That fingerprint is the same idea as vector embeddings semantic relationships. Ours is qwen3-embedding:4b. It is tiny, it runs in Ollama right next to the chat model, and it is installed and loaded on this Spark right now. Idle. Waiting. Unreachable from inside the room.
So the question is not "does NVIDIA have an embedding model." They do. The question is what NemoClaw memory search can actually reach from the sandbox.

The house

NemoClaw house diagram - agent room, inference.local hallway, Ollama kitchen with chef and embedding librarian, and four gates between the room and semantic search
The house: one room for the agent, a chat-only hallway, a kitchen door that no longer opens from inside, and a librarian the room cannot reach.
NemoClaw builds a house around the agent and owns every door. That is the point of the sandbox. The agent sits in one room. The models cook in the kitchen: Ollama on the host, with the chef (gemma4:26b) and the librarian (the embedding model).
From the room to the kitchen there used to be two ways. A hallway that NVIDIA built, inference.local, guarded by a proxy with a password. And a kitchen door, the direct line to Ollama on port 11434.
>
The sandbox is doing its job. The librarian sits in the kitchen. From inside the room, I cannot reach him. That is the problem I need to understand.

Finding 1: the kitchen door only listens on localhost

The 0.0.123 update rewrote the Ollama settings on my machine. OLLAMA_HOST=0.0.0.0 became OLLAMA_HOST=127.0.0.1. From inside the room the door now answers upstream_unreachable. And it is not a one-time slip: every time NemoClaw touches the sandbox, it writes that line again. I take that as design, not a slip. The sandbox is not supposed to talk to Ollama on the LAN.

Finding 2: the hallway carries chat, not fingerprints

Fine, use the hallway. Ask it for a chat completion: works, fast, cached. Ask it for an embedding: 501 This server does not support embeddings. Ask it the Ollama way, /api/embed: connection not allowed by policy. The corridor NVIDIA built is excellent at one job. Today that job is chat.

Finding 3: no embedding plugin is loaded

Even with a corridor, OpenClaw needs its bundled Ollama plugin to talk to an embedding model. NemoClaw's image does not put that plugin on the allow list. The gateway says so itself, in its log, every start:
text
1no loaded plugin registered a memory embedding provider … falling back to keyword/FTS-only search.
The log is honest. NemoClaw memory search knows it is on keywords. That is the state I reproduced, not a hidden failure.

Finding 4: sqlite-vec is not in the image

The library that stores fingerprints in the database, sqlite-vec, is not in the image. require("sqlite-vec") → Cannot find module. Without it, a vector table cannot exist. Not "does not work". Cannot exist. Same story: the room was not packed for a librarian of its own.

What the docs still say

NemoClaw 0.0.123 still ships a documentation page titled "Configure Memory Search". I followed it today. It says: reach Ollama from the sandbox at http://host.openshell.internal:11434. That is Finding 1. It says: write the provider with nemoclaw <sandbox> config set. That command was removed in 0.0.120.
>
The manual still describes a path the current image does not open. That is a docs gap, or a path that has not landed yet. Either way, I need to look further.
Four independent gates, each one enough to keep search on exact words. I do not read that as anyone dropping the ball. I read it as a house that got the sandbox right, and a semantic-search path that has not caught up with the house.

OpenClaw has a bolt of its own

Suppose the in-sandbox path opened tomorrow. Run the one command that builds the fingerprint index, openclaw memory index --force, and OpenClaw 2026.7.1 crashes: "unable to open database file." On a fresh, writable database, owned by the right user, in a writable folder, with /tmp writable. I opened that same file from node's own sqlite module by hand: fine. The crash is inside OpenClaw's reindex routine. So even a perfect NemoClaw image would still hit this. I filed that one with OpenClaw.

What 0.0.123 already got right

This part matters, because it is most of the hard work.
Given, and I mean it: the agent's notebook is finally writable, changes apply in a second without a lock window, gateway restart behaves, the dashboard forward survives, the token survives a restart. The house got more comfortable. The library card is what is missing.
Taken, and I want to understand why: the configuration lock ("Shields"), the validated config get/set, the local image build, restoring older snapshots, and the direct Ollama route. Not one of these is mentioned next to the memory-search page it invalidates. If that is the intended sandbox, the page should say so. If it is not, the page is the bug.

The in-sandbox path, as of today

Five things would have to be true at the same time for semantic search inside the room:
  1. A route from the room to the librarian that carries embeddings
  2. The Ollama plugin allowed
  3. sqlite-vec present
  4. memorySearch.provider pointed at that route
  5. An index command that does not crash
Today: 0 of 5 on the documented path.
I am not going to fight the sandbox //
A brave admin can allow the plugin, drop sqlite-vec into a writable folder, and either hand the agent the hallway password or reopen Ollama on 0.0.0.0. I have done the equivalent dance. The next lifecycle command closes the kitchen door again. The next image drops the plugin. That is not a path. That is a patch against a house that was built to stay closed, and I respect the house.

If semantic search belongs inside the room

These would close the gap. Small, concrete, offered as notes, not a punch list:
  • Let inference.local carry /v1/embeddings and /api/embed
  • Allow the bundled Ollama plugin, or ship an embedding provider on the managed route
  • Put sqlite-vec in the image
  • Fix or retitle the "Configure Memory Search" page so it matches the image
  • Raise the default context floor: onboarding wrote a 16k window for an agent whose prompt alone is 9k tokens
  • Stop regenerating the gateway token at every container start
And on the OpenClaw Gateway configuration side: fix memory index --force on a plain, writable SQLite file. One bug, one repo.

What I need to investigate next

I could fork NemoClaw, fork OpenClaw, patch five things and be done by Monday. I would also be alone with my fork forever, re-applying patches after every release, which is exactly the life I had under the previous version. That is the wrong investigation.
The sandbox model is right. It kept me honest more than once. So the next question is not "how do I shove a librarian into the room." The next question is NemoClaw MCP memory: can the agents walk out to a librarian who already lives on this Spark, through a door the OpenClaw Gateway configuration already supports?
The hardware is there. gemma4 and the embedding model sit side by side in 121 GB of unified memory. The hallway already authenticates, caches, survives restarts. 0.0.123 already did the hard part: the notebook is open, config changes apply in a second, restarts are clean.
I want to write the follow-up. Not "it is fixed inside the room." The other question: is the librarian supposed to live next door?
>
Problem first. Semantic search is not reaching the sandbox. I am going to look further.

Latest Blog Posts

Scenario 12026-10-04

The Closed-Loop Sovereign Dividend and the Death of Scarcity

When one player owns the whole board, the game ends. How machine monopolies, a sovereign dividend, and universal high income make money itself obsolete.

Scenario 22026-10-04

The Planetary Machine Infrastructure: Why Robotics Cartels Naturally Become Public Utilities

Code can be offshored. Physics cannot. Why robotics and AI cartels anchored in gigawatt power plants and sovereign soil will end up as public utilities.

Scenario 32026-10-04

Breaking the Thermodynamic Barrier: Energy, AI Biology, and the True Gateway to Abundance

Atoms are not bits. Real abundance has to be engineered out of the dirt: cheap clean energy, AI biology, and a two-tier economy that is built on top of both.

Scenario 42026-10-04

The Cosmic Symbiosis: Superintelligence, SpaceX, and the Quest for Our Creators

Machines have no DNA and no ego. Why superintelligence becomes our muscle and our telescope, Starship our bridge, and the search for our creators our purpose.

A
B
C

This article is part of a Semantic Cluster. All links are managed by the Digital Architect AI.

Agent GatewayMCP · REST · WebMCP