Evidence and evaluation

You do not have to trust claims on paper. You can verify every invariant against live CockroachDB clusters and raw logs in the evidence/ directory.

Vector recall vs search beam size

CockroachDB uses C-SPANN approximate nearest neighbour search for vector indexes. You must balance retrieval speed against recall quality.

We measured recall across 3,000 vectors on a live CockroachDB cluster. We evaluated two distinct distributions. Dataset A contains uniform-random vectors. Dataset B contains clustered unit-norm vectors with 60 centers.

Beam size	Dataset A recall@10	Dataset A recall@50	Dataset B recall@10	Dataset B recall@50
1	0.233	0.096	0.193	0.073
2	0.295	0.155	0.229	0.128
4	0.375	0.238	0.324	0.216
8	0.513	0.400	0.473	0.403
16	0.716	0.638	0.695	0.638
32 (Default)	0.916	0.929	0.931	0.938
64 (Lambo default)	0.931	0.970	0.960	0.990
128	0.927	0.972	0.960	0.987
256	0.836	0.954	0.862	0.968

What this measurement teaches you

1. At the engine default beam of 32, you lose roughly 6% to 7% of true nearest neighbours. In Lambo, a missed neighbour causes a near-duplicate concept rather than an error.
2. Higher beam sizes do not always improve recall. Beam 256 degraded recall compared to beam 64 in both datasets.
3. Setting DEFAULT_VECTOR_BEAM_SIZE = 64 raises recall@50 to 0.990 with negligible query overhead.

You can reproduce this test with ./scripts/seed-vector-index.sh --count 3000.

Deterministic convergence

Multi-agent graphs must reach a stable state regardless of write arrival order. The demo’s rendered OUTCOME block is asserted to be byte-identical across two runs of the same scenario, by a test that compares the blocks directly. There is no tolerance, no ordering leniency, and no rounding.

Measured on 2026-08-16 at commit 0006824:

What was measured	Result
Consecutive parity runs, frozen build	100 of 100 passed, 0 failed
Independent run-pairs, every diff line classified	20 pairs, 0 unexpected differences
Distinct places two runs can differ at all	3, all intentional, all outside the asserted block
Rendered scores across all 20 pairs	identical

The three permitted differences are the freshly-minted session id, a random node id quoted in the high-risk warning, and a narration line’s canonization cycle index. The third is a genuine timer interleaving and fires in about 18 of 20 pairs, which is why two runs may differ in two places or three and both are healthy.

The classification matters more than the pass count. Twenty pairs were compared with every changed line sorted into a known bucket, and anything unsorted counted as unexpected. That turns “we saw nothing else vary” into “we looked for a fourth source of variation and there is none”. Full logs and the scripts that produced them are in evidence/demo-determinism/.

The same scenario against a live cluster

Run twice against a live CockroachDB Cloud cluster:

$ diff -u evidence/demo-live-1.txt evidence/demo-live-2.txt
IDENTICAL - T8.4 x2 met

• Exactly 12 interactions and 27 concepts created.
• Exactly 114 graph edges established.
• Exactly 5 canonization events recorded in CockroachDB.
• user schema promoted to Canonical with an exact blast radius of 9.

When this became a guarantee

That live-cluster capture predates the fix, and is worth reading with its date in mind. When it was taken, identical output was not yet guaranteed. A time-derived term in the daemon score varied between runs, so roughly one run in ten rendered two near-tied concepts in the opposite order. The term is recency, measured against the session’s wall-clock temporal extent. The capture is genuine, but it passed at about nine attempts in ten rather than by construction.

The cause was that the demo replays in around 110ms of logical span while stamping interactions from the wall clock, so ordinary scheduler jitter moved the denominator that every recency value is measured against. The demo now stamps from a script clock, making a session’s temporal extent a property of the script rather than of the machine it runs on.

The assertion was never weakened to accommodate the bug. It is the test that caught it, and it failed openly for as long as the bug existed. The earlier capture is kept exactly as recorded rather than re-taken, because a capture edited after the fact stops being evidence.

Live conflict and blast radius interception

When Agent B attempts to modify a load-bearing pillar, Lambo halts the regression. The engine returns structural blast radius warnings and conflict age.

── ACT IV — agent-b: recall("update user schema") ──────────────────────
  daemon event: Conflict on 'user schema' - contesting agents: agent-a, agent-b

  user schema [Entity, canonical] (score 2.27, blast radius 9)
  ⚑ Load-bearing pillar - 9 nodes depend on this. Modify with caution.
  Agent A wrote to it 15 seconds ago
  High-risk modification: high-value node a30fef36-9c49-412c-8f25-2eabf8dc27b4 (Canonical, blast radius 9) modified within 30s

  agent-b does not make the breaking change.

Immutable audit trail in CockroachDB

Lambo writes all promotions into canonization_events. You can inspect this audit trail directly through CockroachDB SQL or the Managed MCP Server.

CockroachDB Cloud - canonization_events table

nrynss-19495 (CockroachDB Cloud) | db: lambo

cockroach>SELECT node_id, from_status, to_status, blast_radius, occurred_at FROM canonization_events ORDER BY occurred_at;

node_id	concept	from_status	to_status	blast_radius	occurred_at
724c92b9…	user schema	None	Candidate	null	2026-08-15T14:50:08.306Z
3e3a6984…	add oauth_id to user schema	None	Candidate	null	2026-08-15T14:50:08.306Z
724c92b9…	user schema	Candidate	Venerable	null	2026-08-15T14:50:08.992Z
724c92b9…	user schema	Venerable	Canonical	⚑ 9	2026-08-15T14:50:09.600Z
1848c6e5…	add rate limiting middleware	None	Candidate	null	2026-08-15T14:50:10.103Z

(5 rows)✓ Verified via CockroachDB Cloud Managed MCP Server

You can view raw logs and execution plans in the evidence/ folder of the repository.

A swarm of very small models

The surface is small enough to be driven by models far below the size you would normally point at a tool-calling task. We ran three concurrent agents against one session for 151 seconds, each an instance of Qwen3-0.6B served locally by llama.cpp, given the agent skill as its system prompt and Lambo’s MCP tools as its only capability.

Measurement	Result
Tool calls	173, of which 165 succeeded
Call mix	86 recall, 45 derive, 40 record_action, 2 inspect
Malformed-argument refusals	0 — every call the model composed was well formed
Recall before write	43 of 43 tasks where the model acted, and none of its 45 derives without a prior recall in the same task
Canonization dedup	0.857 — 36 of 42 successful derives landed on concepts that already existed
Durability after SIGTERM	82 interactions and 12 concepts, matching the driver’s ledger exactly

What this measurement teaches you

1. A 0.6-billion-parameter model can operate the MCP surface without composing a single malformed call. The tool schemas are small enough to fit a model this size, which is the property that makes a swarm cheap.
2. The tasks in this run named the sequence to follow, so it measures whether a very small model can execute the protocol reliably against a live surface, not whether it reaches for memory unprompted.
3. Dedup rises as the swarm runs. Concurrent agents observing overlapping ground truth collapse onto shared canonical nodes rather than each growing a private copy of the graph.

Note

Not every small model can do this. Under the same llama.cpp build, LFM2-350M and functiongemma-270m cannot emit tool calls at all: probed through both an agent harness and the raw OpenAI tools API, they return prose where a tool_calls field should be. Tool-calling ability is the gate, and it does not track parameter count neatly. Every probe transcript is in evidence/swarm/probes/.

Ledgers, server logs, and the durability readbacks are in evidence/swarm/.

Recorded demonstrations

Every claim on this page has a log or a test behind it. Some of them also have footage, captured 2026-08-17 against the live cloudops-exhibit session in CockroachDB and the deployed portal. The walkthrough is on YouTube as Lambo: Durable canonical memory for agents.

Demonstration	What it shows
lambo demo	The full scenario with no config and no database, ending on a recall block that carries [canonical], a blast radius, and the load-bearing warning
Recall against the exhibit	what depends on SG-Base-VPC answered by graph traversal, with RDS-Lambo-Demo-DB returned as the top hit at score 9.50
Inspect	SG-Base-VPC at blast radius 5, its typed dependency edges, and which agent provisioned each dependent
The crossover guard	03_crossover_protect.py refusing a security-group deletion, naming the four concepts that would have been stranded, and issuing no AWS call on any path
The portal	The structure tree, a focused concept with its canonization gates, and a recall rendered as cards with score bars
An agent consulting memory	A model asked about deleting a security group, calling lambo_recall and lambo_inspect unprompted, and refusing. See Agent skill

The agent demonstration is the one worth reproducing yourself, because it is the only one where nothing is scripted: the prompt does not mention memory, and the tool calls are the model’s own choice. The Agent skill page has the exact prompt, the observed calls, and the conditions that change the outcome.

One honest boundary, stated because the footage makes it easy to blur: the CloudOps scripts in evidence/cloudops-run/ are deterministic Python playing agent roles. They provisioned real infrastructure and wrote real derives, but they are not model-driven. The agent demonstration above is.