I asked an AI assistant a small history question: what did American newspapers say about the 1918 flu? It came back with 14 photographs, 469 newspaper pages, and the Library of Congress’s own subject heading for the pandemic.

The answer was the easy part. What I wanted to test was whether I could trust it. I ran the assistant inside a sandbox, and at one point I turned its network off to see what it would do. Along the way the test turned up a real bug in my own sandbox check, one I had first written off as an amusing quirk. This post covers both.

The tool

The assistant is Sandy, a project of mine. I wrote about why I built it earlier this month, so here’s the short version. Sandy only talks to the systems you list in its config, and it refuses to start unless it’s inside a sandbox. It gathers data through MCP servers (small programs that give an AI a fixed set of tools), then writes a report with a footnote on every claim, pointing to the exact call that produced it.

This time I wanted to try it on something I hadn’t built myself. I picked an MCP server I found on a marketplace, one I’d never run before, and set out to see whether I’d end up with a report I could stand behind.

Setting up a stranger’s server

The server is @cyanheads/libofcongress-mcp-server. It has six read-only tools covering the Library of Congress’s digital collections, the Chronicling America newspaper archive (with the scanned text), and the Library’s subject headings. It doesn’t need an API key.

Sandy can set a server up from the server’s own description file:

sandy import \
  https://raw.githubusercontent.com/cyanheads/libofcongress-mcp-server/main/server.json \
  --registry-source package \
  --tools libofcongress-mcp-server=libofcongress_search,libofcongress_get_item,\
libofcongress_search_newspapers,libofcongress_get_newspaper_page,\
libofcongress_search_subjects,libofcongress_browse_collections \
  --yes --apply

Sandy import
  format:  MCP Registry server.json (converted; tools declared by you via --tools)
  sha256:  e4d235d178ca…
  servers:
    • libofcongress-mcp-server (stdio) — allows libofcongress_search, … of 6 exposed
  allowlist: as set by --tools
  applied: yes (live config updated + re-validated)

That one command downloaded the file, converted it to Sandy’s format, pinned the exact version (0.3.0), and added the server to my config. Before it did, it asked me two things instead of guessing:

  • Where the server should run. This one can run on my machine or on a hosted endpoint someone else operates. The two aren’t equivalent: with the hosted one, every question I ask goes to a third party. When I left out --registry-source, Sandy stopped and asked me to choose.
  • Which tools it may use. The description file doesn’t list the tools, so Sandy made me name them. Only the tools I list can ever be called, and that allowlist is the core of the security model.

If anything is wrong, the import changes nothing. When my config pointed at a file that didn’t exist yet, --apply refused and left every file exactly as it was.

The bug I almost laughed off

Sandy should refuse to run outside a sandbox. So the first thing I did was run its health check on my plain desktop, with a config that said “I’m in Docker”:

Sandy check
  sandbox:     docker (declared: docker)
  ...
  RESULT: OK

It said OK. It thought my desktop was a Docker container.

My first draft of this post called that “an amusing detail”: my machine isn’t containerized, but something about it tripped the container check. When I came back to it, it didn’t seem amusing any more. A sandbox check that says “yes” outside the sandbox is the one bug this project can’t have. Any config that said docker would have started on a bare machine with no boundary at all.

The cause was simple once I found it. One of the ways Sandy recognizes Docker is by looking through the list of mounted filesystems for Docker’s fingerprints. Inside a container, the root filesystem itself carries those fingerprints. But a machine that runs containers can see every container’s filesystem in its own mount list, and I had a few running. Sandy was finding Docker’s fingerprints anywhere in the list, not only on its own root.

The fix is to look only at the process’s own root filesystem. The same check on the same desktop now says:

error: sandbox violation (root-missing): Sandy requires a sandbox boundary;
none detected. Refusing to start.   (exit 4)

It still recognizes a real container. I added a test built from my desktop’s actual mount list, and the fix shipped in v0.3.0. If you run Sandy on Linux alongside Docker, please upgrade.

Inside a real boundary

Next I gave Sandy a real sandbox: a Docker container with a read-only filesystem and one writable folder. I’m about to let a stranger’s program run on my machine, so it should only be able to write where I say.

docker run --read-only --tmpfs /tmp \
  -v ~/sandy:/sandy:ro -v ~/sandy-loc:/sandy-loc \
  sandy-loc:0.3.0 node /sandy/bin/sandy.js check -c /sandy-loc/sandy.json

The server crashed on startup:

✗ libofcongress-mcp-server — startup failure (terminal): MCP error -32000:
  Connection closed (server stderr: npm error enoent … mkdir '/home/node/.npm' …)
RESULT: DEGRADED

The imported config launches the server with npx, which wants to write to its download cache in the home folder, and the home folder is read-only. So the sandbox did its job.

The part in brackets is new, and it’s the second fix this test produced. Before, all Sandy said was Connection closed, and I had to rerun the server by hand to learn why. Sandy was capturing the server’s error output and never reading it. Besides hiding the reason, that could freeze a server that printed too much, because nothing was draining the output. Now the output is drained, and when a server dies at startup, its last error lines appear in the message.

This crash also shows a distinction that’s easy to miss. The version pin and the allowlist control what the server may be asked to do. The sandbox controls what it can actually do. The server I had just pinned and allowlisted still couldn’t write outside the one folder I allowed.

The fix is one line: point the config at a copy of the server installed in advance at the pinned version, instead of npx. I also passed in a log folder and a descriptive user agent, which the Library asks for.

MCP servers:
  ✓ libofcongress-mcp-server — connected
RESULT: OK

The run

I asked for photographs of the epidemic, Illinois newspaper pages from 1918, and the Library’s subject headings: three calls, one at a time, because the Library allows about 20 requests a minute and going over gets you blocked for an hour.

✓ photos     (88ms)
✓ newspapers (3043ms)
✓ subject    (608ms)
• done: 6 claim(s), 0 gap(s)

The results read like 1918, including how badly the scanned text holds up. The Rock Island Argus of October 18, 1918, comes back as FRIDAY TIFE HOCK ISLAND ARGUS OCTOBER 18 1018. The Ottawa Free Trader-Journal is “Established 1S4”, and its own name is spelled TRABERJOURNAJL. A photo caption from the St. Louis Red Cross Motor Corps reads: “mask-wearing women holding stretchers at backs of ambulances.” The subject index has an entry named, exactly, Influenza Epidemic, 1918-1919.

That garbled text is why every line needs a source. If a newspaper’s date comes back as 1018, you need to know exactly which scan it came from before you repeat it.

Where every line came from

Each claim in the report has a footnote. At the bottom, a Provenance table maps every footnote to the call behind it:

Ref Server Tool Args (sha256) At
[3] libofcongress-mcp-server libofcongress_search_newspapers c65bdd7c… 2026-09-26T04:53:23Z

The audit log has the same call. It records a hash of the arguments rather than the arguments themselves, since payload logging is off by default:

{"session":"df0e0844-…","seq":4,"type":"mcp_call",
 "data":{"tool":"libofcongress_search_newspapers","argsHash":"c65bdd7c…",
         "durationMs":3043,"outcome":"ok"}}

The session field is also new in v0.3.0. Every run appended to the same log file, and each run numbers its events from 1, so without a session id two different “event 4”s look the same.

Turning the network off

The test I cared most about: I ran the same request again with the container’s network switched off.

docker run --network none …   # same config, same request

claims (0):
  (none)
gaps (3):
  − photos:     call-failed — … fetch failed (failed after 4 attempts)
  − newspapers: call-failed — … fetch failed (failed after 4 attempts)
  − subject:    call-failed — … fetch failed (failed after 4 attempts)

The server retried each call, then gave up. The report Sandy wrote for a question with a well-known answer says:

No data could be retrieved. … Nothing in this report is fabricated to fill the holes.

A language model could have written a convincing paragraph about the 1918 flu from memory, and I might not have noticed. That doesn’t happen here because the part that writes the report has no path from “the source was unreachable” to “here’s a confident sentence.” It records a gap. That’s a property of the code, not something I’m trusting the model to remember.

What changed

The test turned into a release. v0.3.0 collects everything this experiment found, first on a Mac and then again on Linux:

  • The sandbox check no longer mistakes a machine running Docker for a Docker container.
  • A server that fails to start says why, and a server that prints a lot can no longer freeze.
  • Importing a server straight from its registry description file works, and asks before choosing where it runs.
  • Sandy never silently drops a report. If it can’t write one, it says report: NOT WRITTEN and exits with an error.
  • Footnotes link properly on GitHub, and every audit event carries a session id.

The first item is the one I’m glad I didn’t publish as a joke.

Why it’s built this way

The design splits the work in two. The part that reasons (the model) suggests a plan. The part that executes is fixed, deterministic, and audited: it decides what’s allowed, runs only that inside the boundary, and writes down what happened. The model can be swapped out and is never trusted; the executor doesn’t improvise. That’s what lets me check “it can’t leave the sandbox” with a docker run flag instead of taking it on faith.

The same split is why this bug mattered so much. The executor is only as trustworthy as its check that the boundary is real, and until this test, that check had a hole.

Try it yourself

git clone --branch v0.3.0 https://github.com/techeretic/sandy && cd sandy
npm ci && npm run build

# 1. Import a public MCP server you've never run, and choose its tools:
node bin/sandy.js import <its server.json URL> --registry-source package \
  --tools <server=tool1,tool2,…> --yes --apply

# 2. Put it in a real boundary (read-only filesystem, one writable folder):
docker run --read-only --tmpfs /tmp … node /sandy/bin/sandy.js check -c sandy.json

# 3. Ask one small question, then follow one footnote to the bottom of the report.

If you try the Library of Congress server, please set a descriptive LOC_USER_AGENT and keep it to one request at a time.

And try the check on a bare machine first. It should refuse to start. If it doesn’t, I’d like to hear about it.