Case studies · Security test

We installed our own early-access build and tried to make it leak

26 of 26 automated checks passed, and 509 samples of the app's network connections all stayed on the machine. Then we found three things to fix.

9 October 2026 · 2 min read · Soham Mitra

How we tested

We downloaded the installer from the public early-access page, checked its SHA-256 against the published value, installed it silently and drove it through its own command line and local API, in a throwaway data folder. A monitor recorded every TCP connection owned by the app's processes, twice a second, for the whole run.

What held

• The API listened on 127.0.0.1 only; a connection to the laptop's LAN address was refused.
• No token, or a wrong one: 401. A request carrying a browser Origin: 403. A forged Host header (DNS rebinding): 403.
• A contract edited after approval was refused until approved again.
• Every connection in 509 samples was to this machine: the local model at port 11434 and the API itself.

Scope: the first run downloads the speech model once, from Hugging Face; the installer itself comes from GitHub. Neither carries meeting data.

What did not hold

• Notes invented decisions from example phrases in our own prompt. Critical, fixed the same day: see the write-up.
• The Ask screen said “needs Pro” inside the Pro build. Fixed the next day.
• Setup picked the smallest local model it found. Open.

The installers are unsigned, which Windows and macOS say on first run; the page explains how to verify each download.