An AI trace store that treats its own data as dirty

Context

BadaBhai is an AI placement platform for skilled-trade workers in India, starting with CNC and VMC machine operators. Most of them have never had a résumé. A chat-first profiling flow in Hinglish turns their answers into a structured profile and a one-page trade résumé, and connects them to employers.

Workers type the way people talk: their name, their phone number, an ID number, the street they live on. That text goes to language models.

The problem

The platform already recorded what each AI call cost, but not what was asked or answered. The team’s observability tool was switched off pending a data-processing sign-off. When a profiling interview went wrong, there was no way to see why afterwards.

Storing prompts and completions fixes that, and creates the most sensitive table in the system.

Design: assume the text is dirty

Write path, on every AI call

1. AI call siteSix across the platform
2. Call metadataMasked text only; no raw-text parameter exists
3. EncryptAES-256-GCM
4. Trace tableCheck constraint rejects anything that is not ciphertext; worker reference mandatory

Read path, for an administrator

1. Feature flagOff by default; same 404 for every role
2. Role and egress cap
3. Audit record committedBefore any decryption
4. Decrypt one trace

Erasure: deleting a worker removes their traces by construction. Platform events carry text lengths, never text.

I designed the store on the assumption that anything written to it may contain personal data, and made the database enforce what code review alone would not:

• Ciphertext only, enforced by Postgres. A check constraint rejects any value that is not an AES-256-GCM token. A row holding readable prose is a constraint violation, not a code-review comment.
• Every trace belongs to a worker. The worker reference is mandatory, so erasing a worker under India’s DPDP Act removes their traces by construction. Calls that cannot be attributed to a worker are dropped and counted, never stored anonymously.
• Text never travels on the event stream. Platform events carry lengths, not content.
• Reading is a separate, audited act. List views return only non-personal fields. Decrypting a single trace sits behind a default-off flag, an egress cap and an audit record that must be committed before any plaintext exists. The flag is checked before the role, so with the flag off, every role gets the same 404 and learns nothing.

Upstream masking of personal data before model calls is a configurable policy on the platform. The store’s guarantees are designed not to depend on it.

The near-miss

The first version recorded each call by serialising the request at every call site. On worker-facing paths, that request is the worker’s raw words. When I reproduced it, a name, a phone number, an ID-shaped string and a street address landed in the trace verbatim — while the table’s own documentation said that could not happen.

The cause was subtle. The AI service was already producing masked text and returning it in the call metadata. But the metadata schema on the receiving side was a plain object schema, which silently strips fields it does not know about. The masked text was being discarded and the raw text recorded instead.

I did not patch the call sites. I removed the text argument from the recorder entirely. Text can now arrive only through the call metadata, after masking, and if it is missing the recorder stores nothing rather than falling back to raw input. A shared fixture of field names is asserted from both the TypeScript and Python test suites, so the two sides cannot drift apart again.

The second bug: rows that vanished

The admin list paginated with a keyset cursor on the creation timestamp. Postgres stores that timestamp to the microsecond; the cursor was serialised to the millisecond. Rows created within the same millisecond failed both sides of the cursor comparison and simply disappeared from the list.

Under concurrent writes, 17 of 60 rows went missing. The cursor now carries the full-precision sort key.

How it was verified

• Three adversarial reviews produced 13 findings. I reproduced all 13 before fixing any; none was a false positive. For two of them, the reviewer’s proposed root cause did not survive execution — the suggested cursor fix would not have worked.
• Twelve deliberate breakages (mutations) turned the new tests red, as they should. One more stayed green, which exposed a test that was asserting nothing useful; I rewrote it.

Trade-offs

• Encryption at rest with a gated decrypt path makes investigation slower than a plaintext log. That is the point.
• Dropping unattributable calls means some traces are lost. The count is recorded, so the gap is visible.
• This is a trace store, not an evaluation system. It makes failures reviewable; it does not score model quality.

What I learned

Fix the class of bug, not the instance. Six call sites could each have been patched to pass masked text. Removing the parameter means a seventh call site, written by someone else next year, cannot make the same mistake.