Encrypted fields

A privacy class says what a field is. encrypted() says how it rests: in its own D1 column as an AES-256-GCM envelope, not as plaintext. The two compose, and you can mark either one alone. The inventory reports both, so which of your PHI fields are encrypted at rest? is a question your control plane answers, not a second list you keep.

Mark a field

import { phi, pii } from "@pithy-sh/core/src/privacy/classes";
import { blindIndex, encrypted, encryptedTable } from "@pithy-sh/core/src/privacy/encrypted";
import { z } from "zod";

export const Patient = encryptedTable(
  z.object({
    id: z.string().describe("The patient's stable id, assigned by the app before the insert."),
    diagnosis: encrypted(phi(z.string().describe("The recorded condition."))),
    email: encrypted(pii(z.string().describe("Where we write.")), { index: "email" }),
    emailIndex: blindIndex("email"),
    notes: encrypted(z.string().describe("Free text the clinician wrote.")).nullable(),
  }),
  { name: "patient", key: "id" },
);

encryptedTable() declares the table’s stable name and its primary-key column on the schema itself. Both are bound into every ciphertext, below, so the app and the rotator read them from one place and cannot disagree. Marking a field adds no column: the envelope rests in the field’s own.

The key is assigned by your app, before the insert. A seal binds the primary key’s value, so the key has to exist before the row does. Use a text id, such as a UUID. encryptedTable() refuses an integer key, because SQLite assigns that at the insert.

The round trip is sealed()

The codec stays synchronous. An encrypted field’s stored side is the envelope as JSON and its parsed side is the envelope object, so Patient.parse(row) and Patient.encode(value) work as every other table’s do. The AES-GCM happens one step above, in a seam you build per request from env. That is how the key reaches the encode without a module-level schema ever holding one.

import { sealed } from "@pithy-sh/core/src/privacy/sealed";

const patients = sealed(Patient, { keys: c.env.PITHY_FIELD_KEYS });

const record = await patients.seal(value);   // seal each marked field, then Patient.encode
await db.insertInto("patient").values(record).execute();

const value = await patients.open(row);      // Patient.parse, then open each marked field

const hashes = await patients.match("email", input);   // one per live version
await db.selectFrom("patient").where("emailIndex", "in", hashes).selectAll().execute();

Forgetting the seam cannot write plaintext. Patient.encode handed a string where the codec expects an envelope fails validation, and a column holding plaintext does not parse.

It keeps the class. An encrypted phi field is still phi. The inventory lists it, redact masks it, and the compiler refuses it on a log line, both as an envelope after a parse and as the plaintext after open. redact(patients.opened, value) masks an opened value.

sealed() refuses at build, naming the schema, for four mistakes: an encrypted field with no encryptedTable() declaration, an indexed field with no blindIndex column, an index column naming no indexed field, and an encrypted field no path of object keys reaches, such as one inside an array.

What each ciphertext binds

The additional authenticated data is pithy/enc/1:<name>:<field path>:<pk>. A ciphertext moved to another field, another row or another table does not open.

• <name> is the name encryptedTable() declares, not the SQL table name. Renaming the table in a migration leaves every row openable.
• <field path> is the field’s path in the schema: diagnosis, or profile.phone for a field inside a sqliteJson column, which rests as its own envelope at that JSON path. A column is named for its field, so renaming an encrypted column changes its field path.
• <pk> is the primary key’s value, which is never updated. That is the guarantee.

Changing the name, a path or a key is a re-seal migration. Read every row with the old schema through sealed(), and write it with the new one.

What encryption costs

• Queryability. An encrypted column cannot answer WHERE, ORDER BY, a range or LIKE. Only equality comes back, and only through a blind index.
• Presence is visible. An absent optional field rests as SQL NULL, not sealed. IS NULL still works, and anyone reading the table sees which rows hold nothing.
• Length is visible. GCM does not pad. A field seals its JSON encoding, so a value whose JSON is m bytes rests as 4·⌈(m+16)/3⌉ + 60 bytes under a one-digit key version. A plain string of n bytes has m = n + 2: 84 bytes empty, 220 bytes at 100.

The blind index, and its leak

{ index: "exact" } or { index: "email" } gives a field a sibling blindIndex("<field>") column. It holds <version>:<hmac>, an HMAC-SHA256 of the normalized value under a key derived from that version’s field key. match() returns one hash per live version, so a lookup still finds a row mid-rotation.

The normalizers are the kit’s, by name. exact hashes the value as written. email applies NFC, trims and lowercases. It is a name rather than a function because the rotator re-indexes inside the secrets manager, which runs no code of yours.

Changing a field’s normalizer is a re-seal migration. The normalizer’s name is part of the key’s derivation, so old hashes stop matching the new lookup. Nothing reports it: no error, no stale count, no doctor finding. Run the migration through sealed() before the new lookup ships.

It leaks equality, and that is the trade. Two rows with the same normalized value have the same index. Ordering, ranges and prefixes do not come back at all.

D1 is the only destination

A KV list() returns keys, never values, so a KV envelope cannot be found by its key version. That means no stale count, no compare-and-swap and no rotation. Encryption that can never rotate looks like the feature and is not, so a KV store holding an encrypted field is refused at boot, naming the store, the field and D1.

A boot check cannot see a schema you hand to something else. So pithy deploy and pithy secrets provision derive the encrypted columns from your schemas, and any encrypted field outside D1 is stamped unrotatable. An unrotatable field blocks every retirement, below.

The field key

Fields seal under their own key, PITHY_FIELD_KEYS, never the secrets master key. The reason is blast radius. Every Worker with an encrypted field has to hold the key it seals with, and handing those Workers the master key would let any one of them open the whole vault.

• pithy secrets provision mints it per environment, beside the master key and in the same format, and only if it is absent. It binds it to every Worker that holds an encrypted field, whether or not that Worker composes secrets, and only those.
• pithy setup mints a local one into the dev secrets file.
• Your project must compose secrets on some Worker, because the secrets manager is what rotates the key. pithy provision refuses an environment where a Worker holds an encrypted field and nothing composes it, before anything is created. A key nothing rotates would be minted once and kept forever.

A field you mark tomorrow joins rotation on the next deploy, with no registration step.

Rotation is lazy

Every write seals under the current version, so a row moves off an old key the next time you write it. The normal state is two live versions: current and previous.

Every 30 days the manager’s nightly cron rotates the key, on its own Workflow, <project>-<env>-secrets-rotate-fields. A pass does three things, in order:

1. It adds a version and makes it current.
2. It re-seals and re-indexes the rows still under the version leaving the window, and only those. A stale row is found by json_extract, and each re-seal is a compare-and-swap on the version it read, so a row your app re-sealed in between is left alone. A large table is cleared in batches, across nights.
3. It retires that version once the guard reads clean.

The retirement guard

Retiring a version is the one act here that cannot be undone. Every ciphertext still under it becomes unopenable. So the guard is clean only when all three hold:

• No stamped envelope or index column holds a row under the leaving version.
• Every schema source the stamp was derived from was read.
• No encrypted field rests outside D1.

On anything else nothing is retired, and rotation and the backfill go on. A version still live when the next rotation is due is a failure, not a backlog. The manager raises the secrets/field_key_retirement_blocked audit event naming what held it, and pithy doctor fails on it.

While four versions are live, no version is added, rather than drift toward the Secrets Store’s 64 KiB ceiling. The backfill still runs and a clean guard still retires, so the ceiling clears itself once its cause is fixed.

Past a dirty guard, only a person retires a version. pithy secrets retire-version <v> --env <env> --past-guard prints the stale counts per column, every unrotatable field and every source the guard could not read. Then it asks you to type <env>/<v> back. There is no --yes, and off a terminal or under --json it refuses with core/person_required, so neither an agent nor a CI script can run it.

What the control plane reports

GET /control-plane/privacy marks each field record encryptedAtRest: true or false. It leaves the key out where the core cannot say, such as a Worker on a core older than encrypted fields, and never defaults it to false. Each schema also lists encrypted[], every encrypted field whether or not it carries a class, with unrotatable where it rests outside D1.

The manifest’s privacy block, and the posture on GET /control-plane/privacy, carry retirement: the guard over this Worker’s own encrypted columns.

retirement is absent on a Worker whose core predates encrypted fields.

The threat model

What it protects: a marked value at rest in D1, against anyone who reads the database without the field key. That is a D1 export, a backup, a dashboard query or a leaked read token. They see envelopes, key versions, approximate lengths, which optional fields are empty, blind-index hashes, and which rows share an indexed value.

What it does not: a compromised app Worker, or anyone holding PITHY_FIELD_KEYS, reads every encrypted field. The rotator’s Workflow journal is not a second copy: it holds version numbers and a digest, never a key. The field key is not the master key, so holding it opens no stored secret. A decrypted value is protected in memory by the class gates, not by this. Client-side or end-to-end encryption is a different threat model.

What rotation buys: a key compromise has a bounded remedy. Rotate, and within one period every row is under a key the attacker does not hold, and the compromised version is retired on evidence. It can be sooner, past the guard, by an operator who has seen what that costs.

What this does not do

• Encrypt the whole database. This is field-level and opt-in. Nothing is encrypted until you mark it.
• Search by range, prefix or order. Order-preserving encryption is not a trade Pithy offers.
• Encrypt on the client. End-to-end encryption is a different design.
• Mark the kit’s own schemas. Each capability adopts it in turn.