feat: stateless AES-256-GCM crypto tokens with SSE-aware detokenization
Replace the in-memory token map with self-contained cryptographic tokens. ggshield-flagged secrets are encrypted with AES-256-GCM under a persistent 256-bit master key; the placeholder carries IV ++ authTag ++ ciphertext as hex, so the proxy is stateless and resolves tokens across restarts and replayed chat histories. The streaming detokenizer now parses SSE content_block_delta events and reassembles placeholders fragmented across many deltas — a raw byte scan can't bridge the interleaved event/JSON framing. Plain JSON responses keep the simpler contiguous byte scan. The master key is generated 0600 on first run at ~/.config/claude-tokenization-proxy/master.key (override GG_MASTER_KEY_FILE); deleting it makes existing tokens unrecoverable.
This commit is contained in:
@@ -18,17 +18,28 @@ claude-code ◄──SSE (detokenized)─── proxy ◄──SSE stream──
|
||||
## How it works
|
||||
|
||||
1. **Request interception** — the JSON body of `POST /v1/messages` is scanned by
|
||||
`ggshield`. Every detected secret is replaced with a unique
|
||||
`<<<gg_token_[hex]>>>` placeholder, and the `token → secret` mapping is kept
|
||||
in memory (`src/store.ts`). The redacted body is forwarded upstream.
|
||||
2. **Streaming detokenization** — the Anthropic SSE response is piped through a
|
||||
stateful `Transform` (`src/detokenize.ts`) that scans the byte stream for
|
||||
placeholders and substitutes the original secret back. It holds back only the
|
||||
few characters that could still become a token, so a placeholder split across
|
||||
chunk boundaries (`…<<<gg_to` / `ken_abc>>>…`) is reassembled correctly.
|
||||
`ggshield`. Every detected secret is encrypted with **AES-256-GCM** under a
|
||||
persistent master key and replaced with a self-contained
|
||||
`<<<gg_token_[hex]>>>` placeholder, where the hex payload is `IV ++ authTag ++
|
||||
ciphertext`. No server-side mapping is kept — the token carries everything
|
||||
needed to restore it (`src/tokenize.ts`, `src/store.ts`). The redacted body is
|
||||
forwarded upstream.
|
||||
2. **Streaming detokenization** — the response is piped through a stateful
|
||||
`Transform` (`src/detokenize.ts`) that finds placeholders, decrypts the hex
|
||||
payload, and substitutes the original secret back. For SSE streams it parses
|
||||
`content_block_delta` events and reassembles the reconstructed text, because
|
||||
the model emits a long placeholder fragmented across many deltas
|
||||
(`…"text":"<<<gg_to"}` / `{"text":"ken_abc>>>"…`) with event framing wedged
|
||||
between the fragments — a raw byte scan can't bridge that. Plain JSON
|
||||
responses, where the token is contiguous, take a simpler direct byte scan.
|
||||
|
||||
Secrets live only in memory for the lifetime of the process — restart the proxy
|
||||
and the mapping is gone.
|
||||
The proxy is **stateless**: tokens are cryptographically self-contained, so they
|
||||
resolve correctly even after a restart or when replaying older chat histories.
|
||||
The only persistent state is the 256-bit master key, generated on first run and
|
||||
stored `0600` at `~/.config/claude-tokenization-proxy/master.key` (override with
|
||||
`GG_MASTER_KEY_FILE`). Anyone with that key file can decrypt captured tokens —
|
||||
treat it like any other secret, and deleting it makes existing tokens
|
||||
unrecoverable.
|
||||
|
||||
## Requirements
|
||||
|
||||
@@ -160,8 +171,8 @@ that session. To make it stick:
|
||||
keys/tokens). A secret literally containing a backslash or double-quote could
|
||||
mis-escape inside the response JSON. See the `ponytail:` note in
|
||||
`src/detokenize.ts`.
|
||||
- The token map is in-memory only; it is not shared across proxy restarts or
|
||||
multiple proxy processes.
|
||||
- Tokens are decryptable by anyone holding the master key file. Multiple proxy
|
||||
instances must share the same `master.key` to resolve each other's tokens.
|
||||
|
||||
## License
|
||||
|
||||
|
||||
Reference in New Issue
Block a user