feat: claude-code tokenization proxy with ggshield DLP

Local DLP proxy between claude-code and the Anthropic API. Scans outgoing /v1/messages bodies with ggshield, swaps detected secrets for opaque placeholder tokens, and restores them in the streamed SSE response on the fly. Licensed under AGPL-3.0-or-later.
2026-06-19 22:48:45 -04:00
commit bd138a8d12
11 changed files with 1193 additions and 0 deletions
@@ -0,0 +1,110 @@
+# claude-tokenization-proxy
+
+A local Data Loss Prevention (DLP) proxy that sits between `claude-code` and the
+Anthropic API. It scans outgoing requests for secrets with
+[GitGuardian's `ggshield`](https://github.com/GitGuardian/ggshield), swaps each
+secret for an opaque placeholder token before it leaves your machine, then
+restores the real secret in the streamed response on the fly — so Claude never
+sees your secrets but tooling that echoes a placeholder still gets the real
+value back.
+
+```
+claude-code ──POST /v1/messages──► proxy ──redacted──► api.anthropic.com
+                                     │  (ggshield scan, secret → <<<gg_token_…>>>)
+claude-code ◄──SSE (detokenized)─── proxy ◄──SSE stream──┘
+                                     (<<<gg_token_…>>> → real secret, streaming)
+```
+
+## How it works
+
+1. **Request interception** — the JSON body of `POST /v1/messages` is scanned by
+   `ggshield`. Every detected secret is replaced with a unique
+   `<<<gg_token_[hex]>>>` placeholder, and the `token → secret` mapping is kept
+   in memory (`src/store.ts`). The redacted body is forwarded upstream.
+2. **Streaming detokenization** — the Anthropic SSE response is piped through a
+   stateful `Transform` (`src/detokenize.ts`) that scans the byte stream for
+   placeholders and substitutes the original secret back. It holds back only the
+   few characters that could still become a token, so a placeholder split across
+   chunk boundaries (`…<<<gg_to` / `ken_abc>>>…`) is reassembled correctly.
+
+Secrets live only in memory for the lifetime of the process — restart the proxy
+and the mapping is gone.
+
+## Requirements
+
+- **Node.js ≥ 23.6** (the proxy runs TypeScript directly via Node's built-in
+  type stripping — no build step, no `ts-node`/`tsx`). On Node 22.6–23.5 run
+  with `node --experimental-strip-types src/index.ts`.
+- **`ggshield`** on your `PATH` and authenticated (`ggshield auth login`). If
+  `ggshield` is missing or errors, the proxy logs a warning and forwards traffic
+  **unredacted** (fail-open) — it never blocks your requests.
+
+## Setup
+
+```bash
+npm install        # dev-only deps (TypeScript types); not needed to run
+npm start          # starts the proxy on http://127.0.0.1:8080
+```
+
+Point `claude-code` at the proxy:
+
+```bash
+export ANTHROPIC_BASE_URL=http://127.0.0.1:8080
+claude
+```
+
+For the [fish shell](https://fishshell.com/):
+
+```fish
+set -gx ANTHROPIC_BASE_URL http://localhost:8080
+claude
+```
+
+That's the only change `claude-code` needs — it keeps using your normal
+`ANTHROPIC_API_KEY` / `x-api-key`, which the proxy forwards untouched.
+
+## Configuration
+
+| Env var                 | Default            | Purpose                          |
+| ----------------------- | ------------------ | -------------------------------- |
+| `PORT`                  | `8080`             | Port the proxy listens on        |
+| `ANTHROPIC_UPSTREAM_HOST` | `api.anthropic.com` | Upstream API host to forward to |
+
+## Tell Claude to leave the tokens alone
+
+So the model preserves placeholders verbatim (e.g. when writing them to files),
+copy [`templates/CLAUDE.md`](templates/CLAUDE.md) into the `CLAUDE.md` of any
+project you run `claude-code` in:
+
+```markdown
+# Security Constraints
+
+Do not modify, remove, or evaluate strings formatted as `<<<gg_token_[a-z0-9]+>>>`.
+Treat them as opaque identifiers and preserve them exactly as they appear when
+writing or reading files.
+```
+
+## Scripts
+
+| Command           | What it does                                          |
+| ----------------- | ----------------------------------------------------- |
+| `npm start`       | Run the proxy                                         |
+| `npm run dev`     | Run with `--watch` (restarts on file changes)         |
+| `npm test`        | Run the detokenizer state-machine self-check          |
+| `npm run typecheck` | Type-check the source with `tsc --noEmit`           |
+
+## Limitations
+
+- Detection is only as good as `ggshield`. It catches known secret patterns and
+  generic high-entropy strings, not arbitrary sensitive prose.
+- Re-inserted secrets are assumed to be escape-neutral (typical API
+  keys/tokens). A secret literally containing a backslash or double-quote could
+  mis-escape inside the response JSON. See the `ponytail:` note in
+  `src/detokenize.ts`.
+- The token map is in-memory only; it is not shared across proxy restarts or
+  multiple proxy processes.
+
+## License
+
+Copyright (C) 2026 Rootiest. Licensed under the
+[GNU Affero General Public License v3.0 or later](LICENSE).