claude-tokenization-proxy

A local Data Loss Prevention (DLP) proxy that sits between claude-code and the Anthropic API. It scans outgoing requests for secrets with GitGuardian's ggshield, swaps each secret for an opaque placeholder token before it leaves your machine, then restores the real secret in the streamed response on the fly — so Claude never sees your secrets but tooling that echoes a placeholder still gets the real value back.

claude-code ──POST /v1/messages──► proxy ──redacted──► api.anthropic.com
             (ggshield scan, secret → <<<gg_token_…>>>)  │  
claude-code ◄──SSE (detokenized)─── proxy ◄──SSE stream──┘
             (<<<gg_token_…>>> → real secret, streaming)

How it works

1. Request interception — the JSON body of POST /v1/messages is scanned by ggshield. Every detected secret is replaced with a unique <<<gg_token_[hex]>>> placeholder, and the token → secret mapping is kept in memory (src/store.ts). The redacted body is forwarded upstream.
2. Streaming detokenization — the Anthropic SSE response is piped through a stateful Transform (src/detokenize.ts) that scans the byte stream for placeholders and substitutes the original secret back. It holds back only the few characters that could still become a token, so a placeholder split across chunk boundaries (…<<<gg_to / ken_abc>>>…) is reassembled correctly.

Secrets live only in memory for the lifetime of the process — restart the proxy and the mapping is gone.

Requirements

• Node.js ≥ 23.6 (the proxy runs TypeScript directly via Node's built-in type stripping — no build step, no ts-node/tsx). On Node 22.6–23.5 run with node --experimental-strip-types src/index.ts.
• ggshield on your PATH and authenticated (ggshield auth login). If ggshield is missing or errors, the proxy logs a warning and forwards traffic unredacted (fail-open) — it never blocks your requests.

Setup

npm install        # dev-only deps (TypeScript types); not needed to run
npm start          # starts the proxy on http://127.0.0.1:8080

Point claude-code at the proxy:

export ANTHROPIC_BASE_URL=http://127.0.0.1:8080
claude

For the fish shell:

set -gx ANTHROPIC_BASE_URL http://localhost:8080
claude

That's the only change claude-code needs — it keeps using your normal ANTHROPIC_API_KEY / x-api-key, which the proxy forwards untouched.

Configuration

Env var	Default	Purpose
PORT	8080	Port the proxy listens on
ANTHROPIC_UPSTREAM_HOST	api.anthropic.com	Upstream API host to forward to

Tell Claude to leave the tokens alone

So the model preserves placeholders verbatim (e.g. when writing them to files), copy templates/CLAUDE.md into the CLAUDE.md of any project you run claude-code in:

# Security Constraints

Do not modify, remove, or evaluate strings formatted as `<<<gg_token_[a-z0-9]+>>>`.
Treat them as opaque identifiers and preserve them exactly as they appear when
writing or reading files.

Scripts

Command	What it does
npm start	Run the proxy
npm run dev	Run with --watch (restarts on file changes)
npm test	Run the detokenizer state-machine self-check
npm run typecheck	Type-check the source with tsc --noEmit

Limitations

• Detection is only as good as ggshield. It catches known secret patterns and generic high-entropy strings, not arbitrary sensitive prose.
• Re-inserted secrets are assumed to be escape-neutral (typical API keys/tokens). A secret literally containing a backslash or double-quote could mis-escape inside the response JSON. See the ponytail: note in src/detokenize.ts.
• The token map is in-memory only; it is not shared across proxy restarts or multiple proxy processes.

License

Copyright (C) 2026 Rootiest. Licensed under the GNU Affero General Public License v3.0 or later.