test(agents-vault): make the suite hermetic against git configuration #127

2026-09-03T23:20:32Z

rootiest commented

2026-09-03 23:20:32 +00:00

Summary

Unbreaks main. Run #907 failed the test job at 246/317 after #126 merged, which skipped the build-docs job and left the generated docs unregenerated.

The vault suite was hermetic against $HOME and ~/.claude — which is what every review checked — but not against git's own configuration. new_repo sets user.email/user.name on the repos the tests create; nothing set one on the repos the tool creates, namely the scaffolded vault and the AGENTS/ sub-repo. On a developer machine those inherited a global identity and committed fine, so the gap was invisible to every local run. On a runner with no gitconfig they died with Author identity unknown, and the suite reported agents-vault's own correct "nothing recorded" handling as 71 failures.

Nothing is wrong with the tool. It behaved exactly as designed — a commit that cannot be made is reported as a failed commit, and the tests asserting that behavior are the ones that passed.

Root cause

Three separate ways the suite read the ambient git config, all fixed by exporting overrides that reach the tool's own git invocations, not just the tests':

GIT_AUTHOR_* / GIT_COMMITTER_* — the actual failure. Environment rather than git config so it reaches the git commit calls inside agents-vault and agents-init.
commit.gpgsign=false — the mirror-image hazard, in the other direction. A developer with signing enabled has tool-created repos reach for a key partway through the run; on a hardware token that means a touch prompt mid-suite. A runner with no key fails the commit outright.
init.defaultBranch=main — the rebase fixtures build an upstream and a clone and need the two to agree on a branch name. A global saying main and git's bare default of master disagree, so the fixture failed to create the very rebase the test was about, reporting a tool failure that never happened. This accounted for the last 5 of the 71.

Notes

The commit's --no-verify-style hazard does not apply here: core.hooksPath was already pinned to /dev/null in new_repo. Only the three settings above were ambient.

Worth a follow-up, not fixed here: when a commit fails for lack of an identity, _agents_repo_sync reports (hook rejected it?). That guess is wrong in this case, and it is the same misdiagnosis pattern as the "rebase conflict" message that was corrected in #126 — a message naming a cause the code never checked.

Verification

GIT_CONFIG_GLOBAL=/dev/null GIT_CONFIG_SYSTEM=/dev/null fish tests/test-agents-vault.fish — reproduces CI exactly: 267/317 before, 317/317 after.
fish tests/test-agents-vault.fish with a normal config — 317/317, unchanged, so the fix does not mask anything locally.
fish tests/run-tests.fish under both conditions — lint 226/226, functional 13/13, vault 317/317, exit 0 each time.
Revert-and-observe: stashing the change drops the CI-like run back to 267/317, confirming the fix is what is doing the work.
Confirm CI is green on this branch, and that the build-docs job now runs and auto-commits the regenerated docs/fish-config.md, docs/fish-config.1, and component registry that run #907 skipped.

## Summary Unbreaks `main`. Run [#907](https://git.rootiest.dev/rootiest/fish-config/actions/runs/907) failed the `test` job at **246/317** after #126 merged, which skipped the `build-docs` job and left the generated docs unregenerated. The vault suite was hermetic against `$HOME` and `~/.claude` — which is what every review checked — but **not against git's own configuration**. `new_repo` sets `user.email`/`user.name` on the repos the *tests* create; nothing set one on the repos the *tool* creates, namely the scaffolded vault and the `AGENTS/` sub-repo. On a developer machine those inherited a global identity and committed fine, so the gap was invisible to every local run. On a runner with no gitconfig they died with `Author identity unknown`, and the suite reported `agents-vault`'s own **correct** "nothing recorded" handling as 71 failures. Nothing is wrong with the tool. It behaved exactly as designed — a commit that cannot be made is reported as a failed commit, and the tests asserting that behavior are the ones that passed. ## Root cause Three separate ways the suite read the ambient git config, all fixed by exporting overrides that reach the tool's own git invocations, not just the tests': - **`GIT_AUTHOR_*` / `GIT_COMMITTER_*`** — the actual failure. Environment rather than `git config` so it reaches the `git commit` calls inside `agents-vault` and `agents-init`. - **`commit.gpgsign=false`** — the mirror-image hazard, in the other direction. A developer with signing enabled has tool-created repos reach for a key partway through the run; on a hardware token that means a touch prompt mid-suite. A runner with no key fails the commit outright. - **`init.defaultBranch=main`** — the rebase fixtures build an upstream and a clone and need the two to agree on a branch name. A global saying `main` and git's bare default of `master` disagree, so the fixture failed to create the very rebase the test was about, reporting a tool failure that never happened. This accounted for the last 5 of the 71. ## Notes The commit's `--no-verify`-style hazard does not apply here: `core.hooksPath` was already pinned to `/dev/null` in `new_repo`. Only the three settings above were ambient. Worth a follow-up, not fixed here: when a commit fails for lack of an identity, `_agents_repo_sync` reports `(hook rejected it?)`. That guess is wrong in this case, and it is the same misdiagnosis pattern as the "rebase conflict" message that was corrected in #126 — a message naming a cause the code never checked. ## Verification - [x] `GIT_CONFIG_GLOBAL=/dev/null GIT_CONFIG_SYSTEM=/dev/null fish tests/test-agents-vault.fish` — reproduces CI exactly: **267/317 before, 317/317 after**. - [x] `fish tests/test-agents-vault.fish` with a normal config — **317/317**, unchanged, so the fix does not mask anything locally. - [x] `fish tests/run-tests.fish` under both conditions — lint **226/226**, functional **13/13**, vault **317/317**, exit 0 each time. - [x] Revert-and-observe: stashing the change drops the CI-like run back to **267/317**, confirming the fix is what is doing the work. - [x] Confirm CI is green on this branch, and that the `build-docs` job now runs and auto-commits the regenerated `docs/fish-config.md`, `docs/fish-config.1`, and component registry that run #907 skipped.

rootiest added 1 commit 2026-09-03 23:20:33 +00:00

test(agents-vault): make the suite hermetic against git configuration 75fbfa1e16

The suite was hermetic against $HOME and ~/.claude but not against git
config. new_repo set an identity on the repos the tests create; nothing
set one on the repos the tool creates -- the scaffolded vault and the
AGENTS/ sub-repo. Those inherited a global user.name/user.email on a
developer machine and committed fine, so the gap was invisible to every
local run and every review; on a runner with no gitconfig they died with
"Author identity unknown", and the suite reported the tool own correct
"nothing recorded" handling as 71 failures.

Supply the identity through the environment, which reaches the git calls
inside agents-vault and agents-init as well as the ones the tests make.

Pin commit.gpgsign off for the mirror-image reason: a developer with
signing enabled would otherwise have tool-created repos reach for a key,
and a hardware token would prompt for a touch partway through the run.

Pin init.defaultBranch because the rebase fixtures build an upstream and a
clone and need the two to agree on a branch name. A global saying main and
a bare default of master disagree, and the fixture then fails to create the
rebase the test is about, reporting a tool failure that never happened.

Verified with GIT_CONFIG_GLOBAL=/dev/null GIT_CONFIG_SYSTEM=/dev/null:
267/317 before, 317/317 after; unchanged at 317/317 with a normal config.

rootiest added 1 commit 2026-09-03 23:34:08 +00:00

test(agents-vault): inject failures in a way that survives uid 0 73d41da07d

Five fixtures forced a failure with chmod 500 on a parent directory. That
is silently useless for root: uid 0 bypasses the mode bits, the operation
succeeds, and a test asserting a failure path then reports the tool as
broken rather than the injection as ineffective.

CI runs the suite as root inside a container, so 21 checks failed there
while passing for every developer and every reviewer -- the same shape as
the git-identity gap in the previous commit. A fixture that quietly stops
injecting is worse than one that never existed: it reads as coverage.

Shim the single command each failure hinges on, matched by a path fragment
so nothing else in the run is disturbed. A command that exits 1 on purpose
does so for every uid. Which command matters: ensure_symlink removes an
existing link before it creates the new one, so the two adopt fixtures
have to fail at that rm to leave the live link in place, and shimming ln
would be too late. The restore fixture deletes the link first and so does
hinge on ln, and the failing --remote fixture hinges on git set-url.

The one remaining chmod 500 asserts a success, not a failure, and its real
proof is the direct assertions on where the stash landed.

Reproduce the root case locally without a container:
  unshare -r fish tests/test-agents-vault.fish

Verified: 296/317 as root before, 317/317 after; 317/317 unchanged as an
ordinary user; and neutering the shims reproduces exactly 296/317, the
count CI reported.

rootiest merged commit dc3236fddb into main

2026-09-03 23:40:57 +00:00

rootiest deleted branch fix/vault-test-git-identity

2026-09-03 23:40:58 +00:00

rootiest added the Area/Functions Kind/Enhancement labels 2026-09-03 23:41:10 +00:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: rootiest/fish-config#127