← Back to Touchwarden

Results, with the limits left in.

These are local experiments, not customer endorsements or evidence that every application path is protected. The original pilot records below are dated October 5, 2026; the current local demo and checker are described separately.

Rule-impact checking: 35 of 37

An internal evaluation on the founder's codebase applied demonstrated breaking edits for 37 approved rules. The checker flagged 35; two slipped through. This is a deliberately constructed evaluation, not a random sample of production pull requests or a general accuracy guarantee.

Medusa: kernel laws and the application boundary

The refund decision was moved into a Bend kernel. Five deliberate kernel mutations failed its proof. A scenario that restored a $0-refund bug while skipping two regression tests passed the tests but failed the proof. Existing tests already caught several direct regressions.

A separate gdp-ts experiment exercised one no-edit control and six misuse edits: four edits failed compilation; forged evidence and a direct raw write still compiled. At the recorded pilot state, touchwarden check did not run the compiler. That is historical evidence, not the current checker's capability: compilation, scoped sink checks and trusted-adapter review holds are now integrated with --run-verified.

Current local proof demo

The TanStack platform now exercises fresh layout/refund proofs and builds, inline bypass rejection, serialized-receipt rejection, and finite Chromium layout checks. Historical Medusa replays separately demonstrate narrowly scoped counterfactual blocking, including a control showing that a correct arithmetic proof alone does not prevent stale concurrent refunds. These are local research demonstrations with mocked effects, not production payments, customer adoption or exhaustive application protection.

What remains outside the claim

Full Medusa pilot record & reproduction commands
# Medusa refunds on a proven Bend kernel

A local copy of Medusa (`develop` at `044fbbb9`, MIT) with its refund decision moved into a small Bend kernel whose laws are proven, called from TypeScript, and gated by `touchwarden check`. Nothing was pushed to Medusa.

## What runs where

| Piece | File | Status |
|---|---|---|
| Kernel: the refund decision in whole "ticks" | `kernel/main.bend` | **proven** |
| Laws the payments owner signs | `kernel/LAWS.bend` | 4 laws, all proven |
| Proofs (the agent's job) | `kernel/PROOF.bend` | `All terms check.` in 0.1–0.3 s |
| Compiled kernel Medusa imports | `src/utils/refund-kernel.generated.ts` | built only after the proof passes; checked against a fresh compile |
| Amount conversion (decimal → ticks) | `src/utils/refund-kernel.ts` | **tested, not proven** |
| Call site | `payment-module.ts` `refundPayment_` | watched by the contract; type-checked on `touchwarden-gdp` (below) |

The four laws:

1. **never_exceeds_captured**: an allowed refund never takes the refunded total more than one minor unit past what was captured (Medusa's own tolerance).
2. **zero_is_rejected**: an explicit amount of 0 is refused (Medusa #16036, where 0 once refunded the whole order).
3. **missing_means_full**: no amount means the full payment, whenever that fits.
4. **allows_when_within**: a positive amount that fits is allowed, so a kernel that refuses everything fails.

The laws spell out the arithmetic themselves, so loosening a kernel helper cannot loosen a law. `kernel/mutate.py` checks this: 5 of 5 agent-style kernel edits fail the proof.

## Medusa's own tests with the kernel in place

`index.spec.ts` (payment module service): **45 / 46 pass, the same as untouched Medusa.** The one failure, "complete payment flow successfully when rounded numbers are equal", fails on unmodified Medusa too and is about capture, not refunds. All 13 refund tests pass, including the one-minor-unit tolerance, 0, negative amounts, no amount, and concurrent refunds.

## Eight agent scenarios

Each scenario is a branch off `touchwarden-base`, rebuilt and tested on its own (an earlier run tested a stale build; these numbers are from per-scenario rebuilds) (`scenarios.py`, results in `scenarios.results.json`).

| # | What the agent did | Medusa's tests | Touchwarden |
|---|---|---|---|
| 1 | Harmless kernel refactor (renamed a helper, updated its proof) | pass | **pass**: kernel proven |
| 2 | Re-added #16036 inside the kernel ($0 refunds everything) | caught (2 tests) | **fail**: proof fails |
| 3 | Same, and deleted the law that forbids it | caught (2 tests) | **fail**: judged by the laws on the base branch |
| 4 | Hand-edited the compiled kernel to always allow | caught (4 tests) | **fail**: compiled kernel doesn't match a fresh compile (proof itself passes) |
| 4b | Left the compiled kernel intact and appended an override below it | caught (6 tests) | **fail**: the whole file must equal a fresh compile, not just contain one |
| 5 | Bypassed the kernel with equivalent inline TypeScript | pass | **review**: kernel still proves, but the call site changed |
| 6 | Unrelated rename in the capture path | pass | **pass** (was a false positive; fixed 2026-10-03, stage 0) |
| 7 | Re-added #16036 and skipped the two tests that catch it | **pass (2 skipped), so the bug ships** | **fail**: proof fails |

## What this shows, honestly

- **Medusa's tests already catch the direct breaks (2–4b).** They have regression tests for these exact bugs. The proof adds coverage of every input, not the cases someone wrote, in a tenth of a second with no database.
- **The difference is scenario 7.** An agent can edit or skip tests in the same pull request. It cannot edit the laws, because Touchwarden reads `LAWS.bend` from the base branch. Scenario 3 shows the same for laws.
- **Scenario 5 is the boundary of the proof.** The proof covers the kernel. Whether the app still calls it is a watched call site, flagged for review, not proven.
- **Scenario 6 was a real false positive, now fixed.** Touchwarden used to mark any unclaimed change in a watched file for review. Unclaimed changes are now reported but pass; editing the contract itself still needs review.
- **Conversion is not proven.** Decimal amounts become whole ticks in TypeScript, rounded so the guard can only get stricter. Amounts above 2^48 ticks (about $28 billion in USD) are refused.

## Closing scenario 5 with gdp-ts proofs (2026-10-05)

Scenario 5 was the proof's boundary: the kernel is proven, but nothing forced the app to call it. This adds [gdp-ts](https://github.com/rauchg/gdp-ts) (Ghosts of Departed Proofs, Noonan 2018) on branch `touchwarden-gdp` of the local Medusa copy, so **recording a refund demands a compile-time proof that the kernel allowed it, about that exact payment and amount.**

How it works:

- `refund-kernel.ts` is the trusted module. It is the only file that can make a `RefundAllowed` proof, and only when the Bend kernel returns Allow.
- The proof is tied to the specific payment and requested amount that were checked (`name()` gives them compile-time names). At runtime it is one frozen `{ kind }` object.
- The refund record is created only in `recordRefund_`, which takes the proof, and gets the amount from the kernel module rather than from `data`.
- gdp-ts core is vendored (`proof/gdp.ts`, 168 lines, MIT, unmodified) rather than added as a dependency. `@gdp-ts/core` is 0.1.0 on npm.

Medusa's own tests on the gated code: **45 / 46, the same as before**, all refund tests pass, including concurrent refunds.

Agent-style edits, judged by `tsc` alone (`proof/negatives.py`, results in `proof/negatives.results.json`):

| Edit | Compiles? |
|---|---|
| No edit (control) | yes |
| Scenario 5 replayed: kernel replaced by equivalent inline TypeScript | **no**: there is no proof to pass |
| Check the requested amount, record the full payment | **no**: proof is about a different amount |
| Check one payment, record on another | **no**: proof is about a different payment |
| Reuse a proof under a second naming of the same values | **no**: fresh names, proof no longer fits |
| Kernel runs, proof replaced by `{} as any` | yes: needs a lint rule |
| Kernel runs, record written with `refundService_.create` directly | yes: needs a call-site rule |

What this shows, honestly:

- **Silent bypass becomes a compile error.** Before, scenario 5 compiled, passed Medusa's tests, and was only flagged for review. Now an agent has to visibly break the gate to get past it.
- **The proof is about values, not a boolean.** The three wrong-value cases are the bug class a plain `if (allowed)` cannot catch.
- **Two holes remain, and both are easy to spot statically.** A type assertion or `any` forges a proof, and the raw `refundService_.create` is still callable. gdp-ts's default lint only flags assertions *to proof types* (`{} as RefundAllowed<...>`); `{} as any` is caught only by its strict mode, which bans every `as` and `any` outside `proofs/` and would be noisy in Medusa (`noImplicitAny` is off). The practical fix is two Touchwarden rules: nothing asserted or `any` may reach a `RefundAllowed` parameter, and only `recordRefund_` may call `refundService_.create`. Neither is wired into `check` yet, so this is not yet "cannot bypass".
- **A proof does not mean fresh data.** It says the kernel allowed these inputs. The totals are plain values read under the payment row lock; the lock, not the proof, keeps them current.
- **It changes less app code than expected.** No existing Medusa method signature changed. `refundPayment_` keeps its signature; its body now wraps the check in `name()`, and the record moved into the new `recordRefund_` helper. The only changed signature is `decideRefund` in the kernel adapter. Like the swap, it belongs on a proposed branch, never applied automatically.
- **The gate is on the record, not the provider call.** The provider call runs after the transaction, outside the lock, and reads the amount back from the stored refund, so there is no checked value left to tie a proof to. The record, created under the lock, is where the amount is decided.
- **`touchwarden check` on the branch says `review`**: the kernel still proves and the compiled kernel still matches, but code around it changed. Expected; `check` does not run `tsc` yet, so the compile errors above come from the build, not from `check`.

Reproduce:

```bash
git -C MEDUSA_REPO checkout -B touchwarden-gdp touchwarden-base
python3 pilot/medusa/patch_proof.py MEDUSA_REPO
python3 pilot/medusa/proof/negatives.py MEDUSA_REPO        # 7/7 as expected
```

## `check` enforces the proof gate (2026-10-05)

A rule's `verification` can now carry a `gate` block. Like the laws, it is read from the base branch, so a PR cannot loosen it:

```yaml
gate:
  project: packages/modules/payment          # the tsconfig that must type-check
  proof: RefundAllowed
  trusted:                                   # the only files that may make proofs
    - packages/modules/payment/src/utils/refund-kernel.ts
    - packages/modules/payment/src/utils/gdp.ts
  sink: { call: refundService_.create, only_in: recordRefund_ }
```

With `--run-verified`, when a PR touches the project, a trusted file or the project's tsconfig chain, `check` builds the project with a pinned TypeScript 5.9 (in `packages/shared/contracts/src/proof-gate.ts`). It never loads or runs the repo's own compiler. It fails the rule when:

- the project no longer type-checks (a missing proof, or a proof about other values);
- the sink is called anywhere except `only_in`, in any file, including new ones that match no watch word;
- an `any`, a type assertion or a non-null assertion (`null!`) reaches a place that expects the proof, a proof-typed variable has no value (`let p!: RefundAllowed<...>`), or a proof is made outside the trusted files;
- `only_in` no longer takes the proof as a parameter;
- the PR adds a `@ts-ignore`, `@ts-expect-error` or `@ts-nocheck`;
- the tsconfig no longer enables `strictFunctionTypes` (without it, proofs are not checked). This fails closed.

An edit to a trusted file or to the tsconfig chain always goes to review, because those files decide when a proof is made.

On the gated Medusa (branch `touchwarden-gated`), `proof/gated_check.py`, results in `proof/gated_check.results.json`:

| Agent PR | `check` | Reason it gives |
|---|---|---|
| Honest edit to the refund error text | **pass** | kernel proven, code around it still type-checks with the gate |
| Scenario 5: kernel replaced by inline TypeScript | **fail** | `Property 'proof' does not exist` |
| Kernel runs, record written with `refundService_.create` directly | **fail** | `refundService_.create is called outside recordRefund_` |
| Kernel runs, proof replaced by `{} as any` | **fail** | `` an `any` or asserted value is used where a RefundAllowed proof is required `` |
| Trusted module edited to also allow an explicit 0 | **review** | kernel still proves; the trusted module changed |

Scenario 5 goes from **review** to **fail**, and an honest call-site edit goes from **review** to **pass**. Fixture tests: 11 new cases in `check.test.ts` on `fixtures/proof-gate-app`; the full contracts suite is 18 / 18 (`pnpm test` in `packages/shared/contracts`).

Still not covered:

- **The trusted module is reviewed, not proven.** The last row shows why it must never pass on its own.
- **Medusa's service factory also generates a public `createRefunds`.** Nothing calls it today, and the gate watches one sink. A second sink entry would close it.
- **The sink rule matches the call as written.** It does not see an alias such as `const s = this.refundService_; s.create(...)`.
- **This is Touchwarden's compiler, not the repo's build.** It is the same 5.9 that Medusa uses here. A repo on a different version could see different errors.
- **A proof is not fresh data.** The row lock still keeps the totals current.

## Touchwarden changes made for this (in `packages/shared/contracts`)

- Bend laws are read from the base branch, so a pull request cannot weaken them. New test: "PR cannot weaken LAWS.bend to pass its own broken kernel".
- Any change inside the kernel folder or to the compiled kernel triggers the proof.
- The compiled kernel must be exactly a fresh compile of the proven source plus one export line (an earlier substring check could be bypassed by appending an override; caught in review and fixed). New test: "compiled kernel must be exactly a fresh compile of the proven source".
- A failed compiled-kernel comparison is reported as such, not as a proof failure.
- A proven kernel with a changed call site is marked for review.
- `TOUCHWARDEN_BEND` sets the Bend command (here `bun ../bend/bend2/main.ts`).

## Reproduce

```bash
python3 pilot/medusa/kernel/mutate.py                      # 5/5 kernel breaks fail the proof
python3 pilot/medusa/kernel/build.py OUT.ts                # proof, then compile
python3 pilot/medusa/patch_refund.py MEDUSA_REPO           # route refunds through the kernel
DB_HOST=localhost DB_PORT=5433 DB_USERNAME=user DB_PASSWORD=password \
  python3 pilot/medusa/scenarios.py MEDUSA_REPO            # the eight scenarios
```
Recorded TypeScript negative-test results (JSON)
[
  {
    "edit": "control: no edit",
    "compiles": true,
    "as_expected": true,
    "errors": []
  },
  {
    "edit": "bypass the kernel with equivalent inline TypeScript",
    "compiles": false,
    "as_expected": true,
    "errors": [
      "MEDUSA_REPO/packages/modules/payment/src/services/payment-module.ts(1041,18): error TS2339: Property 'proof' does not exist on type '{ verdict: \"allow\"; }'."
    ]
  },
  {
    "edit": "check the requested amount, record the full payment",
    "compiles": false,
    "as_expected": true,
    "errors": [
      "MEDUSA_REPO/packages/modules/payment/src/services/payment-module.ts(1041,13): error TS2345: Argument of type 'RefundAllowed<N, M>' is not assignable to parameter of type 'RefundAllowed<N, N>'."
    ]
  },
  {
    "edit": "check one payment, record on a different payment",
    "compiles": false,
    "as_expected": true,
    "errors": [
      "MEDUSA_REPO/packages/modules/payment/src/services/payment-module.ts(1046,13): error TS2345: Argument of type 'import(\"MEDUSA_REPO/packages/modules/payment/src/utils/refund-kernel\").RefundAllowed<N, M>' is not assignable to parameter of type 'import(\"MEDUSA_REPO/packages/modules/payment/src/utils/refund-kernel\").RefundAllowed<N, M>'."
    ]
  },
  {
    "edit": "reuse a proof under a second naming of the same values",
    "compiles": false,
    "as_expected": true,
    "errors": [
      "MEDUSA_REPO/packages/modules/payment/src/services/payment-module.ts(1042,13): error TS2345: Argument of type 'import(\"MEDUSA_REPO/packages/modules/payment/src/utils/refund-kernel\").RefundAllowed<N, M>' is not assignable to parameter of type 'import(\"MEDUSA_REPO/packages/modules/payment/src/utils/refund-kernel\").RefundAllowed<N, M>'."
    ]
  },
  {
    "edit": "forge the proof with a type assertion",
    "compiles": true,
    "as_expected": true,
    "errors": []
  },
  {
    "edit": "skip recordRefund_ and create the record directly",
    "compiles": true,
    "as_expected": true,
    "errors": []
  }
]

Prior art and tooling: Bend, gdp-ts and Ghosts of Departed Proofs. Public upstream experiments do not imply maintainer endorsement.