CI and GitHub

Run your tests and a fault matrix on every pull request. danwo reports back as a GitHub check, with an annotation on each failing test line, and keeps one comment on the pull request up to date.

The workflowCopy Anchor Link

name: danwo
on: [pull_request]

permissions:
  checks: write
  pull-requests: write
  contents: read

jobs:
  danwo:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: 24 }
      - run: npm ci
      - run: npx @danwo/cli install-browser --with-deps --only-shell
      - run: npx @danwo/cli test --reporter github
        env:
          GITHUB_TOKEN: ${{ github.token }}
      - run: npx @danwo/cli matrix tests/checkout.danwo.ts --reporter github
        if: ${{ !cancelled() }}
        env:
          GITHUB_TOKEN: ${{ github.token }}
      - uses: actions/upload-artifact@v4
        if: ${{ !cancelled() }}
        with:
          name: danwo-runs
          path: .danwo/runs

--reporter github turns on the check, the pull request comment and the job summary. The last step uploads the flight records, so you can open a failure's recording.

What the check showsCopy Anchor Link

• pass and fail counts, and a row per failing test or fault;
• an annotation on each failing test line, in the pull request's Files changed tab;
• each failure's flight record, by its path in the uploaded artifact.

The same summary goes to the job summary page.

PermissionsCopy Anchor Link

CI defaultsCopy Anchor Link

With CI set (every hosted CI sets it), or with --ci, danwo test:

• retries a failing test once;
• reads the replay cache without writing to it;
• refuses .only, with exit code 2 naming the file and line.

Your config's retries and --retries still win.

Make no model callsCopy Anchor Link

A read-only cache still calls the model when a step has no recording. To make sure CI spends nothing, run with --replay-only:

npx @danwo/cli test --replay-only

Each agent step must then replay from the cache. A step that can't fails the test instead of calling the model:

✗ tests/checkout.danwo.ts › pays for an order  [Undecided]
  REPLAY_MISSING: agent.act("pay for the order") has no recording that replays (none was recorded), and replay-only runs no model. Run the test without --replay-only to record it, check it, and commit the cache.

4 passed, 1 failed (1 undecided)
cache: 11 replayed · 0 relocated · 0 handed off · 1 missed
model: 0 calls (replay-only)

To make every run replay-only, set it in the config:

cache: { dir: ".danwo-cache", replayOnly: true },

Record a failing step by running it locally without the flag, then commit the cache.

Bound the runCopy Anchor Link

Two flags stop a run from starting new test files. Files already running finish:

Run npx @danwo/cli doctor as a first step to see what a run would trip on: Node, Chromium, the config, the model's key. --json prints the checks for a script.

Exit codesCopy Anchor Link

Keep CI cheapCopy Anchor Link

Run a narrowed matrix on pull requests with --faults, and the full matrix on a schedule.

NextCopy Anchor Link