The AI code reviewer that cancelled itself

Our automated Claude code reviewer spent a day reviewing nothing. Its own "Reviewing PR #N" progress comment cancelled the review that posted it, 24 seconds in, on every new pull request.

Days later SWARECO found the same failure on a client's repository, where the canceller was CodeRabbit's auto-summary comment. Every review was dead within 10 to 20 seconds of the PR opening.

The root cause was the same both times. GitHub evaluates a workflow's concurrency block before any job-level if:, and the two obvious fixes for that do not work. This is the account of both incidents, the two dead ends, and the fix, including the part where we asserted something false and our own CI corrected us.

The cost problem that armed the trap

The cancellation bug was created by a cost fix. The sequence is the lesson, so it is worth telling that part first.

We run anthropics/claude-code-action as an advisory reviewer on our CRM's pull requests. As first configured, synchronize was in the trigger list, so every push to a PR billed a complete Opus review of largely the same code.

One merged PR accumulated 13 full Opus reviews. Another ran 10.

The org's API credits drained until runs began failing on their first model call with a signature worth memorizing:

{ "type": "result", "subtype": "success", "is_error": true,
  "duration_ms": 300, "num_turns": 1, "total_cost_usd": 0 }

num_turns: 1 with total_cost_usd: 0 after 300ms is the API refusing to serve. This is credit exhaustion, not a bad key.

The tuning that fixed the bill was to drop synchronize so a review runs once when the PR enters scope. We added an issue_comment trigger so any maintainer can comment @claude review for a fresh one. We also routed the model by risk: Opus on the escalation surface, Sonnet otherwise, and no model call at all on docs-only diffs.

The bill went down. And the new issue_comment trigger now shared a concurrency group with a pull_request run that was, for the first time, reliably in flight when a comment arrived.

24 seconds to cancellation

The action's first act on a PR is to post a progress comment. That comment fires issue_comment: created.

The comment run enters the same per-PR concurrency group, and cancel-in-progress: true kills the in-progress workflow run that just announced itself. The comment run then reads the comment, finds it is not @claude review, and skips.

We measured this on one of our own PRs:

Time	Event
18:23:07	pull_request run 32170704123 starts
18:23:28	claude[bot] posts "Reviewing PR #303"
18:23:31	issue_comment run 32170741106 enters the group; the review run ends cancelled; the comment run skips

The net effect was a red "cancelled" check and no review, on every new PR. Nothing errors. Nothing pages.

The reviewer simply stops existing, and the only symptom is an absence.

How GitHub Actions concurrency actually works

The semantics that matter, condensed from GitHub's own documentation, because two of them are easy to misread:

• By default, multiple workflows — and multiple runs of the same workflow — run concurrently; nothing waits for anything. A concurrency block is the opt-out: it puts a workflow run (or a single job, if set at job level) into a named concurrency group. The usual group key is ${{ github.workflow }}-${{ github.ref }}; ours is the pull request number.
• GitHub Actions ensures that only one workflow run in the same concurrency group is in progress at a time, with at most one more queued as pending. The pending run is waiting on the concurrency group until the in-progress one finishes; a previously pending run gets canceled when a newer run arrives — the queue holds a single slot for queued jobs, and order is not guaranteed.
• With cancel-in-progress: true, a new workflow run does not wait on the concurrency group: it cancels the in-progress run and takes its place. Right for CI on stale commits; a deployment pipeline usually wants the opposite — queue the new run so only one deploy to the main branch is in progress at a time, and never cancel a deploy mid-flight.
• The concurrency evaluation happens at the run level, before any job-level if: conditional is read.

Two edge cases we didn't hit, noted while we were in the documentation: a reusable workflow called with uses: runs inside the caller's run, so the caller's workflow-level concurrency governs it; and matrix jobs that share a job-level concurrency group will cancel or queue behind each other — the docs' own example keys the group by the matrix value to avoid exactly that.

The last two bullets are the whole story below: cancel-in-progress treated a bot's comment-triggered run as a supersede order, and run-level evaluation made every job-level guard useless.

Dead end one: the job-level guard

The obvious fix is a job condition: if: github.event.comment.user.login != 'claude[bot]'. It does nothing.

The reason is the GitHub Actions concurrency semantic above, and it is easy to not know. concurrency is evaluated at the run level, before any job's if: is considered. By the time a guard could refuse to run, the cancellation has already happened.

The skipped run was already skipping correctly; it cancels first anyway.

Dead end two: splitting the concurrency group by event name

The next obvious fix is to suffix the concurrency group with ${{ github.event_name }}, so pull_request and issue_comment workflow runs never share a concurrency group.

This covers only half the problem. A human's @claude review and the bot's own progress comment are both issue_comment events, so they still share a group.

The on-demand path stays broken exactly the way the automatic path was. We measured that on the same PR before backing it out.

The fix: cancel in-progress runs only on a real supersede

The condition belongs on cancel-in-progress, not on the job and not in the group name:

concurrency:
  group: claude-review-${{ github.event.pull_request.number || github.event.issue.number }}
  cancel-in-progress: ${{ github.event.comment.user.login != 'claude[bot]' }}

The group stays shared per PR, which preserves the cost guarantee that mattered from the original tuning. Two @claude review comments in quick succession still collapse to the latest, so a superseded review never runs to completion and bills for a diff nobody is waiting on.

What is no longer possible is the action's own comment cancelling the review it announces. On a pull_request event the comment context is absent, the expression is truthy, and behavior is unchanged.

The part where CI corrected us

An earlier revision of the fix's PR description claimed no rspec coverage was possible for a workflow file. That was wrong, and the suite caught it.

The cost tuning had already shipped a spec for exactly this file. The argument was that tuning which keeps a reviewer affordable lives in YAML nothing else tests, so a careless edit shows up as a bill rather than a failure. Its cancel-in-progress example failed on the first push.

We had asserted the absence of a test without looking for one.

Both dead ends are now pinned as failing examples rather than prose, so the next attempt at either dies in the suite instead of in production:

• A job-level guard cannot work, because concurrency is evaluated before if:.
• An event-name split covers only half the problem, since both halves of the on-demand path are issue_comment.

Then it happened at a client's repo, with a different bot

Days later, the same mechanic surfaced in a client's Rails repository running the same reviewer. Every pull_request review run in the sampled window was cancelled within 10 to 20 seconds of the PR opening.

The canceller there was CodeRabbit's auto-summary comment. The bot was doing exactly its job, posting a summary the moment a PR opens, and thereby entering the shared concurrency group and killing the in-flight review — whose job-level if: then skipped the intruding run, exactly as ours had.

CodeRabbit did nothing wrong, and neither did our progress comment. That is the point. Any bot that comments on PR open — an AI summarizer, a coverage reporter, a linter — is now a first-commenter, and a per-PR concurrency group with unconditional cancel-in-progress treats its comment as a supersede order.

The fix proposed for that repo takes a third shape — the group expression mirrors the job's trigger condition, so a genuine review trigger keeps the shared per-PR group while every other comment gets a throwaway per-run group and cancels nothing — and is in review as this is written. The condition-on-cancel-in-progress form above is the one we have verified in production.

What this generalises to

If your review workflow listens on both pull_request and issue_comment, shares a per-PR concurrency group, and sets cancel-in-progress: true, then the first bot to comment on a new PR cancels your review. In 2026, some bot always comments first. Remember that concurrency is the exception in Actions — everything else runs in parallel — so the one place you have declared scarcity is the one place a stray trigger can destroy work.

Three things SWARECO now checks on any repo running an AI reviewer:

1. Look for red "cancelled" review runs, not errors. This failure is silent. The review is skipped, not failed, and the PR merges without the review its own policy requires.
2. Put conditions where GitHub evaluates them. concurrency runs before if:. A guard below that line is decoration.
3. Spec the workflow file. The YAML that controls what a reviewer costs and whether it runs at all is production code that no test exercises by default. Ours failed its first honest test, which is the strongest argument for having one.

The reviewer itself, for the record, has been worth the trouble. The same run history that exposed the 13-reviews bill also shows it catching an over-broad escalation regex in its own tiering config on the first run after the fix.

But an automated reviewer only enforces anything if it runs. And for a while, ours provably didn't.