Walls and rails for agentic coding

Hi, I'm Orr. I run SWARECO.

We build and maintain software for a handful of companies. That means Rails on the back end, React on the web, and React Native on mobile - plus our own product alongside that.

AI changed how we work, but not in the way most posts claim.

It didn't mainly make us faster at writing code. It moved where the hard part is. The hard part used to be building the thing. Now, it's knowing whether what you were just told is true.

So over the last year, we built a framework for it. This isn't about prompts. It's a set of walls and rails that every piece of work goes through, no matter who or what is doing the work.

Walls are non-negotiable. They stop the work. No ticket, no start. No test, no merge.

Rails don't stop anything. They shape how the agent behaves, and they're what actually cuts hallucination.

Most of the value is in the rails. Most of the safety is in the walls.

You need both, and they do different jobs.

This is the whole thing.

The setup

Seven pieces. The rules file is the centre and everything else reads it.

• CLAUDE.md - the contract: conventions, invariants, corrections. Lives in the repo root.
• Ticket skills - one per project, so a ticket matches that project's house style and its real constraints. In ~/.claude/skills/.
• Component library - the UI source of truth. Every component and variant rendered, so markup gets reused rather than invented. Storybook on React and React Native, Lookbook on Rails.
• MCP servers - the evidence layer: error tracker, issue tracker, Slack, analytics. Wired into the session.
• Review action - reads CLAUDE.md and reviews every PR, advisory only. In .github/workflows/.
• Recurring audit - a graded pass over security, coverage and duplication that reconciles against the last run. .claude/commands/audit.md.
• CI - the actual gate: tests, linter, security scan, dependency audit. Unchanged from before any of this.

How they relate matters more than the list.

CI is the only thing that can stop a merge. It was the gate before AI and it still is.

The review action reads the same rules file a session does. It enforces a written contract, not opinions about style.

The skills exist because the facts a project needs are per project and expensive to rediscover.

The component library is the same idea for UI - one place that says what already exists, so nothing gets invented. And the MCP servers are there so that checking a claim costs one call instead of twenty minutes.

One thing about the shape: this is per repo, not per company.

We have eight CLAUDE.md files across Rails, React and React Native. The walls in them are identical - the same working contract, copied.

Everything below that is stack-specific. The part worth writing is the part that doesn't transfer.

The longest one isn't the Rails app. It's a React front end at 410 lines.

Our biggest Rails one went from 33 lines to 252 across ten commits in 28 days.

Every jump is attached to a ticket, so the file explains itself in git.

The walls

These are in the working contract at the top of CLAUDE.md. They're binary. Work that hasn't cleared them isn't done, and saying it's done anyway is itself a violation.

Every unit of work is tracked by a ticket. No ticket = the work isn't ready to start. If none exists, create one that states the goal and scope.

One unit of work = one branch off fresh main. Never stack unrelated changes, never commit to main. When two pieces of work touch the same file, serialize them - finish and merge one before starting the next.

Every change ships with a test that exercises the real behaviour. A bug fix ships with one that fails before the fix and passes after. Never a skip or pending placeholder - a spec file where every example is skipped is worse than no file, because it reads as covered when it isn't. Not covered = not done.

Green before merge, no exceptions. The whole command, not the part you remembered:

bundle exec rspec && rubocop . && bundle exec brakeman --no-pager -q \
  && bundle exec bundler-audit check --update

If something is red or was skipped, say so plainly with the output - don't report it as done.

That's the Rails command. The React and React Native repos have their own, and which one it is isn't the point - the wall is that all of it goes green, including the parts you didn't touch.

That last sentence in the quote is the wall, more than the command is.

An agent that runs the suite, sees a failure, fixes something unrelated and reports success has broken the rule.

The command is checkable. The reporting is the part that needs a rule.

Every review finding is resolved before merging. Fixed in a follow-up commit, or answered in a reply saying why it doesn't apply. Silence is not an answer.

Rejecting a finding you have actually checked is fine and often correct - a reviewer can be wrong, and CI passing is not a rebuttal to a real bug. What is not allowed is merging past a comment nobody looked at.

The domain invariants hold after every change. Ours are for a multi-tenant app: every controller action goes through the authorization layer, and every lookup is scoped through the current tenant.

An unscoped finder is a cross-tenant leak even when an authorization check is present, because the record loads before it is authorized.

Yours will be different. The point is that they're written down as walls, not as things everyone knows.

The rails

Walls stop bad work from shipping.

Rails stop the agent from being confidently wrong in the first place. That is the failure mode that actually costs you.

Here's the most valuable line in our rules file:

This repo HAS JS-driven system specs. Do not claim otherwise.

That repo has 204 spec files. 29 of them are browser system specs running against headless Chromium through Playwright.

Before that line existed, a fresh session would look at the app, not find the setup where it expected it, and decide client-side flows couldn't be tested.

It would then propose a request spec for a JavaScript bug. It was articulate about it. It was wrong.

That's what you're defending against. Not an agent that writes bad code - one that tells you something false in a completely reasonable tone. The rails all attack that.

Rail 1 - state what exists, not just what to do. About half our rules file isn't instruction at all. It's facts about the codebase.

These facts are written down at the moment a session assumed otherwise, so the assumption can't be made twice.

It's the same with the ticket skills. Most of a 402-line skill is a table like this.

| Project        | team-managed (next-gen)                                |
| Issue types    | Task (the default), Bug, Epic                          |
| Components     | Not available on this project - never pass components  |
| Story points   | No such field exists. This project does not estimate.  |
| Assignee       | Always unset. Single-maintainer project.               |

"This project does not estimate. Don't invent one" is there because an agent filing a ticket will happily fill in a field the project doesn't have.

The API call fails, and you lose four minutes working out why.

Every row in that table is a failed API call that happened once.

Rail 2 - for UI, that means one browsable source of truth. Most "the page doesn't match the design" bugs come from someone hand-writing markup instead of reusing what exists, and an agent does that more readily than a person does.

Ask for a detail page and you get a card layout that looks completely reasonable and matches nothing else in the app.

This is why the component library is in the setup rather than being a nice-to-have. Storybook on the React and React Native side, ViewComponent + Lookbook on Rails - Storybook for Rails, effectively - with every component and every variant rendered rather than described.

The rule in the file is one line:

Prefer an existing component over hand-written markup or a raw partial.

The word that matters there is rendered. A written list of components is just another document to hallucinate against - the agent reads "we have a badge component", infers an API for it, and produces something that compiles and is wrong.

A library that renders every variant is checkable the same way a test is: the thing either appears or it doesn't. Point a session at it before it writes markup and the design stops being a matter of recollection.

The small specific facts matter as much as the principle. Ours says form-field components live under Ui::Form::, never top-level Form, because Form is already an ActiveRecord model - exactly the kind of thing an agent guesses wrong once and then keeps guessing wrong.

Rail 3 - verify before asserting, and record the correction.

Do not make claims about what this codebase does or doesn't have from assumption. Check the relevant file first, then state it. When a claim turns out to be wrong, correct it and record the corrected fact here so it isn't repeated.

That last clause is what turns the rules file into a correction log instead of a style guide. It's the highest-leverage habit in the whole setup.

Most people write a rules file once, at the start, describing how they want things done.

The useful version is written continuously, afterwards, recording what went wrong.

Rail 4 - don't claim a number you didn't measure. This one sits under a heading called "Honesty rules" in every ticket skill:

Don't claim a grade, a coverage number, or an event count you didn't measure. Flag everything you inferred so it can be checked. Security specifics come from reading the code, not from memory.

Coverage is the one that catches people. Local coverage artifacts are git-ignored, so they hold whatever the last single-file run wrote, and a single-file run reports coverage of the whole app while exercising a fraction of it.

The figure it yields can be out by a factor of several and still look completely authoritative. So the rule is: report coverage qualitatively, or from a full-suite run whose command and date you record, and otherwise say "not measured" - which is strictly better than a number that looks solid and isn't.

Rail 5 - evidence is file:line and a real snippet. Never a code dump, never invented code. A ticket that can't tie a finding to a specific line doesn't get filed. And when the cause isn't verified, the words are "suspected cause" plus what would confirm it - a confident wrong diagnosis sends the next session down the wrong path.

Rail 6 - make checking cheap, or the honesty rules are unenforceable. This is what the MCP servers are for. They're not a productivity feature here, they're the evidence layer.

Our production performance work is a whole epic whose children came out of real error-tracker issues, not a guess about what might be slow.

The ticket template has a slot for it:

**Sentry issues:** SWARE-CRM-B (40), F (23), …

Event counts, in the ticket, from the tool. And it closes on the other end - putting Fixes <issue-id> in the commit body means the tracker resolves the issue itself when the PR merges. The tool that reported the problem is the one that confirms it went away.

Slack is where I expected to write "we pull context from the thread". Our rule is close to the opposite:

When a ticket depends on specific wording - user-facing copy, notification text, error and empty-state strings, enum or label values - put that wording in the ticket, verbatim. Do not leave it behind a link to a design file, a wiki page, or a Slack thread. A link is a pointer that rots, moves, or isn't accessible to whoever picks the ticket up.

So we read the thread and transcribe it. The ticket becomes the source of truth and the thread becomes disposable.

Using the integration to add a link would be the failure, not the feature.

Chain of custody

The rails only help if the trail survives. One identifier threads all of it, and each artifact points at the next.

error-tracker issue -> ticket -> branch -> PR title -> squash commit -> issue auto-resolved
and back again: close-out comment -> ticket marked Done

The ticket key names the branch, prefixes the PR title, and so becomes the squash commit, because GitHub appends the PR number on merge. 174 of our last 259 merged PR titles carry a key. The other 85 are archaeology from before we had the convention.

Then the close-out comment, which is what makes a Done ticket auditable:

PR: <link>

<One sentence on what changed.> <bullets: per-file detail, with the
identifiers and why the obvious fix was wrong>

**Scope note - <what was carved out> to <ticket>.** <why.>

"Why the obvious fix was wrong" is the part I'd defend hardest.

Six months later nobody asks what changed, the diff says that. They ask why it wasn't done the simpler way, and that answer lives in one person's head unless a rule forces it into the ticket.

Enforcement: a reviewer that reads the same file

Every pull request gets reviewed by Claude through a GitHub Action. It's advisory and never fails the build.

A reviewer that can block a merge becomes something you learn to route around. One that only comments stays worth reading.

We ran CodeRabbit before this and dropped it. Not because it was bad at reading a diff - it wasn't - but because it reviewed our code against general good practice.

What we needed was our code reviewed against our contract.

A generic reviewer produces generic findings. Every PR gets notes, the notes are mostly reasonable and mostly not about anything that matters here, and a review that always has notes stops being read. Once nobody reads it, an automated reviewer is worse than none, because the green tick still implies somebody looked.

Its prompt starts by telling it to read CLAUDE.md, because the file is the contract and the review is enforcement of it, not a second opinion about style. Then it weights the review toward this repo's known failure modes rather than general code quality. It ends with:

If the diff is clean, say that plainly in one line rather than manufacturing findings - a review that always has notes stops being read.

That's a rail pointed at the reviewer itself. An AI reviewer asked for findings will produce findings.

The wall that admits when it isn't there

This is the one I'd take even if you take nothing else.

The Claude action refuses to run when the workflow file on the branch differs from the copy on the default branch. That guard is right - otherwise a PR could rewrite the reviewer to leak secrets and have CI run it.

The problem is it refuses and reports success.

A green check that reviewed nothing looks exactly like a green check that found nothing.

That's the whole failure mode of this setup in one place - an unverified thing wearing the costume of a verified one. So the workflow has a second step that re-derives the condition from the repo contents and comments on the PR when it fires:

The check above is green because the job succeeded, not because the code was reviewed. Nothing here has been looked at.

It fired on the PR that introduced it, which is the right kind of proof.

Fifteen lines, and it's the only reason we can tell a review that found nothing from a review that never happened.

Generalise it: every automated step in your pipeline should be able to say it didn't run. A tool that skips its work and reports green turns an unreviewed diff into a reviewed-looking one, which is worse than having no tool.

Grading ourselves on a schedule

The reviewer only ever sees one diff. It can't see drift - the thing where every individual change was fine and the codebase still got worse. So there's a second command, /audit, that runs periodically over three pillars: security, test coverage, and duplication.

It fans out parallel subagents by dimension, each one reading code directly and returning findings with file:line evidence, then grades each pillar A to F and gives an overall grade.

The grade matters more than the findings list, and that's the part worth stealing.

A grade is a thing you can move. A list of findings is a thing you can add to forever without ever knowing whether you're winning.

Findings become tickets under three standing epics; the grade is what tells us whether the next pass is genuinely better or we just wrote more tickets.

The real payoff isn't the first run, it's the second. The command reconciles against the last one rather than rediscovering everything:

For each open ticket, check whether the code now satisfies it, and mark it likely fixed (with evidence) or still open. Call out any regressions - a fixed pattern that came back - and any new findings not matching an existing ticket.

A one-off audit is easy and mostly useless. It produces a list nobody can tell apart from last quarter's list. Reconciliation is what turns it into a measurement.

And the rails apply to the audit itself, harder than anywhere else - it's the one command whose entire output is claims about the codebase. That coverage-percentage rule from Rail 4 lives in this command, in capitals, because an audit is exactly where a confident wrong number does the most damage.

Does it work

Claims about tooling should come with evidence. The reviewer went live and reviewed two PRs in the hours after, about three minutes each.

On a fix for an undecryptable API key prefix it raised three findings. The rescue covered the decryption error but not the configuration error, and a missing credential is a different failure from a wrong one. The fix changed a second page's behaviour and that path had no spec, which it flagged by quoting our own rule back at us. And a judgment note that the error reporting was per-row with no ceiling, so under the PR's own second hypothesis every row would emit an event on every render.

We fixed two. We rejected the third on the record with reasoning - the per-row events ARE the outstanding diagnostic, and suppressing them throws away the signal the ticket exists to gather. That's the finding wall working in both directions.

On the next PR, an N+1 fix with a regression spec, it found the thing I would not have found.

The regression spec didn't exercise the regression. The fix eager-loaded an avatar attachment and its blob, but our user factory never attaches an avatar, so the helper took its else branch and rendered a default image on every row. The spec passed. It would have passed with the fix reverted.

It also flagged that the sibling controller has the same N+1 and wasn't fixed, which is a rail firing on its own without anyone pointing it at it. The rule it applied is one line in the file: where two models are built to mirror each other, a change to one almost always needs the other, so check the sibling before calling it done.

Not everything worked. That second review errored out partway through and still posted its findings. The tooling is days old and it shows.

What it costs, and what it doesn't fix

The speed is the side effect, not the point, but people ask. On our own product the last 259 merged pull requests have a median open-to-merge time of 187.5 minutes, 39% merge in under half an hour, and on one day in August we merged 53 pull requests inside a 15.7 hour window.

The 75th percentile is about six days, and the distribution is bimodal rather than centred: work that fits inside one session merges in minutes, work that doesn't waits until it gets picked up again as a fresh unit.

That's a scoping property, not a tooling one, and no wall or rail changes it.

The lever is how the work is cut - a unit small enough to finish in one pass is a unit that merges the same day.

The other thing worth saying is that none of this is setup you do once. Every rule in these files earned its place by catching something. Rules written in advance, before anything has tested them, are the ones nobody follows.

If you're building your own

1. Separate the walls from the rails. Walls are binary and few - ticket, branch, test, green, findings resolved. Rails are many and specific, and they're where hallucination actually gets cut.
2. Write down what exists, not just what to do. Half a good rules file is plain statements of what the codebase contains, written down at the moment a session assumed otherwise.
3. Give UI a browsable source of truth. A component library the agent can render - Storybook, Lookbook, whatever your stack has - plus a written rule that an existing component beats hand-written markup. Rendered, not listed: a document describing your components is one more thing to hallucinate against.
4. Write the rules file continuously, not upfront. It's a log of corrections, and the corrections are the valuable part.
5. Make every automated step able to say it didn't run.
6. Give the automation a written contract to enforce, not taste. "Review this PR" gets you style opinions. "Review this PR against the file that defines good here" got us the sibling-controller catch.
7. Wire in the tools that make claims cheap to check. "Don't claim a number you didn't measure" is only enforceable when measuring is one call away.
8. Let the reviewer be wrong. Findings you have to answer but are allowed to reject stay useful. Findings you have to obey become something to route around.

None of this makes the agent trustworthy. It makes it checkable, and that's the only property that survives contact with production.

Our CLAUDE.md skeleton, the ticket-skill template, the audit command and the review workflow are all at github.com/SWARECO/dotclaude.