Agents in the pipeline, people at the gate
Coding assistants, CI checks and a review agent run every change. A person still approves each release. Here is how that works in a team of nine.
Rui Valente, Ines Kortmann
Over the past year our release process changed more than our tech stack. Coding assistants write a share of the first drafts, an agent reviews every pull request, and our CI runs about 1,400 checks per change. None of that ships without a person pressing the button.
The pipeline, step by step
Draft. A developer writes the change, often with an assistant in the editor. The assistant is fast at boilerplate and tests, slow at judgement.
Review agent. When the pull request opens, a review agent reads the diff, the ticket and the related files. It comments on missing tests, risky queries and inconsistent naming. It cannot approve.
Checks. Type checks, unit tests, visual snapshots and an accessibility scan run in parallel. Median run time is 6 minutes.
Gate. A second developer reads the change and the agent’s comments, then approves or sends it back. Releases go out twice a day.
What the agent catches
In three months the review agent left 2,300 comments. Developers accepted 61% of them. The most useful were missing error states and unindexed database queries; the least useful were style notes our linter already covers.
What it misses
Product sense. The agent cannot tell whether a feature solves the client’s problem, whether the copy reads well, or whether a change will confuse an editor who uses the CMS once a week. That is the job of the person at the gate.
Rules we keep
Agents never merge. Every agent comment is visible to the client. Any change that touches payments or personal data needs two human approvals. We review these rules every quarter.
The result: more changes per week, the same number of people, and no release we cannot explain.
We will report back after a quarter of use, with the same numbers as this post.
A second agent that writes the release note from the merged changes, for a person to edit. Our first tests produce notes that are accurate but flat, which is close to what we want.
What we will try next
When something breaks, the log shows who approved what and which check should have caught it. We fix the check before we fix the code.
Each client project has a shared changelog that lists every release, the person who approved it and the checks it passed. Two clients asked for agent comments to be hidden; we kept them visible and explained why. Neither has asked again.
How clients see it
Build minutes went up by 18% because checks run on every push, not only on merge. We accepted that; a failing check found on a branch is cheaper than one found after release.
Assistant licences and agent runs cost us about €38 per developer per month. Review time per pull request dropped from a median of 41 minutes to 26. The saving pays for the tools several times over, but the bigger change is that reviewers read the risky parts first.
Costs, in numbers
A typical day
A developer opens a ticket at 9:00, drafts the change with the assistant by 10:30, and opens a pull request. The review agent comments within four minutes. Checks finish at 10:42. A colleague reviews after lunch, asks for one change, and approves at 14:10. The change ships in the 16:00 release.
Before the pipeline, the same change usually waited a day for review, and the reviewer started from zero.
Where it does not fit
Data migrations and infrastructure changes skip the assistant entirely. These run from written plans that two people sign before anything executes. Speed is not the goal there.
Small content fixes skip the agent, too. A typo on a landing page does not need 1,400 checks.