Cowboy sorts every new pull request as it arrives, so you know where to look first. It looks back at the ones that merged, to see what went wrong and whether the sorting, or any rule of yours, would have caught it. And it shows which of your team's habits pay off in your repo, then turns all of that into instructions your coding agents follow. The screens here come from the demo that ships with Cowboy, which runs on a made-up repo, except where a caption says otherwise. All of it is free. This is a preview release: the last section says what's tested and what isn't yet.
Every new pull request lands in easy, normal or careful by fixed rules, with the reasons written on it, and agent work is tagged. Treat the pens as a starting point: the next part shows how Cowboy checks them on your repo.
One page shows every open pull request, sorted into three pens, with agent work tagged. It updates itself as Cowboy works.
9 pull requests in the corral
Check for straysSkim it. Small, safe, checks green.
A normal review.
Slow down. A code owner should look.
Click any card to open its page. Everything Cowboy knows about that pull request is in one place.
| Lines changed | 5 (3 in source) |
| Blast radius | 39 files import session.ts |
| Sensitive paths | auth/** |
| Checks | Passing |
| Code owners | @maria |
On GitHub, Cowboy only adds labels and one comment, edited in place on every push, and asks the code owners of careful pull requests for a review. GitLab (gitlab.com or your own), Forgejo and Gitea work the same way and are in preview.
Sensible defaults out of the box. Change them per repo with one small file on your default branch. A pull request can't loosen its own rules.
The rule book · acme/app
{
"agents": { "codex-bot": "codex" },
"careful": {
"paths": ["auth/**", "billing/**",
"migrations/**"]
},
"detect": { "label_at": 80 }
}
This is the part that earns trust. Cowboy follows every merged pull request, traces trouble back to the one that caused it, and checks whether the pens, or simpler rules, would have pointed at it. In our own tests the careful pen didn't beat simpler rules, and Cowboy will tell you the same if it's true on your repo.
Cowboy follows every merged pull request for 30 days and ties what happened back to it: reverted, broke main, needed a fix, or caused an incident.
After merge
Careful pull requests had trouble 1.6× as often as average. That's not enough to say the rules beat a random pick.
297 rated · 12 had trouble · 1 waiting for you
Waiting for you (1)
#512 Incident? Checkout errors: shipped in v2.3; the errors are in files it changed (src/pay/api.ts)
Yes No
| Pen | Had trouble |
|---|---|
| Easy | 3 of 140 (2%, likely 1–6%) |
| Normal | 5 of 96 (5%, likely 2–12%) |
| Careful | 4 of 61 (7%, likely 3–16%) |
Example numbers.
A rule that flags a quarter of your pull requests will catch some trouble by luck. So Cowboy gives the same share of pull requests to simple alternatives and compares.
Do your rules earn their place?
Same share of pull requests for each · share of later-reverted ones caught
Real numbers, not the demo: Cowboy's own rules, replayed on next.js, Airflow and Kibana history.
How often pull requests here change tests with code, avoid shortcuts, stay a reviewable size, get reviewed and merge with checks green. And whether each habit went with less trouble after merge, in this repo.
Habits
vercel/next.js · 544 merged pull requests in the last 90 days
238 of 405 · 53% the 90 days before
332 of 405 · 85% the 90 days before
Real numbers from the next.js history in the demo. “Did worse here” is a question to look into, not a reason to write fewer tests.
Coding agents follow the instructions file in your repo. Cowboy writes a section for it from what it already knows, so agents open pull requests your team can review quickly.
<!-- cowboy:guide start --> ## Working in vercel/next.js ### What gets a closer review - Changes under `auth/**`, `migrations/**`, `payments/**` are sent for careful review. Keep them small. - Pull requests of 50 lines or fewer, in 2 folders or fewer, get the quickest review. One concern per PR. ### Don't take these shortcuts - Don't silence warnings or type errors to get checks passing. Fix the cause, or say why the warning is wrong. - Don't skip or delete tests to get a change through. ### This team's habits (last 90 days) - **Tests with code**: 59% of pull requests here do this. - **No shortcuts**: 82% of pull requests here do this. ### Say when an agent helped - Add a Co-authored-by trailer naming the agent. <!-- cowboy:guide end -->
Shortened. The full section lists every careful path and shortcut check.
Which agents send the most careful work, cut corners, or leave strays behind. And how often your team moved a pull request to another pen.
The tally
| Author | Pull requests | Easy | Normal | Careful | Doubles | Strays | Shortcuts | Moved by a person |
|---|---|---|---|---|---|---|---|---|
| copilot | 3 | 1 | 1 | 1 | 1 | 0 | 1 | 0 |
| claude-code | 2 | 1 | 0 | 1 | 0 | 0 | 1 | 0 |
| jules | 1 | 1 | 0 | 0 | 0 | 0 | 0 | 0 |
| codex | 1 | 0 | 0 | 1 | 0 | 0 | 1 | 0 |
| devin | 1 | 0 | 1 | 0 | 0 | 0 | 0 | 0 |
| cursor | 1 | 0 | 1 | 0 | 0 | 1 | 0 | 0 |
Webhooks get lost. Every 5 minutes a catch-up sweep finds any pull request Cowboy missed, and the Activity page lists everything it did, newest first.
Activity
Run cowboy and your browser opens on a setup page. No config files, no environment variables.
One click if you use GitHub’s command-line tool. Otherwise the page opens GitHub’s token form already filled in with read-only access: tick your repos, generate, paste.
Connect GitHub
Choose from the repos your login can see. Cowboy sorts their open pull requests and rates the last 90 days of merged ones.
Pick repos
Nothing is written to GitHub until you switch labelling on, and Cowboy checks it has permission before it does.
What Cowboy does
cowboy■ Team server? A GitHub App, set up from the same page
Cowboy is a preview release. Here's what has been tested on real code, what works but is still in preview, and what isn't built yet.
Tested on real code
Works, in preview
cowboy, connect with your GitHub CLI login or a token, pick repos, and switch labelling on when ready. Tested against a stand-in for GitHub, not yet github.com.Not built yet
All of it. Whatever's free today stays free.