How to choose an AI coding tool
Pick a bug your team already fixed. You know the expected behavior, which test should fail, and how small the patch can be.
Where to start
Start with the part of your last fix that took longest. If it was finding the caller, test how the tool traces code. If review took longer than writing, check whether it can keep a patch within scope. Leave your debugger and CI checks in place.
What to check
Choose a known failure
Give both candidates the same starting commit and one reproducible regression.
Set boundaries
Use a test branch and limit access to the repositories and tools the task requires.
Count review time
Record time spent checking callers, correcting the patch, and explaining it to a reviewer.
Budget for a heavy week
Find out which model calls or background jobs add cost beyond the seat.
Run a trial
Fix a regression without widening the diff
Use the same repository and failing test. Keep the original fix out of both sessions.
- Ask for the relevant files before permitting edits.
- Run the existing tests and inspect every changed file.
- Ask a teammate who did not see the prompts whether they would merge it.
What a good result looks like
The fix passes and the reviewer can explain the patch without the chat transcript.
Common mistakes
- Judging a tool only on a new repo with no conventions.
- Granting broad access before testing a narrow task.
- Counting generation time and ignoring cleanup.