Using AI Agents for Large Migrations and Codemods

AI agents are genuinely good at large migrations, and genuinely bad at being trusted with one. The mechanical breadth is exactly what a model is best at. The judgment calls, the conventions, and the question of whether the work is actually complete are not, and that is where migrations go wrong. The work that decides the outcome happens before and after the model runs.

We keep watching teams reach for agents on the wrong job first. They hand over the novel feature, where judgment matters most and verification is expensive, and they keep grinding through the 400 file framework upgrade by hand because delegating it feels too risky. That is backwards.

Why is a migration the ideal first job for a coding agent?

Three properties make this class of work unusually well suited to delegation. Framework upgrades, API deprecation sweeps, design token swaps, logging or auth standardization, moving off a library that got abandoned: they all share the same shape.

The work is repetitive, which is where a model is strongest and a human is weakest by hour four. Nobody makes their best decision on file 180 of an identical transformation. A model makes the same decision on file 180 that it made on file 3, and that consistency is the actual product.

Correctness is knowable per file. You can check the work without trusting the worker. That is a rare and valuable property, and it is exactly what a novel feature does not give you.

It parallelizes cleanly. File 200 does not depend on file 12. There is no ordering constraint to reason about, which means the wall clock is bounded by your slowest single file rather than the sum of all of them.

What goes wrong when you just point an agent at the repo?

Saying "migrate this codebase to the new API" and walking away produces a specific and predictable set of failures. We have seen all four.

Partial coverage that looks complete. This is the expensive one. The agent works through what it found, reports success, and nobody knows which files it never saw. You now have a codebase in two states with no record of the boundary, which is strictly worse than where you started, because before you at least knew nothing had been done.

Three patterns for the same change. Each run reinvents the convention. One file uses the new helper, another inlines the logic, a third writes a wrapper. All three work. Together they are a mess someone has to reconcile later, usually a person who was not there for the migration.

Silent scope creep. The agent tidies unrelated code along the way, renames a variable it found ugly, reformats a block. The real diff is now buried in noise, and the reviewer who was going to spot the one genuine mistake gives up around file 40.

Merge collisions. Parallel runs touching the same files stomp each other. This is what git worktree isolation exists to prevent, and it is the sort of unglamorous plumbing that decides whether an agent fleet works or produces garbage.

Notice that none of these are model quality failures. A better model does not fix any of them.

What does a safe migration pattern look like?

Five steps. The model only shows up in step three.

One: discover the sites with code, not with a prompt. Grep, an AST query, a compiler error list, a type checker run. Whatever produces the list, it must be deterministic and reproducible, because you are going to re-run it repeatedly to measure what remains. A model's answer to "find all the places that use X" is a sample, not an inventory. This distinction is the whole post.

Two: migrate three files yourself and write the convention down. Do the work by hand until the shape of the change is clear, then capture it. Which helper to use, how to handle the edge case where the old API was called with two arguments, what to do when the file has no test. That written artifact is the spec every agent run follows, and it is the difference between one pattern and three.

Three: one unit of work per site, isolated. One file per agent run, in its own workspace. A failure then costs you one file instead of poisoning a batch, and you can retry it independently. This is also where isolation earns its cost: if runs mutate files in parallel without it, you spend the afternoon untangling collisions.

Four: verify per site with something deterministic. Tests, a type check, a lint rule you wrote for exactly this migration. An agent reporting that it finished is not a signal, it is a sentence. If you cannot articulate the automated check that proves one file is correctly migrated, you are not ready to run the fleet yet, and figuring that out is the highest-leverage hour in the project.

Five: a human reads the first several diffs and spot checks the rest. Approval is where the judgment lives. Read the first ten closely, because that is where you catch a convention drifting. Then sample.

Where does plain code still beat the model?

More places than people expect, and knowing which is most of the skill.

Building and re-running the inventory is plain code. So is computing what remains, deduping the list, and reporting what got skipped. Every one of those is a loop over a list, and a loop over a list executed by a language model is slower, more expensive, and less reliable than the same loop written in twenty lines of Python.

A hand-written AST codemod is often the right tool for the mechanical 80 percent. It is precise, it is cheap, it runs in seconds, and it does exactly the same thing every time. The agent earns its keep on the ragged 20 percent the AST cannot express: the call site where someone wrapped the old API in a helper three layers deep, the file where the change requires understanding what the code is trying to do rather than what it says. Use both, in that order.

This is the same principle behind our agent orchestration methodology generally. The deterministic parts belong in deterministic code. The model handles the part that genuinely needs judgment, and the harness around it decides whether that judgment is trustworthy.

Does a bigger context window change the calculation?

Somewhat, and less than the announcements suggest.

Anthropic shipped Claude Opus 5 in July 2026 with a 1 million token context window, which meaningfully changes how much of a codebase an agent can hold at once. Discovery gets easier. Cross-file reasoning gets better. A convention established in one place is more likely to survive to another.

What it does not change is whether the work was verified. A model that can read your whole repository can still miss files, still invent a second pattern, still report done on something that does not compile. More context is a better starting position, not a substitute for the check at the end.

The broader industry shift through 2026 has been away from single-prompt assistance and toward long-running autonomous execution, where context management, permissions, sandboxes, audit logs, and cost control matter as much as raw model quality. Migrations are where most teams feel that shift first, because the job is simply bigger than one conversation. The named skill change is from prompt design to context engineering, and a large migration stresses exactly that.

How do you know when the migration is actually done?

Done is the inventory going to zero and the guard staying green. It is not an agent reporting success, and it is not a pull request that got approved.

Re-run the same discovery you ran in step one. If it returns nothing, the migration covered every site it knew about. If it returns eleven files, you have eleven files, which is a much better position than believing you have zero.

Then leave something behind. A lint rule or a CI check that fails on the old pattern is what keeps the migration from quietly reversing over the next two quarters as people copy from old files. A migration without a guard is a snapshot, not a change.

And say out loud what did not get migrated and why. If you capped coverage, sampled, or skipped a directory, write it down where the next person will find it. A silent cap reads as full coverage six months later, and someone will build on that assumption.

What this means for a team about to try it

Start with a migration, not a feature. Pick one with a clean verification signal, ideally one where a type check or an existing test suite already tells you whether a file is correct. That constraint will feel limiting and it is the entire reason the first attempt succeeds.

Budget the review time honestly. Reading 400 diffs is real work even when writing them was not, and teams routinely plan for the generation and forget the reading. The generation is the cheap half now.

And resist the urge to make the prompt smarter when something goes wrong. Nine times out of ten the fix is a better inventory or a stricter check, not better wording.

This is the kind of work we do at Yikes Dude, and migrations are usually where a team finds out whether their agent setup is real or theater. If you have one coming up that you cannot afford to half finish, get in touch and we will walk through how we would structure it.