How AI code review is changing pull requests

At work we’re relying increasingly more on coding agents to write code and open pull requests (aka PRs, or merge requests, since we’re on GitLab) and over the last few months since adoption I noticed a significant increase in PR volumes without a matching improvement in time-to-merge.

At the beginning it was clear that the team was struggling with the number of open PRs because there’s only so much a single person can review in a fixed amount of time, and the team size wasn’t changing. But after a while something clicked. The PRs started to fly through the review process, despite no changes in team or PRs size. One of the possible explanations could come from the fact that the team had started embracing AI for the code review as well. Another one was that people had simply stopped looking at the changes.

But I wasn’t satisfied with this simplistic explanation. I expected behavioral change around code reviews was happening across the industry thanks to AI, so I started looking around for information. Opinions on the various social networks are quite polarized these days so I was hoping to find a more objective source of information.

Eventually, I found something interesting.

Figures that surprised me

I came across a paper (arXiv 2607.07980, Agarwal, Miller, Kästner, Vasilescu, July 2026) that outlines the findings by four Carnegie Mellon University researchers who used an LLM-based pipeline to go through ~3,100 practitioner blog posts and Reddit threads. From those, they extracted a causal model of code review under coding agents.

What I love about this paper is that it combines LLM-automated grey-literature analysis with data coming from GitHub repositories.

The researchers relied on a neat project that I wasn’t aware of to collect data points to integrate the grey literature findings: Agents in the Wild. It’s a project that tracks all pull requests created since May 15th 2025. As part of this research they re-scraped the histories of 3000 repositories associated with coding agents between May and November 2025 and found that:

[…] agent-authored PRs are merged several times faster than human PRs, attract less review discussion in absolute terms and per line changed, and are reviewed less independently […]

This matched my anecdotal experience. But I was also surprised to discover that over 40% of agent-written PRs were reviewed only by the invokinghuman (whoever triggered the agent to make the code change) compared to 21.5% for traditional human-written code changes.

Additionally, another claim I was surely not expecting is that agent-authored PRs merged without human review fell from 50% mid-2025 to ~14% in early 2026. This is also in contrast with what Yu et al. discuss regarding the rising approval and declining scrutiny in human review of AI agent code¹.

The metrics we used to measure the health of your engineering organization are changing meaning.

As the CMU researchers clearly state in their paper, what impacts the conclusions is whether we consider the agent invoker a reviewer or not, and this is ultimately changing the meaning of the findings.

For example, from my experience, at least early on in our adoption arc, we would consider an agent to be opening a PR on behalf of the invoking developer and therefore requiring a colleague to review it, as opposed to the agent invoker themselves. According to the paper, instead, it seems common for teams in other organizations to consider PRs by coding agents as if they were stand-alone members of the team.

The metrics we used to measure the health of your engineering organization are changing meaning.

A helpful vocabulary

Even though this paper is based on grey-literature and naturally over-represents vocal early adopters, it provides five useful dimensions to help assess the code review practice. Those dimensions are:

• depth - the level of scrutiny the reviewer takes on while reviewing the proposed change;
• motivation - the reviewer’s willingness to do the job well;
• effectiveness - whether the review process correctly highlights gaps in the proposed change before they get checked into the main source code branch;
• efficiency - how many changes the reviewer is able to go through in a fixed amount of time;
• budget - how many proposed changes a reviewer can take on to review;

With those dimensions in mind let’s review some of the findings that I found helpful to better understand the impact on my team, and the industry at large.

More volume, less depth

At the beginning of this article I mentioned how I recently observed a change in volume of PRs my team had to deal with and a corresponding variation in time-to-merge.

The paper helps reason about this behavioral change by calling out how review depth is being impacted by the increasing throughput of agentic PRs.

The capacity the team has to dedicate to reviewing PRs is limited: that is what the paper calls budget. And the budget is not increasing: the team size is the same and the time spent on reviewing is capped. Therefore teams can rarely match the throughput with a sufficient review budget, leading to code reviews losing effectiveness: reviewers more easily miss defects and poor design decisions.

As we know from pre-AI literature, code review is attention-limited and cognitively demanding². An increased number of PRs doesn’t just translate into a reviewing bottleneck. Humans get exhausted from reviews and end up rubber stamping as soon as they feel drained.

Unsurprisingly, with increasing pull request load comes declining motivation. Software engineers are spending less time writing code and more time reading it and evaluating PRs. With an increasing load of pull requests they are less incentivized to do a good job per PR because they see a ton more in front of them to complete; and this ultimately impacts the depth of each review.

But there is another aspect contributing to a reduced review depth: surface plausibility. AI might produce polished, clean and idiomatic code that looks good on the surface and is therefore assumed to be safe, even when it includes design flaws or vulnerabilities. Not only that: PRs are also getting harder to review because coding agents produce excessively lengthy descriptions, with text walls of explanations in place of carefully selected and presented pieces of information. Finally, the extremely confident tone used by the agent-produced descriptions makes the reviewer lower their guard and miss important design gaps or implementation flaws.

I remember we used to be skeptical about AI-generated code early on in our adoption arc. It was a time when you would still hear the word hallucination thrown here and there (is anybody talking about model hallucinations any more these days?) and developers would be taking extra care when reviewing AI-generated changes. The paper catches that too: skepticism towards AI-generated pull requests is an effective safeguard against the excessive confidence exhibited by the tone of the change descriptions produced by the coding agents.

Anecdotally, I also see my team take a different stance towards PR reviewing depending on how the PR originated. If it came from a completely autonomous agent reacting to a signal like a monitoring alert, they would pay more attention to whether the problem real and whether the fix actually addresses it, compared to how they would scrutinize a PR originating from an agent directly invoked by one of their colleagues.

The erosion of collaboration

Agentic coding affecting collaboration

PRs are not just about gating failure. In the old days, they used to be an opportunity to solidify knowledge sharing at the end of a collaboration process that started at ideation time.

In the pre-AI world developers would collaborate throughout the development of a bug fix or a new feature. They would ideate and discuss the approach, even pair-program on the same issue (can you believe that?)! The changes coming out of this synchronous collaboration would then fly through the PR review because the knowledge about it had already mostly been consolidated within the team’s shared brain.

In my experience, reviews would very rarely identify major design and implementation flaws if all the work leading to the Pull Request had gone through a sufficient amount of synchronous collaboration.

With agentic coding becoming the norm for making code changes, and the conversational modality of that developer experience, I’m observing that the iteration of ideas and implementation considerations all happen between the invoking human and the agent (if at all: sometimes it’s just the agent that decides!).

Decisions are often taken consulting only with the agent and once those decisions are made they fall through the cracks of the terminal (or coding agent interface). They get lost in the session history.

Even when they get recorded in a shared knowledge base system, since they are not produced as part of a synchronous collaboration exercise within the team, they are not really consolidated in the team’s shared understanding. They’re just stored somewhere for somebody (likely an agent) to discover them only reactively (for example, when a critical incident occurs and the responders don’t have sufficient knowledge to know straight away what to look for to resolve it).

My take on the impact of collective knowledge is at least partially supported by one of the paper’s propositions:

P15. Low review depth and opacity increase comprehension debt, undermining review skill, maintainability, collective ownership, and knowledge transfer.

The trend I’m observing is that teams are losing the slow build up of a shared understanding of a domain and a true grasp of the design decisions behind agent-driven implementations.

Reverse-engineering intent

PRs carry a significant amount of intent that needs to be discovered for the first time at review time.

There’s another facet to the impact that this reduced synchronous collaboration is having. With agentic coding, PRs carry a significant amount of intent that needs to be discovered for the first time at review time.

From the paper it emerges that the code is increasingly opaque to the reviewer because there is no shared intent within the team about the changes that get presented for review.

The impact is significant: the review efficiency declines because it requires significant intent reconstruction effort, as opposed to just verifying that the changes match the original purpose. Effectiveness, additionally, is reduced, because the reviewer likely doesn’t have a solid grasp of the original intent behind the change and doesn’t therefore have a reference to measure against.

Finally, review motivation declines since the whole review moment becomes excessively draining. All of this then leads to an accumulating comprehension debt that increases the cognitive distance between the team and the implementation running in production.

Automating judgment

Since automating the generation of code seems to be making the code review a miserable experience for the reviewer, how about we automate review too? The paper collects opinions on that front as well.

It’s not surprising that review efficiency increases when it’s driven by AI: reviews come minutes (if not seconds) after publishing the PR and they provide mostly useful feedback across quality, security, and design. The value of the feedback is contested but there are encouraging and documented results about AI’s effectiveness at catching (planted) bugs. On the other hand, I also agree with the counterargument: AI is great at catching local issues but will easily fail to flag systemic issues in the implementation, e.g. “this will cause a race condition under load”.

Additionally, some of the AI-generated review feedback often sounds overconfident about negligible or even non-existing issues, leading to a higher effort by the agent owner to understand it and longer but unnecessary time to merge.

With AI focusing on the shape of the PR, reviewers are moving their attention to assessing the final output: does the changed software behave as I would expect? That means deep understanding of the implementation details is eroding and comprehension debt about the implementation choices accrues with every PR: the team ends up getting exposed only to the end result and less on the journey that leads to that result.

There’s an interesting consideration that’s been in the back of my head from the early days of generative AI. Are we losing our ability to reason about code if we’re not spending time doing it as much as before? According to the paper, some argue that being an effective reviewer means having deep experience with coding. But as the percentage of our time spent on coding shrinks in favor of reviewing, are we losing the skills to be able to review with depth? The paper’s propositions:

P12. Lower review depth limits growth in reviewer skills.

P13. Lower coding skills undermined by less time coding and receiving low quality reviews reduce review skills.

If that’s really the case then maybe that’s one more reason to embrace AI for code review as well, since our atrophying skills might make us the weaker reviewer.

What we should be doing, instead, is deliberately engineering how we spend our (scarce and human) attention to create the environment agents can work within.

As outlined in the paper, governance around code review is gaining attention from software teams as a way to clearly scope how review should be conducted and what should be left to the humans, while strictly defining checklists for reviewers. Aspects like security, authentication, core data structures are some of the areas many organizations are trying to keep under deliberate control while adopting AI. Some report that risk-tiered policies, review checklists and clear accountability help maintain control of the review.

Who is in control?

Many worry that the increasing reliance on AI to produce code is undermining our ability to engineer software.

My take is that our ability to write code is definitely at risk: as we just saw, reviewing code alone won’t help us keep the ability to write it and with that, our ability to review code is eroding as well.

At the same time, the act of writing code by hand is quickly losing importance. With AI capable of generating correct code at a much higher pace than humans, steering that generation becomes the critical aspect of a software engineer’s job. And I firmly believe that engineering software was never just about coding: the ability to really understand what problems to solve and how to do that sustainably has always been the most important aspect. It just so happened that code could only be produced by humans typing it.

Now that we can make machines generate it, we need to retain the ability to choose what to build and validate quickly, as opposed to just building because it’s easy.

Our job must shift towards being able to specify the right signals coding agents and autonomous coding agents should rely on to continuously improve the system and be able to react to issues quickly without the need for the individual to maintain a strong hold on implementation details.

At the same time, I don’t think this will come naturally or soon. We will learn from the flaws and gaps we have inevitably let slip through the code review and we will have to adjust. It’s always been important to curate a set of non-functional requirements that the system needs to work within to guarantee its sustainability, like build pipeline speed, time to recover, page latency, cyclomatic complexity, code coverage, vulnerability surface, dependency management, and many more. AI is making this need even more prominent.

Practice changes

As we lose our ability to review implementation details we’re going to have to focus more on specifying what a working system looks like. Agents will need to rely on those specifications to validate that the system still holds end-to-end. This doesn’t mean our job ends up being just specification authoring. I think this change poses interesting software engineering challenges like designing verification pipelines that are fit for the agent speed.

The ability to quickly spin up a cloud environment that is an anonymized copy of production so that the agent can run an extensive set of security, user experience and usability and quality tests is non-trivial at most companies: a type of technology investment that I’m sure most teams have had to neglect or defer in order to focus on features development. I can see these priorities changing faster as teams embracing AI need to adapt to be able to retain quality while increasing output speed by orders of magnitude.

We can leverage those signals to harness our agents in such a way that they can build software within the right feedback loops, so that the non-functional aspects we believe are right for the sustainability of our organization can stay in control.

We might be losing control of the minuscule decisions behind coding but we remain in control of what to build and how to deliver it to our end users.

Re-architecting our job

Reading this piece back, I realize the analysis is bleak if you’re clinging to your identity as a reviewer. I suspect that’s a common feeling among experienced practitioners right now. A lot is changing and it might feel like losing purpose. Minuscule details will become harder to care about as we lose control over implementation decisions, at least up to a point.

We need to change level of abstraction. What excites me is being able to design software systems that can truly evolve and improve autonomously. And this means being prescriptive about how to interpret the signals that now must be instrumented as the system gets built.

Many non-functional aspects used to be (or are still?) neglected by software teams because they are considered a less worthy citizen in the big picture of the business prioritization exercise. This must no longer be the case because AI is an accelerator. A mistake recorded in the context of your source code or a lack of instrumentation can lead to a flaw whose effects compound at the speed of AI. Those are aspects that we need to move our focus on.

As software engineers, we need to be deliberate about how the context impacts the agents that build our software and that means engineering the harnesses that drive them.

Our practice is moving its focus away from inspecting how the code gets written and organized by humans to designing an environment powered by signals and automation where agents can work at scale.

The paper we discussed shows how a deliberate approach to code review governance can help teams multiply their code review efficiency rather than maximising the bottleneck of human-gated approval. As we embrace a more AI-led code review practice, this should help teams reinvest their time into engineering a continuously evolving context that agents can rely on to iterate on building the right thing, efficiently.

Footnotes

1. H. Yu, L. Liu, X. Jiang, Y. Jia, S. Wang, P. Qian, and Y. Chen, “Habituation at the gate: Rising approval and declining scrutiny in human review of ai agent code,” arXiv preprint arXiv:2606.22721, 2026. ↩

2. P. W. Gonçalves, P. Rani, M.-A. Storey, D. Spinellis, and A. Bacchelli, “Code review comprehension: Reviewing strategies seen through code comprehension theories,” in 2025 IEEE/ACM 33rd International Conference on Program Comprehension (ICPC). IEEE, 2025, pp. 589–601 ↩