-
Close Search
Explorer
35 posts
You can feel it in the muscle memory.
There was a time when “being good at coding” meant you could hold a big chunk of a problem in your head, translate it into working code, and then patiently chase the weird little edge case that only appears when a user in Ohio clicks the button twice. Now a new reflex is forming: describe the goal, accept a suggestion, tweak it, run it, repeat.
That shift is not just about speed. It changes what we produce, how we verify it, what we learn while producing it, and what kind of developer we become over time.
What follows is my best attempt to answer the real question hiding underneath the hype: in what ways does AI assisted coding affect outcomes, output, cognitive development, efficiency, and long term effects, and is that net good or net bad?
Outcomes and Output: More Work Gets Done, But “Done” Moves Around
If you measure output as “how much code changes ship,” AI assistance often looks like a turbo button. In controlled settings, developers using GitHub Copilot completed a programming task substantially faster in at least one well known randomized experiment (building an HTTP server), with the Copilot group finishing about 55 percent faster.
In real organizations, the story is messier but still leans positive on throughput. A large field experiment analyzing Copilot access in professional workflows tracks outcomes like pull requests, builds, and lines of code changed, and reports evidence of meaningful increases in activity. For example, in one of the reported organizational settings, the total number of pull requests may increase by as much as 22 percent, with an 11 percent increase in lines of code changed.
At the same time, a lot of this “more” depends on what you count as the work. If you only count typing, AI wins. If you count prompting, context loading, verifying, and rewriting, the gains can shrink fast. That becomes especially visible when the task requires learning something new rather than applying something familiar.
Anthropic studied this directly by asking developers to complete tasks using a new asynchronous Python library (Trio), then measuring not just task completion but skill acquisition. In their main randomized trial, participants with AI help scored lower on a post task evaluation of library understanding, roughly a 17 percent reduction, and completion time was not meaningfully faster on average.
So AI can increase output, but it can also move effort from writing to supervising. You may ship more code, yet spend more of your limited brain budget deciding whether the code is correct, idiomatic, secure, and maintainable.
Efficiency: Faster on Familiar Ground, Not a Shortcut to Mastery
The most consistent efficiency uplift shows up when AI is helping you do work you already understand, or at least understand well enough to evaluate. That is why benefits often skew toward routine tasks, boilerplate, glue code, and common frameworks.
When the work is actually skill formation, the productivity picture changes. In the Trio experiment, the AI group frequently spent significant time interacting with the assistant, sometimes asking many questions, which helps explain why speed gains were not guaranteed.
This is the first big lesson that tends to surprise people: AI assistance is not automatically a time machine. It is a trade. It can reduce keystrokes, but it can increase the amount of judgment you must apply per line of code, because now you are reviewing a proposal rather than executing a plan you built yourself.
And there is a second lesson buried in the same research: the way you use the assistant matters more than the fact that you used it. In Anthropic’s work, some interaction patterns preserved learning outcomes better than others, particularly approaches that kept the developer cognitively engaged, like asking conceptual questions or requesting explanations alongside generated code.
Put differently, delegation is efficient in the short term, but it is often expensive in the long term.
Quality: The Review Load and the “Verification Gap”
The moment you accept that code can be generated faster than it can be understood, a new bottleneck appears: verification.
A 2026 developer survey report from SonarSource suggests AI assisted coding is now routine for many developers, with a large share using AI tools daily, and developers estimating a substantial portion of committed code is AI generated or significantly assisted.
Sonar’s accompanying communications also highlight a “verification gap”: many developers say they do not fully trust AI generated code, yet a smaller share always verify it before committing, and a meaningful slice report that reviewing AI output can take more effort than reviewing a colleague’s code.
This is where quality risk sneaks in. Not because AI is “bad at code” in some global sense, but because humans underinvest in the boring part when the output looks confident. That creates a path for bugs that are plausible, consistent, and wrong.
Independent analysis of pull requests can make this more concrete. A CodeRabbit report comparing AI authored versus human authored PRs found AI PRs had about 1.7 times more issues per PR on average, with more problems in logic and correctness, readability, error handling, and in some categories, security.
This does not mean AI always produces worse code than humans. It means that at scale, the review burden often increases, and when it increases, teams either pay it with time or pay it later with incidents.
Security: Overconfidence Is the Real Vulnerability
Security is where the “looks right” problem is most expensive.
A well cited user study on AI code assistants found participants with access to an AI assistant wrote significantly less secure code on most tasks, and they were also more likely to believe they had written secure code.
That combination is dangerous. If the tool makes you both faster and more confident, it can also make you faster at shipping the wrong thing.
On the model side, security performance is also uneven. Veracode reports testing many models across multiple languages and finding a large share of generated samples failed security tests, with common vulnerabilities like cross site scripting showing up frequently in relevant tasks.
Policy focused analysis from Georgetown’s CSET frames the risk landscape in three categories: insecure outputs, model vulnerability to attack or manipulation, and downstream systemic impacts like feedback loops that can propagate insecure patterns.
Taken together, these sources point to a simple conclusion: AI assistance does not remove the need for secure engineering, it increases the need for it, because it increases the volume of code that can be produced with a false feeling of correctness.
Cognition: From Doing the Work to Supervising the Work
Now to the part that makes people uncomfortable, because it is not about a tool anymore, it is about the shape of your thinking.
Microsoft researchers surveying knowledge workers found a consistent theme: using generative tools can reduce the perceived effort of critical thinking while shifting effort toward verification and integration, and higher confidence in the tool is associated with less enacted critical thinking.
Even if you disagree with parts of that framing, it matches what many developers are already noticing in practice. When you are generating rather than composing, you spend less time planning in your head and more time judging on screen. Your cognition shifts from construction to curation.
The Anthropic skill formation experiments make the same shift measurable in a coding specific setting. When developers relied heavily on AI to complete unfamiliar tasks, they showed weaker conceptual understanding, code reading, and debugging ability on follow up evaluation.
Debugging deserves special attention, because it is where learning often happens. In the Trio study, the control group encountered more errors during task completion, and that struggle is plausibly part of why they learned the library better. AI assistance can route around that struggle by handing you a working solution, but you also lose the deep feedback loop that teaches you why the solution works.
If you have ever learned a framework by breaking it repeatedly until it finally made sense, you already understand the risk. Remove the breaking, and you may remove the understanding.
Long Term Effects: A Fork in the Road, Not a Single Destination
It is tempting to ask, “Does AI make developers better or worse?” The evidence suggests the answer is: it makes developers different.
Used one way, AI assistance becomes scaffolding. You ask conceptual questions, request explanations, compare alternatives, and treat the assistant as a fast pair programmer who never gets tired. This style can preserve learning while improving throughput, and it fits with the interaction patterns that correlated with higher skill outcomes in the Anthropic work.
Used another way, AI assistance becomes cognitive offloading. You delegate the thinking, accept what looks plausible, and pay only enough attention to make it compile. That style can produce short term productivity for certain tasks, but it can erode the very skills you need to supervise automated output, especially debugging and code comprehension.
There is also a motivation dimension. In large scale experiments of human and generative AI collaboration outside of coding, researchers have found immediate performance gains that did not persist once the tool was removed, along with signals that motivation and engagement can suffer when people feel less ownership of the work. This matters for developers because curiosity and stubbornness are not just personality traits, they are part of the job.
My determination, based on the research above, is that AI assisted coding is net positive for outcomes when the developer remains responsible for understanding and verification, and net negative for cognitive development when it replaces the struggle that builds fluency. In other words, it is not a free lunch, it is a reallocation of effort from writing to judgment.
The organizations and individuals who win will be the ones who treat verification as a first class activity and treat learning as something worth protecting, even when the assistant can hand you a working answer in five seconds.
Sources
Anthropic, “AI assistance and coding skills” research summary.
Shen and Tamkin, “How AI Impacts Skill Formation” (arXiv, 2026).
Peng et al., “The Impact of AI on Developer Productivity: Evidence from GitHub Copilot” (arXiv, 2023).
“The Productivity Effects of Generative AI: Evidence from a Field Experiment with GitHub Copilot” (MIT / PubPub release).
SonarSource, “State of Code Developer Survey report 2026” (PDF).
CodeRabbit, “State of AI vs Human Code Generation Report.”
Pearce et al., “Do Users Write More Insecure Code with AI Assistants?” (arXiv HTML version).
Veracode, “2025 GenAI Code Security Report” blog summary and topline results.
CSET (Georgetown), “Cybersecurity Risks of AI Generated Code” (PDF, Issue Brief, 2024).
Lee et al., “The Impact of Generative AI on Critical Thinking” (CHI 2025 paper PDF).
You can feel it in the muscle memory.
There was a time when “being good at coding” meant you could hold a big chunk of a problem in your head, translate it into working code, and then patiently chase the weird little edge case that only appears when a user in Ohio clicks the button twice. Now a new reflex is forming: describe the goal, accept a suggestion, tweak it, run it, repeat.
That shift is not just about speed. It changes what we produce, how we verify it, what we learn while producing it, and what kind of developer we become over time.
What follows is my best attempt to answer the real question hiding underneath the hype: in what ways does AI assisted coding affect outcomes, output, cognitive development, efficiency, and long term effects, and is that net good or net bad?
Outcomes and Output: More Work Gets Done, But “Done” Moves Around
If you measure output as “how much code changes ship,” AI assistance often looks like a turbo button. In controlled settings, developers using GitHub Copilot completed a programming task substantially faster in at least one well known randomized experiment (building an HTTP server), with the Copilot group finishing about 55 percent faster.
In real organizations, the story is messier but still leans positive on throughput. A large field experiment analyzing Copilot access in professional workflows tracks outcomes like pull requests, builds, and lines of code changed, and reports evidence of meaningful increases in activity. For example, in one of the reported organizational settings, the total number of pull requests may increase by as much as 22 percent, with an 11 percent increase in lines of code changed.
At the same time, a lot of this “more” depends on what you count as the work. If you only count typing, AI wins. If you count prompting, context loading, verifying, and rewriting, the gains can shrink fast. That becomes especially visible when the task requires learning something new rather than applying something familiar.
Anthropic studied this directly by asking developers to complete tasks using a new asynchronous Python library (Trio), then measuring not just task completion but skill acquisition. In their main randomized trial, participants with AI help scored lower on a post task evaluation of library understanding, roughly a 17 percent reduction, and completion time was not meaningfully faster on average.
So AI can increase output, but it can also move effort from writing to supervising. You may ship more code, yet spend more of your limited brain budget deciding whether the code is correct, idiomatic, secure, and maintainable.
Efficiency: Faster on Familiar Ground, Not a Shortcut to Mastery
The most consistent efficiency uplift shows up when AI is helping you do work you already understand, or at least understand well enough to evaluate. That is why benefits often skew toward routine tasks, boilerplate, glue code, and common frameworks.
When the work is actually skill formation, the productivity picture changes. In the Trio experiment, the AI group frequently spent significant time interacting with the assistant, sometimes asking many questions, which helps explain why speed gains were not guaranteed.
This is the first big lesson that tends to surprise people: AI assistance is not automatically a time machine. It is a trade. It can reduce keystrokes, but it can increase the amount of judgment you must apply per line of code, because now you are reviewing a proposal rather than executing a plan you built yourself.
And there is a second lesson buried in the same research: the way you use the assistant matters more than the fact that you used it. In Anthropic’s work, some interaction patterns preserved learning outcomes better than others, particularly approaches that kept the developer cognitively engaged, like asking conceptual questions or requesting explanations alongside generated code.
Put differently, delegation is efficient in the short term, but it is often expensive in the long term.
Quality: The Review Load and the “Verification Gap”
The moment you accept that code can be generated faster than it can be understood, a new bottleneck appears: verification.
A 2026 developer survey report from SonarSource suggests AI assisted coding is now routine for many developers, with a large share using AI tools daily, and developers estimating a substantial portion of committed code is AI generated or significantly assisted.
Sonar’s accompanying communications also highlight a “verification gap”: many developers say they do not fully trust AI generated code, yet a smaller share always verify it before committing, and a meaningful slice report that reviewing AI output can take more effort than reviewing a colleague’s code.
This is where quality risk sneaks in. Not because AI is “bad at code” in some global sense, but because humans underinvest in the boring part when the output looks confident. That creates a path for bugs that are plausible, consistent, and wrong.
Independent analysis of pull requests can make this more concrete. A CodeRabbit report comparing AI authored versus human authored PRs found AI PRs had about 1.7 times more issues per PR on average, with more problems in logic and correctness, readability, error handling, and in some categories, security.
This does not mean AI always produces worse code than humans. It means that at scale, the review burden often increases, and when it increases, teams either pay it with time or pay it later with incidents.
Security: Overconfidence Is the Real Vulnerability
Security is where the “looks right” problem is most expensive.
A well cited user study on AI code assistants found participants with access to an AI assistant wrote significantly less secure code on most tasks, and they were also more likely to believe they had written secure code.
That combination is dangerous. If the tool makes you both faster and more confident, it can also make you faster at shipping the wrong thing.
On the model side, security performance is also uneven. Veracode reports testing many models across multiple languages and finding a large share of generated samples failed security tests, with common vulnerabilities like cross site scripting showing up frequently in relevant tasks.
Policy focused analysis from Georgetown’s CSET frames the risk landscape in three categories: insecure outputs, model vulnerability to attack or manipulation, and downstream systemic impacts like feedback loops that can propagate insecure patterns.
Taken together, these sources point to a simple conclusion: AI assistance does not remove the need for secure engineering, it increases the need for it, because it increases the volume of code that can be produced with a false feeling of correctness.
Cognition: From Doing the Work to Supervising the Work
Now to the part that makes people uncomfortable, because it is not about a tool anymore, it is about the shape of your thinking.
Microsoft researchers surveying knowledge workers found a consistent theme: using generative tools can reduce the perceived effort of critical thinking while shifting effort toward verification and integration, and higher confidence in the tool is associated with less enacted critical thinking.
Even if you disagree with parts of that framing, it matches what many developers are already noticing in practice. When you are generating rather than composing, you spend less time planning in your head and more time judging on screen. Your cognition shifts from construction to curation.
The Anthropic skill formation experiments make the same shift measurable in a coding specific setting. When developers relied heavily on AI to complete unfamiliar tasks, they showed weaker conceptual understanding, code reading, and debugging ability on follow up evaluation.
Debugging deserves special attention, because it is where learning often happens. In the Trio study, the control group encountered more errors during task completion, and that struggle is plausibly part of why they learned the library better. AI assistance can route around that struggle by handing you a working solution, but you also lose the deep feedback loop that teaches you why the solution works.
If you have ever learned a framework by breaking it repeatedly until it finally made sense, you already understand the risk. Remove the breaking, and you may remove the understanding.
Long Term Effects: A Fork in the Road, Not a Single Destination
It is tempting to ask, “Does AI make developers better or worse?” The evidence suggests the answer is: it makes developers different.
Used one way, AI assistance becomes scaffolding. You ask conceptual questions, request explanations, compare alternatives, and treat the assistant as a fast pair programmer who never gets tired. This style can preserve learning while improving throughput, and it fits with the interaction patterns that correlated with higher skill outcomes in the Anthropic work.
Used another way, AI assistance becomes cognitive offloading. You delegate the thinking, accept what looks plausible, and pay only enough attention to make it compile. That style can produce short term productivity for certain tasks, but it can erode the very skills you need to supervise automated output, especially debugging and code comprehension.
There is also a motivation dimension. In large scale experiments of human and generative AI collaboration outside of coding, researchers have found immediate performance gains that did not persist once the tool was removed, along with signals that motivation and engagement can suffer when people feel less ownership of the work. This matters for developers because curiosity and stubbornness are not just personality traits, they are part of the job.
My determination, based on the research above, is that AI assisted coding is net positive for outcomes when the developer remains responsible for understanding and verification, and net negative for cognitive development when it replaces the struggle that builds fluency. In other words, it is not a free lunch, it is a reallocation of effort from writing to judgment.
The organizations and individuals who win will be the ones who treat verification as a first class activity and treat learning as something worth protecting, even when the assistant can hand you a working answer in five seconds.
Sources
Anthropic, “AI assistance and coding skills” research summary.
Shen and Tamkin, “How AI Impacts Skill Formation” (arXiv, 2026).
Peng et al., “The Impact of AI on Developer Productivity: Evidence from GitHub Copilot” (arXiv, 2023).
“The Productivity Effects of Generative AI: Evidence from a Field Experiment with GitHub Copilot” (MIT / PubPub release).
SonarSource, “State of Code Developer Survey report 2026” (PDF).
CodeRabbit, “State of AI vs Human Code Generation Report.”
Pearce et al., “Do Users Write More Insecure Code with AI Assistants?” (arXiv HTML version).
Veracode, “2025 GenAI Code Security Report” blog summary and topline results.
CSET (Georgetown), “Cybersecurity Risks of AI Generated Code” (PDF, Issue Brief, 2024).
Lee et al., “The Impact of Generative AI on Critical Thinking” (CHI 2025 paper PDF).
You must be logged in to post a comment.
- Most Recent
- Most Relevant
