AI makes you faster. But does it make you better?

New research from Anthropic has confirmed what many of us had long suspected: overreliance on AI can hinder the acquisition of new skills.

The findings are worth unpacking, not just for coders, but for anyone who uses AI regularly in their work. Including writers.

The test

We already know that AI can help people do their jobs faster. But does this come with trade-offs?

To find out, Anthropic conducted a study on 52 software engineers with experience in Python coding. In the test, participants were asked to use a Python library they were unfamiliar with to perform a relatively advanced task involving asynchronous programming.

After an initial 10-minute warm-up, they were given a 30-minute coding assignment followed by a 25-minute quiz. The quiz covered concepts they had used in the coding portion, testing for genuine mastery rather than just task completion.

The findings

Participants in the AI group finished very slightly faster but fared significantly worse in test scores for the quiz.

However, AI use didn't automatically mean a lower score. How the engineers used AI influenced how much information they retained. Here, the researchers found two distinct patterns.

Those who scored poorly tended to rely on AI to code and complete the task, asked only one or two questions before delegating all code to AI, and used AI to debug or verify code, including solving bugs.

Those who scored well took a different approach. They manually copied AI code in and asked questions. They asked AI for both code and explanations. They used AI for understanding, but solved bugs manually.

The distinction is telling. It wasn't whether participants used AI that mattered, but how deeply they engaged with it.

The concern

Two conclusions from the research write-up are worth highlighting. For one, the researchers noted that productivity benefits may come at the cost of the skills necessary to validate AI-written code, particularly if junior engineers' skill development has been stunted by using AI in the first place.

They went further, arguing that to accommodate skill development in the presence of AI, we need a more expansive view of the impacts of AI on workers. In an AI-augmented workplace, productivity gains matter, but so does the long-term development of the expertise those gains depend on.

It's also worth noting that the setup here is different from agentic coding products like the far more sophisticated Claude Code. Which means the impact could be even more pronounced than what this study found, as the researchers themselves admitted.

My take

What is clear is that how we use AI matters to our skill development. But what constitutes a good way to use AI to code or write?

Again, the problem with AI appears to be with us, the humans using it. After all, it is far more likely that junior programmers or novice writers will rely more on AI under time pressure, to the detriment of their skills.

Because who can resist?