Anthropic published a study on how AI impacts skill formation. Within days, both AI skeptics and AI evangelists claimed it proved their point. It does neither.
The finding is more specific than the headlines. It tells us something about how junior devs learn unfamiliar tools. It doesn't settle whether AI makes engineers worse.
What They Actually Measured
52 mostly junior developers learned Trio (a Python async library they'd never used), with and without an AI assistant. Then they took a quiz.
| Measure | AI Group | No-AI Group | Theo's Take |
|---|---|---|---|
| Completion time | Slightly faster | Slightly slower | Too noisy to care |
| Quiz score | 50% | 67% | That's a real learning gap |
| Biggest weakness | Debugging | Less severe | The safety concern that matters |
| Task context | New library, short window | Same | Not how real devs work |
The AI group finished about two minutes faster - not statistically significant. But the quiz gap was nearly two letter grades.
The biggest drop was debugging ability. That's the part that should worry us. Human oversight still matters, and debugging is where humans catch what AI misses.
The debugging gap shows up in real work too. You describe what you want, review the output, ship it. The code works. Tests pass. Three weeks later something breaks in production - an edge case the agent hadn't considered - and you find yourself staring at error traces without a complete mental model. The agent built it, but you didn't build the understanding of how it works.
The response isn't to avoid agents. It's to change how you use them. Ask for explanations before implementation. Make them walk through control flow. If they use a pattern you don't recognise, stop and ask why. The conversation takes longer. The output is better. And when things break, you can actually fix them.
The study measures learning a new library in 30-35 minutes. That's not your daily workflow with Claude Code or Cursor.
Why the Speed Headline Is Weak
Theo's pushback is methodological. Participants were told to work fast, in a browser they'd never used, while knowing a quiz was coming. Time spent learning the interface and framing prompts eats any code-generation gain.
Sean Goedecke adds another layer: many AI users in the study retyped generated code by hand instead of copy-pasting or using agent workflows. So the study is partly about novice tool use, not the ceiling of AI-assisted development.
Familiarity with the environment and workflow is a huge missing variable. Theo says agents compress hours or days of work for him. A 30-minute browser task doesn't capture that.
How You Use AI Matters More Than Whether You Use It
The useful part of the paper is the behaviour breakdown. Heavy delegation and AI-led debugging correlated with the worst quiz scores. Asking for explanations and writing code manually preserved more learning.
The pattern shows up everywhere. One engineer treats the agent like a senior dev to delegate to. "Build me a user authentication flow." Working code ships. The engineer moves on. Three months later, they can describe what authentication is but struggle to debug why a session isn't persisting across subdomains.
Another engineer treats the agent like a tutor. "Explain how JWT tokens work. Now show me how to implement refresh tokens. Now explain why we need a blacklist for revoked tokens." Initial velocity is slower. But three months later, they can trace a token through the entire lifecycle, identify why an edge case fails, explain the security trade-offs. Same tool. Different posture. Different outcomes.
| Pattern | What It Looks Like | Speed | Learning |
|---|---|---|---|
| AI delegation | "Do most of it for me" | Fastest | Worst |
| Progressive reliance | Start manual, hand over more | Moderate | Poor |
| Iterative AI debugging | "Fix this without explaining" | Slow | Poor |
| Generate then understand | Get code, then interrogate it | Moderate | Strong |
| Hybrid explanation | Code + explanation together | Moderate | Strong |
| Conceptual inquiry | Ask about concepts, write manually | Slowest | Strong |
The study encouraged speed, which probably discouraged the slower, more educational habits. Ask follow-ups. Request explanations. Validate understanding before moving on.
The practical line: using AI to skip thinking versus using AI to sharpen thinking.
This distinction isn't just about learning. It's about long-term capability. The engineer who delegates everything might ship faster this quarter. But the engineer who engages deeply will ship faster next year, because she'll have built the intuition that lets her spot problems before they become bugs. She'll know which corners can be cut and which can't. She'll have the taste that tells her when something is subtly wrong, even when all the tests pass.
That kind of judgement doesn't come from delegation. It comes from struggle.
The Skateboard Analogy
Learning hard things means feeling stupid for long periods. Many people quit before they hit the first rewarding milestone.
AI can help here - not by replacing learning, but by getting someone to an early win that keeps them in the game.
Beginner starts coding
-> Feels stuck
-> Gets early progress?
-> No -> Motivation drops -> Quits
-> Yes with guided AI -> Builds something -> Confidence rises -> Keeps going
Better tooling doesn't automatically make better developers. But it can keep more people in the game long enough to build real skill. That's especially relevant for newcomers who might quit before their first "I can actually do this" moment.
Summary
- The study is narrow. New library, time pressure, browser environment. Not real-world delivery.
- Speed evidence is weak. 30 minutes in a browser isn't your local agent workflow.
- Learning gap is real. Debugging drops most. Human oversight still required.
- Cognitive engagement is the key. Explain concepts, don't just delegate.
- AI can prevent dropout. Right posture is moderation, not abstinence or full delegation.
What's Next
If you're learning: use AI to explain concepts, unblock yourself, and get small wins. Don't use it to skip building debugging intuition.
If you're experienced: this study says almost nothing about your workflow. Keep using agents where they excel, keep your judgement engaged where it matters.