Engineers try AI twice, it doesn't work, they declare it useless. I see this constantly.
The tool is fine. The missing skill is closing the feedback loop.
What Closing the Loop Means
Iteratively steering the conversation until you get what you need. Not accepting early failures as final.
Initial Prompt -> AI Response -> Evaluate Output
|
Not Quite Right <- Identify Gap <- Refine Prompt
| |
--------> Steer Conversation --------
|
Close Enough -> Accept and Iterate -> Good Enough? -> Success
| Open Loop (Fails) | Closed Loop (Succeeds) |
|---|---|
| Accepts first/second attempt as final | Iterates until outcome achieved |
| Blames the tool | Adjusts approach based on feedback |
| Gives up after 2-3 tries | Persists through 5-10+ iterations |
| Treats AI as magic black box | Understands AI needs steering |
| Vague, single-shot prompts | Specific, conversational refinement |
IMPORTANT
The Mental Model Shift Most engineers treat AI like a search engine. Expect a correct answer from one query. AI is closer to pair programming. Needs back-and-forth to refine shared understanding.
My Approach: The 30-40 Minute Yap Session
I spend 30-40 minutes talking to an AI to work through problems. That time investment is the core skill.
The steering process:
- Set context - problem space, constraints, desired outcome
- First attempt - let it generate something
- Evaluate - what's right, wrong, missing
- Steer - specific feedback on gaps
- Refine - AI adjusts
- Repeat - until output meets requirements
- Validate - check contradictions, edge cases, missed requirements
Real Example: Playbook YAML Generation
I used this to generate a platform architecture timeline. AI with MCP tools searched Slack, Confluence, and Jira directly, then generated YAML with timestamped events.
I told it: "When you create these files, capture myths, gaps, and contradictions."
The first attempt is usually wrong in multiple ways. Events in the wrong order. Decisions attributed to the wrong people. Gaps where nothing happened but politics. Meetings that were cancelled. Similarly-named projects confused. A mess.
Most people stop there. "AI can't do this kind of research." But the first attempt is never the final attempt. The next thirty minutes are steering: "This event is out of order - check the ticket dates." "That's the wrong person - search for the other one." "There's a gap between March and June. Search Slack for what happened during that period."
By the seventh iteration, the timeline is accurate. By the ninth, it includes context you'd forgotten - a thread where the team discussed an approach and rejected it, which explains why the architecture looks the way it does. The final output isn't just correct. It's useful in ways you hadn't anticipated.
Total time: 40 minutes. Manual research would have taken a day. The AI doesn't replace judgement. It amplifies the ability to apply that judgement across a large information space.
What makes it work: real data from multiple sources. Contradiction detection flags when sources disagree. Iterative refinement means keeping at it until it's accurate.
Why Engineers Fail to Close the Loop
The Persistence Gap
Most engineers drop off after 2 attempts. I persist through 5-10+.
| Attempt | Typical Accuracy | Most Engineers | Me |
|---|---|---|---|
| 1 | ~60% | Continue | Continue |
| 2 | ~75% | Continue | Continue |
| 3 | ~85% | Give up | Continue |
| 4-5 | 90-95% | Abandoned | Continue |
| 6-10 | 98%+ | Never reached | Success |
The Feedback Quality Problem
Most engineers say "this is wrong" without saying what or why.
Bad example: "This doesn't work."
Good example: "This contradicts the Slack message from 15 Jan where we decided TrueFoundry over LiteLLM. Search Slack for that decision and update the timeline. Also check for other contradictions between Slack and Confluence."
Specificity is everything. Diagnose the gap, don't just flag the symptom.
The Pressure Paradox
Sprint pressure discourages 30-40 minute sessions. But spending 30 minutes upfront saves hours later. Feels like a luxury when deadlines are tight.
The paradox is real. When you're busiest, you have the least time to invest in the skill that would save you time. But that's exactly when the payoff is highest. A production issue needs root cause analysis by Friday. The temptation is to just write it yourself - you know the system, two hours max. Or spend forty minutes with an agent: setting context, iterating, steering. The final output often catches edge cases you'd forgotten, structures the timeline more clearly, and takes forty minutes instead of two hours.
Teaching Loop-Closing At Scale
The goal is to make it teachable and repeatable.
Steering Language
| Situation | Steering Phrase |
|---|---|
| Output too vague | "Be more specific about X. Include concrete examples." |
| Missing context | "You're missing context about Y. Here's what you need to know…" |
| Contradictory info | "This contradicts Z. Search [source] and reconcile." |
| Wrong direction | "Stop. Let's try a different approach. Instead of X, try Y." |
| Almost there | "Good, but refine this part: [specific section]. Here's what I need…" |
How to Teach It
Demonstrate. Run real-time sessions showing first attempt failing, then steering through 5-10 iterations. Show what changed at each step.
Create safe practice. "Office hours" where engineers bring real problems and work through them with someone who knows how to close the loop.
Document case studies. Full arc: initial problem, first attempt, steering conversation, final output, time saved. Example: "How I generated 3 months of platform timeline in 40 minutes."
Teaching loop-closing works best through demonstration. Show the full arc: first attempt failing, steering through iterations, final success. Some adopt it immediately. Some take weeks to build the habit. Some never get it - they still treat AI like a search engine, expecting correct answers from single queries.
The difference isn't intelligence or technical skill. It's patience. The engineers who succeed are willing to be wrong five times before they're right. The ones who don't give up after the second attempt, every time. Loop-closing is a patience skill as much as a technical one.
What Worked in Practice
I've taught this pattern to groups of engineers through a structured enablement programme. The most effective format was hands-on workshops where participants brought their own problems and worked through them in real time.
One pattern emerged consistently: the participants who struggled most were those who expected a "correct" way to prompt. They wanted the magic phrase that would make AI work perfectly. The ones who succeeded fastest were those who treated it as a conversation - trying, failing, adjusting, trying again.
The programme proved something important: loop-closing can be taught, but not through documentation alone. It requires live demonstration, safe practice environments, and repeated exposure to the full arc of failure → steering → success. After running multiple sessions, we found that showing the full feedback loop in real-time - first attempt through iterative steering to final output - was more effective than any slide deck.
The metric that mattered wasn't how many people attended. It was how many kept using the tools after the session ended. The ones who internalised loop-closing stayed. The ones who didn't went back to manual methods within a week.
TIP
Cultural Narrative Shift Change from ❌ "I tried AI and it didn't work" to ✅ "I iterated 8 times and got exactly what I needed." Celebrate persistence as the skill.
Measuring Success
Signs Someone Has Learned It
- Continue past 3 attempts without frustration
- Provide detailed feedback on what's wrong and why
- Know when to steer vs restart
- Willingly invest 30-40 minutes
- Trust the process when early attempts fail
- Can explain their steering approach to others
Signs Someone Hasn't
- Give up early: "I tried twice, it doesn't work"
- Give vague feedback: "This is wrong" without specifics
- Blame the tool: "AI is useless for this"
- Expect perfect results in 5 minutes
- Revert to manual methods to avoid iteration
Summary
- The real AI skill is loop-closing, not prompting. Iteration and steering beat clever one-shot prompts.
- Most engineers give up too early. The 4th-5th attempt is where quality jumps from 75% to 95%.
- Specific feedback beats vague complaints. Diagnose the gap.
- 30-40 minutes upfront saves hours later. Expensive-feeling investment with compound returns.
- Organisations need to protect iteration time. Sprint pressure kills the skill before it develops.
What's Next
Next time you hear "I tried AI and it didn't work," ask how many iterations they ran. If the answer is 2-3, the problem isn't the tool. It's the loop.