Here is a scene I've watched more than once. Six months into a program, something goes wrong in a way that feels familiar — a data migration that slips because the source system owners were never actually committed, a cutover weekend that runs long because nobody rehearsed the rollback. Someone senior says, with real frustration, "didn't we have exactly this problem on the last one?" And someone else goes and finds the lessons-learned document from the last one. It's there. It describes the problem precisely. It recommends the fix. Nobody on the current program had read it, and the few who had didn't change anything because of it.
The standard diagnosis is that the lessons were hard to find: buried in a SharePoint folder, written in inconsistent formats, locked in the heads of people who have since moved on. That diagnosis is now being answered with the obvious tool. Point a model at twenty years of closeout reports and anyone can ask, in plain language, what went wrong last time we migrated a claims system. The answer comes back in seconds, well-organised and properly sourced.
I'd like to believe that fixes it. The evidence says otherwise, and the evidence is old.
Search was the fix in 2002
When GAO looked at NASA's lessons-learned practice in 2002, it found an agency that had done most of what you'd prescribe: an agency-wide Lessons Learned Information System that managers were required to review, reinforced through training, program reviews and policy updates. Its conclusion was still that there was no assurance lessons were being applied to future missions. The barriers it named weren't technical. They were a lack of time to capture lessons and a perceived intolerance for mistakes. One of GAO's recommendations was, tellingly, an easier search capability. NASA built it.
Sixteen years later GAO reviewed the Department of Energy after the MOX facility was terminated with nearly $6 billion spent. DOE required lessons to be documented and shared. The gap GAO pointed to was further downstream: there was no requirement to evaluate whether the corrective actions taken in response to those lessons had actually worked.
Read those two findings together and the shape of the problem is plain. Capturing lessons is solvable. Finding them is solvable — and AI has now solved it more thoroughly than any search box ever could. What neither report found solved was the part after retrieval: someone deciding that this program will do a specific thing differently because of what happened on the last one, and someone checking later whether it did.
Three ways the machine makes it worse
1. Retrieval starts to feel like learning
When a lesson takes an afternoon to dig out, finding it is an event. Someone brings it to a meeting and it gets argued over. When the model surfaces twelve relevant lessons in the kickoff pack, reading them becomes a box that was ticked. The team has been informed of the history. Being informed of a lesson and being changed by it are different states, and a well-formatted summary makes the first one feel like the second.
2. Synthesis sands off the sting
The useful lessons are specific and slightly embarrassing: this vendor's estimates ran forty percent light, this business unit agreed to requirements it had no intention of honouring, we didn't escalate because the sponsor didn't want to hear it. Summarise a hundred of those across a portfolio and they converge on the same bland generalities — engage stakeholders early, validate estimates, communicate proactively. Every one is true. None of them would have changed a single decision. The model isn't hiding anything; averaging is simply what synthesis does, and the specific detail was the lesson.
3. Capture becomes free, so it stops being honest
If the lessons-learned document writes itself from meeting transcripts and the RAID log, nobody has to sit in a room and say out loud what they got wrong. That room was always the uncomfortable part, and it was the valuable part. GAO's "perception of intolerance for mistakes" doesn't go away because a machine drafts the report; it just means the report now reflects what the record shows rather than what the people know. The recorder in the room captures what was said. It can't capture what everyone agreed not to say.
A lesson isn't learned when it's written down, or when it's found. It's learned when a named person changes a specific decision because of it — and someone checks later whether the change held. AI has made the first two steps almost free. The third one was always the whole job.
What actually works
Stop treating lessons as a library and start treating them as inputs to decisions. At charter, I want the incoming program to pull the relevant history — this is where AI earns its keep, and it's genuinely good at it — and then commit, in writing, to a short list of things it will do differently. Not twelve. Three. Each one tied to a specific plan element: a rehearsal added to the cutover plan, a named commitment from the source system owner before migration starts, an estimate challenged by someone outside the vendor. If a lesson doesn't change a line in the plan, it wasn't applied, however many people read it.
Give each of those commitments an owner the way you'd give an owner to a risk, because that's what it is — a known failure mode with a mitigation. Put them on the governance agenda at the stage gate where the old failure happened last time. And at closeout, before you write this program's lessons, check last program's: did the three things we committed to actually happen, and did they work? That's the step DOE was missing, and it's the one that turns a document into a feedback loop.
For capture, keep the room. Let the machine draft the chronology, then put the leads in a room without it and ask one question: what would you tell the next program manager privately? Write down the answers in the specific, slightly embarrassing form they come out in, and resist every instinct to make them read well.
- Use AI for retrieval at charter — it's excellent at it — but require the program to commit to three specific changes, not to "review" the history.
- Tie every applied lesson to a line in the plan. If nothing in the plan changed, the lesson wasn't applied.
- Give each committed change an owner and treat it like a risk mitigation, reviewed at the gate where the failure happened last time.
- At closeout, audit the previous program's commitments before writing your own lessons. Did they happen, and did they work?
- Let the machine write the chronology. Keep the uncomfortable conversation human, and keep the specifics that make it useful.
The bottom line
Organisations have been failing to learn from their own programs for as long as they've been writing lessons-learned reports, and for most of that time they blamed the filing system. AI is about to remove that excuse completely. Every lesson will be one question away, summarised and sourced. I think that's a good thing, and I also think it will expose, quite quickly, that the filing system was never the reason. The reason was that learning costs something — a changed plan, an admitted mistake, a check that the fix held — and nobody was assigned to pay it. That's still true. The only difference is that now the lesson will be sitting in plain view while you repeat it.