An audit is a snapshot. It checks, on a fixed date, whether the documentation meets requirements and whether the procedures are described as they need to be described. What it does not check is whether the organisation sees its own weak signals and learns from near misses. The distinction sounds academic but is operationally consequential. It decides whether an organisation builds its safety on the audit date or in between.
There is no need for polemic against audits. Audits do what they should. The serious question is what they structurally cannot do. And what needs to stand beside them so that what happens between audits does not become a blind spot for the organisation.
What audits actually measure
Audits measure conformity within a defined observation window. They check whether documentation at the time of inspection shows what it must show, and whether procedures are described as they must be described. This is a legitimate and non-trivial task. It has its place in the trust landscape that complex societies require. No one would board an airline aircraft without certification, no hospital operates without accreditation, no industrial operator runs without a regulatory framework. Audits produce this trust through a social procedure whose function Michael Power described precisely back in the late nineties in The Audit Society: they are “rituals of verification”, not measuring instruments for the property they claim to test. They produce a legible picture of order, and this picture is compatible with insurance, law and corporate reporting.
Power’s point is not that audits are useless. It is that what they produce is not identical to what they appear to produce. A passed audit says that what was documented at the time of inspection met the requirements. It says nothing about whether the organisation sees the weak signals of its own practice, whether it learns from near misses, whether its adaptive capacity holds under real pressure. These properties are not documentable in the form an audit requires for its findings. They are processes, not states, and an audit is built to test states.
Anyone who does not draw this distinction builds a conception of safety in which compliance and safety are the same thing. Before the Texas City explosion in 2005, BP had a “lost time injury” rate below the industry average, and an audit result confirming that number. What was deteriorating simultaneously in the same plant was process safety: a domain not captured by the prevailing KPIs. Andrew Hopkins described this in Failure to Learn with a clarity that still hurts: the organisation managed what it could measure, and screened out what evaded measurement. The audit confirmed the measurement.
Audits check whether what is supposed to be documented is documented. They do not check whether the organisation sees what it should be seeing.
Audit preparation becomes a permanent task
From this structural property follows a second, which becomes visible in many organisations as soon as you look at how effort is distributed across the year. Audit preparation has become a permanent task, with its own resources, its own roles, its own quarterly rhythms. An internal compliance function that works all year toward a smooth external inspection. Pre-audits, mock-audits, action lists, “gap analyses” meant to anticipate the result. This isn’t dumb bureaucracy, but the understandable response to an audit regime that has grown denser, more formalised and more consequential over the past twenty years.
The difficulty in this rationality arises where it displaces attention. Attention is finite, line time is finite, and what pays off in an audit competes for these exact resources with what does not appear in an audit. Erik Hollnagel describes this effect in Safety-II in Practice as a systematic reinforcement of Work-as-Imagined: the denser the specification, the more energy the organisation puts into maintaining the world of specification. The attention it spends there is missing from the observation of Work-as-Done. The gap between the two grows widest precisely where the most is documented, because the documentation creates its own reality that needs maintenance.
The displacement happens without bad intent. It follows the mechanics of prioritisation under time pressure: an operational signal that isn’t audit-relevant gets moved to the back, and “to the back” means, on the annual rhythm, until the next quarter, when the same logic will apply again. At the moment of prioritisation, the shift is materially correct. Over time, it is a pattern: what doesn’t fit the audit form rarely comes back on the table.
Concretely: in November, a shift lead reports an anomaly at a measurement point that doesn’t appear on any audit checklist. Her supervisor files it as “forward to Q1”. The Q4 audit runs cleanly. In Q1, preparation for the next audit cycle is already underway, the anomaly sits on a list nobody opens any more. In April, a different shift at the same measurement point has a near miss that is connected to the original report. Nobody sees the connection any more.
To see this pattern, you have to step outside the audit logic. From within its sorting, every individual shift looks like clean work.
What needs to stand beside it
Abolishing audits is neither possible nor sensible. They have their function, they are built into the logics of insurance and regulation, they produce the trust a division-of-labour economy requires. What they cannot deliver has to be delivered alongside. Alongside, not instead.
That isn’t an elegant answer. It demands a second layer that belongs in normal operations and works on a different logic from the audit layer. This second layer has a name in the resilience and HOP literature: Operational Learning. It is not the collection of Lessons Learned from incident reports. It is the ongoing reconciliation of the picture of work with what actually happens in operations, before it becomes an event.
In Pre-Accident Investigations, Todd Conklin develops two tools designed exactly for this second layer. The first are Learning Teams: small, time-limited groups of operating personnel and a facilitator, who sit down for one to two hours after a near miss or a routine task. Their job is not to find a solution. It is to reconstruct what was actually done and compare it with what should have been done. The output is an observation, not an action item. Precisely this refusal of the action-item format is the condition under which the observation becomes sharp. Whoever starts looking for solutions immediately stops seeing.
The second are Pre-Job Briefs. These are not the formal safety briefings everyone knows, but short structured conversations at the beginning of a non-routine task: what could go wrong. What trigger means we abort. Who has authority in which situation. The output is a shared mental model, not a list. A well-run Pre-Job Brief practice is hard to show in an audit, because it leaves no paper trail. It is effective in daily safety because it brings what can go wrong into the conversation before the doing.
Both tools share a principle. They are oriented toward seeing, not toward steering. The audit layer steers what is visibly documented. The learning layer makes visible what doesn’t reach the steering. Steven Shorrock and Claire Williams, in their work on human factors in practice, call this the “professional curiosity” of a learning organisation: the willingness to keep reconciling one’s own picture of work with the actual work. This is not a one-off project. It is an ongoing practice.
The distinction from what many organisations report as “Lessons Learned” is important. Lessons Learned are an output format: what we take from a closed incident, formulated as an action item or insight, filable in a system, citable in the next audit. Operational Learning in Conklin’s sense is not an output but an ongoing mode in which the organisation continuously refines its picture of work. One closes something. The other holds something open.
One example shows the difference most sharply. A near miss in a control room: an operator categorises an alarm differently from how the designers intended. In the Lessons-Learned format, this becomes an action item. “Refine alarm labelling, retrain operator.” Done, ticked off, filable in the system. In the Operational-Learning format, it becomes an observation. “Operator under load X reads the alarm in the context of other signals differently from how the designer assumed. The pattern recurs under conditions Y. What we don’t know is which contextual cues drive the interpretation.” A question instead of a result. Rather than closing the investigation, it keeps it open.
Whoever forces Operational Learning into the Lessons-Learned format has just lost the concept.
Why this is harder than it sounds
The audit logic and the learning logic compete for the same resource: the attention of the line. The audit logic almost always wins this competition because its consequences are short-term and visible. A failed audit pulls inquiries, reports, justifications upward. A skipped Learning Team session pulls nothing. It simply doesn’t happen, and nobody notices, until an event happens whose connection with the omitted learning work can no longer be cleanly shown.
On top of that: Operational Learning has no pretty KPI. An organisation can count the number of Learning Teams conducted, but the moment it does, it starts fulfilling the format rather than using it. Charles Goodhart described this effect in 1975 for economic regulation: once a metric becomes the target of control, it loses the property that made it a good metric. This makes the learning layer awkward in upward reporting, and awkwardness is a scarce property in organisations under efficiency pressure.
Whoever wants to build this second layer accepts that it isn’t reportable in the same language as the audit layer. It requires a leadership willing to release time and protected spaces without immediately demanding an impact measurement. It requires a line that recognises observation as a legitimate activity in its own right, not merely as a means to an action-item end. This is the more demanding form of safety work, and it is precisely the kind that happens between the audits, or doesn’t.
Where the audit belongs
Audits will stay because they are functional. What is missing beside them is the second layer: an ongoing learning practice that doesn’t replace the audit but takes on the safety work the audit structurally cannot do. Only with this second layer does the audit become what it should be: a confirmation of the state the organisation knows. Not the principal source of its safety knowledge.
Sources
- Michael Power – The Audit Society: Rituals of Verification, Oxford University Press 1997
- Todd Conklin – Pre-Accident Investigations: An Introduction to Organizational Safety, Ashgate 2012
- Erik Hollnagel – Safety-II in Practice: Developing the Resilience Potentials, Routledge 2018
- Andrew Hopkins – Failure to Learn: The BP Texas City Refinery Disaster, CCH Australia 2008
- Steven Shorrock & Claire Williams – Human Factors and Ergonomics in Practice, CRC Press 2017