When Paper Fails

In the book “The Checklist Manifesto” [1] in Chapter 6 “The Checklist Factory”, the story revolves around Boeing and specifically Daniel Boorman, a veteran pilot with twenty years of experience in compiling checklists for pilots. Today, this period has already surpassed 30 years.

According to information from the ResearchGate website, Daniel is the author of 27 articles [2]. He stopped working for Boeing in 2014, so he is not connected to the latest unpleasant news. His articles are actively cited. It is pleasing to note that his latest article was published last month, which means that one of the world's leading experts on checklists is still active.

In this blog, there has already been an article describing paper and its limitations. But it is one thing when a small blog writes about it, and another thing when a large corporation does. Trust in its research is much higher.

But why would a large and recognized corporation suddenly pay attention to such trifles as the interaction of checklist users with the paper containing it? The answer to this question is quite obvious. Interaction with paper becomes important when lives depend on this interaction.

Let’s return to Boorman and his work from 2001. It is interesting that Daniel did not just study checklists and their varieties. He also promoted new forms of aviation checklists at the time: electronic checklists (ECL). In particular, the article we will quote today is called “Safety benefits of electronic checklists” [3].

In the mentioned article, Boorman lists various types of errors that occur when crews interact with paper checklists. There are two types of checklists in the operation of aircraft: normal and non-normal. Normal checklists are used for ordinary preparations for routine operations on the aircraft. Non-normal checklists are needed in case of various situations that are not considered routine.

Let’s look at an example of a Boeing normal checklist [4]:

What stands out here? I noticed the lack of a place for marking the completion of a particular action. A clicking checklist for small aviation now makes much more sense. The lack of a place for marks does not go unnoticed and is reflected in the list of errors made when interacting with checklists.

The list of errors in the article is based on the analysis of incidents and accidents, literature reviews, and observations of the training process.

Interaction with both normal and abnormal checklists is fraught with the following errors:

• One or more items skipped in checklist
• Place lost in checklist when crew distracted by higher priority task or checklist
• Incorrect switch selected
• Item incorrectly confirmed complete
• Excessive psychomotor workload due to holding, turning/marking pages, recovering dropped or misplaced paper checklist
• Checklist unreadable due to poor illumination

Interaction with normal checklists can include the following problems:

• Normal checklist skipped (subsequent checklist accomplished before critical flight phase)
• Normal checklist omitted (all checklists related to critical flight phase are omitted)

Problems arising from interaction with abnormal checklists:

• Incorrect non-normal checklist accomplished for the annunciated condition
• Non-normal checklist skipped or left incomplete
• Incorrect steps accomplished in a branching checklist
• Steps to be accomplished later in flight not accomplished
• Operational notes or revised limitations following a malfunction forgotten
• Wrong steps accomplished when multiple related failures have conflicting actions
• Omitted non-normal checklist or other errors due to excessive cognitive workload in multiple failure case

Flight safety requirements are probably some of the strictest demands humanity places on anything. Aviation systems do not forgive mistakes as many other systems do. We are unlikely to be satisfied if, in case of errors, management elegantly scolds the guilty. It is better if there are no mistakes at all. And for this, it is necessary to work hard and search for deeply hidden problems that are not visible under usual circumstances.

References:
[1] Atul Gawande “The Checklist Manifesto: How to Get Things Right”, ISBN 978-0312430009
[2] “Daniel J. Boorman” on ResearchGate
[3] “Safety benefits of electronic checklists: an analysis of commercial transport accidents” on ResearchGate
[4] “Boeing 737-700/800 - Normal Checklist” on Fly UK