Paper vs. Digital Checklists vs. ?. Pilots Asked

Checklists come in different media. Depending on the form that the checklist takes, its properties also change. Read on to find out how.

Paper vs. Digital Checklists vs. ?. Pilots Asked

Once Again on the Topic

In last July, there was already a blog post on the problems of paper checklists and about the positive side of the digital ones. Is the digital monopoly evident? What do people in aviation think? Is there another alternative? Let’s figure it out in this blog post.

Some starting thoughts

On the “So List” blog, checklists are regularly praised. Regular readers already know what a checklist is, how to create a checklist that works (with a prompt for those who subscribe!), and even the history of checklists. Please navigate to the mentioned articles if you are interested in these topics. Or use the “Checklist” tag if you are interested in even more knowledge about checklists.

As written before, even paper-based checklists have already been guests on the blog. Moreover, some “So List” readers use quite common paper forms of checklists to keep their activities in order. Look if you are interested. It’s evident in our days that we use digital tools for many activities, and purely paper processes are rare. However, digital documents have some significant drawbacks.

Let’s compare these two checklists’ means.

Benefits and Drawbacks of Paper Checklists

Benefits of Paper-Based Checklists

What is good about traditional paper checklists? Paper documents are well-known and common tools for many people. Here is a paper, here is ink on it depicting some information. Usually, you don’t need to “navigate” in any way to discover what’s necessary. This interface is very intuitive and has almost no learning curve.

You don’t need batteries for paper-based processes to work. As long as your paper hasn’t fallen apart, it is suitable for use. If you laminate your piece of paper, it will last much longer.

People on Reddit also mention that writing things down solidifies their memory. [1] Another regularly mentioned benefit is that crossing out paper records is just a pleasurable act in itself. I have heard this argument an infinite number of times, even when I didn’t ask for it.😅

At some scale, paper checklists are very cheap. For example, when you need to create a visual reminder for your children, buying even the cheapest monitor will cost you much more than printing costs.

And again, the greatest benefit of paper checklists is that they don’t have a single fault point. As long as you have a light source, a paper checklist will serve you well. Even if you are on the Moon.

So, the list of paper benefits:

  1. Almost no learning curve.
  2. Long autonomous work, no energy required.
  3. Aesthetic feelings when interacting with paper.
  4. No single point of failure.

Drawbacks of Paper-Based Checklists

In the “When Paper Fails” blog post, I mentioned checklist expert Daniel Boorman. You might know him from the “Checklist Manifesto” book [2]. In Boorman’s article “Safety Benefits of Electronic Checklists” [3], there are 15 problems that paper checklists have during flights. I won’t mention them all in this blog post and will concentrate on others.

Paper checklists' execution is error-prone. It’s good when you have checkboxes to make marks in, but occasionally, there are no checkboxes at all, and you need to keep the execution in memory. There is well-known research showing that human memory is capable of keeping 7±2 items at the same time [4].

Let’s look at the following Boeing pre-flight checklist.

Boeing 737- 700/800 pre-flight checklist [5]

It contains much more than 9 (7 + 2) items. If distracted, pilots might forget something important.

The same is true for emergency medical services. Here is an example of a medical checklist created by the World Health Organization:

Medical Emergency Checklist by World Health Organization [6]

Hardly anyone thinks of it when there is just a single checklist copy, but when it comes to the scale of giant companies, prices rise enormously.

First, printing costs come to mind. Then you need to store paper records. When checklists are updated, you need to gather the old ones, throw them away, or recycle them. We know that the manual documents’ management is error-prone if you don’t have strict procedures.

Another consequence of such manual work is that data collection and the following data analysis result in several associated costs. You have to pay them every time you want to apply analytics.

If we come back again to the individual level, a good set of checklists takes up some space. Here is the reply I received in the r/aviation Reddit community [7]:

An iPad can hold a lot more documents for a lot less weight. A single iPad replaced literal BINDERS of paper charts for us pilots, we're never going back.

So, the list of paper drawbacks:

  1. Paper execution is error-prone due to possible distractions and the characteristics of human memory.
  2. When scaled, direct costs for creating, updating, and disposing of checklists grow enormously.
  3. The analysis of checklists’ execution is costly.
  4. Even for individuals, paper checklists sometimes take up too much space.

Benefits and Drawbacks of Digital Checklists

Benefits of Digital Checklists

For many people, the natural way of working with checklists is the paper way. I have been fully digital for more than a decade, and clicking for task completion is my second nature now. I use checklists in different digital forms: importing them into my Todoist, adding them to the tasks of my colleagues, and more.

So I have to think a little more to recall the benefits of digital processes.

First, digital checklists provide significant cost savings for scaled usage. You can create, update, and delete your checklists across the whole organization in a matter of minutes.

Furthermore, analysis has become significantly cheaper. Accurate data collection happens in a matter of minutes. Some services provide you with checklist completion rates, even individual task completion rates.

In the case of aviation, electronic checklists are also capable of guiding you in a stressful situation. Here is a quote from Reddit:

If you have a CAS system, being able to just click on the failure item and go right to the right checklist saves time but more importantly stops you going to the wrong checklist when things are stressful. Also even for normal checklists there’s so ambiguity if they were accomplished or not.

Not only is the proper checklist provided, but also the progress is tracked, overcoming the limitations of pilots’ concentration.

Also, when in digital form, there is a chance to do automation. If some routine computer-related activity regularly leads to human error, delegating it to a computer is a solution.

So, the list of digital benefits:

  1. Low costs of scaling creation, updates, and disposal.
  2. Almost real-time analysis capabilities for central management.
  3. Guidance through stressful situations.
  4. Keeping track of accomplished actions.
  5. Automation possibilities.

Drawbacks of Digital Checklists

No solution, even if it is a digital solution, comes without a drawback.

First, the low cost of checklist management and analysis comes at the cost of infrastructure creation. It’s not even the hidden costs; it’s the capital cost. Without expensive equipment and a stable internet connection, there is no way to create a working digital system.

As already mentioned, there is a need for a stable internet connection. Without it, there would be no way to gather the information from digital checklists and come up with analytics in a timely manner.

Electricity is usually the single point of failure that can bring a lot of confusion. No software applications work with your computer turned off.

In simple cases, you don’t have easy access to the checklists. For example, visual reminders for children won’t work on a locked iPad.

Moreover, in simple cases, digital technology won’t save money.

So, the list of digital drawbacks:

  1. Significant infrastructure costs.
  2. A stable internet connection for instant analytics is required.
  3. Electricity is a single point of failure.
  4. Too complex for simple cases.
  5. Too costly for simple cases.

The Unsung Hero: Checklist Device

Recently on the “So List” blog, there was a blog post about vintage grocery lists. In a sense, they are a special type of checklist. It’s typically a device with signs and a mechanism for checking.

Not only can we have one for groceries, but there are also different trackers for medical activities, pet feeding, and more. You can find many variants on Etsy.

An example of a device checklist from Etsy [8]

These devices are like the expensive paper. The difference comes in the sense that they can keep track of the actions done.

Pilots also have their checklist device. And it’s costly if scaled.

Paper vs. Digital Checklists or Paper With Digital Checklists?

Through almost the whole blog post, we compared paper and digital checklists. But are they that opposite? Sometimes yes, but not in the cases where safety is critical. Let’s refer to more quotes from the Reddit discussion.

Digital checklists are nice when they're integrated into the plane and react to changing systems automatically.

Digital check lists can also be revised quickly and easily, compared to reprinting thousands of paper check lists.

However, our planes are also filled with physical placards displaying critical information all over. Exit signs, seat names, flap speeds, tail numbers. Don't discount the simplicity of something like a metal placard.

And another quote:

I prefer digital. I fly the A320 and it does not have checklists inbuilt into ECAM like the A380 and A350. However, with Airbus performance software (flysmart) you can access all normal and QRH normal/abnormal checklists digitally. We still have physical QRH and checklists for backup. Some still used them for normal checklist usage but for all abnormals almost everyone uses the software. It is easier with the software as you do not have to flip through pages.

Conclusion. We Are Not Fully Switching From Paper to Digital Checklists

Digital enthusiasts might think that the age of paper checklists is over. However, according to the analysis of benefits and drawbacks, and according to the words of practicing pilots, this is far from the truth.

Moreover, medical investigations sometimes do not show that digital checklists increase productivity [9]:

We found that using the digital checklist led to more frequent completions of the initial airway assessment task but fewer completions of ear and lower extremities exams. We did not observe any significant differences in time to task performance, but found increased compliance with the checklist. Although improvements in team performance with the digital checklist were minor, our findings are important because they showed no adverse effects as a result of the digital checklist introduction.

So it's a matter of context, a matter of your need, and a matter of costs. And also, paper is our safeguard for the most critical situations.

Was it good to compare the means of checklists? Subscribe to the “So List” blog — you won’t miss other tips about these subtle yet powerful tools. Are you already familiar with them? I doubt it — there are too many secrets they hold. Moreover, you’ll get the checklists’ creation prompt mentioned above.

[1] “Paper or Digital Checklist? : r/intj” on Reddit
[2] Atul Gawande “The Checklist Manifesto: How to Get Things Right”, ISBN 978-0312430009
[3] “Safety benefits of electronic checklists: an analysis of commercial transport accidents” on ResearchGate
[4] “The Magical Number Seven, Plus or Minus Two” from Wikipedia
[5] “B737 Checklist - UKV-PRD-B737-CHECKLIST-V2.pdf” from “Fly UK Group”
[6] “WHO Medical Emergency Checklist” from “World Health Organization”
[7] “Paper vs. digital checklists. Need your opinion : r/aviation” from Reddit
[8] Image from the “Weekly Task Reminder Slider Board/chore Chart/to Do List/new Year Resolution/ Magnetic Task Board” page on Etsy
[9] “Comparing the Effects of Paper and Digital Checklists on Team Performance in Time-Critical Work” from National Library of Medicine