So Business List. Software QA Checklist

Guest from the Software Quality Assurance World Today

Twice already, guests of the “So Business List” section were found in Telegram. Today's guest of the blog, Alexander, works as a QA engineer and also runs a community on a recently appeared social network — Setka.

Recently, Alexander created a concise post about the QA checklist and its application in the testing process to achieve software quality. As far as you understand the theme of this blog, I got captured. Moreover, the author of this blog is seriously involved in software development, so it was a match.

Alexander's work in his community isn't all about the QA checklist. There are many more quality assurance process tactics and strategies described in there to ensure comprehensive software testing practices.

QA in IT is a Whole World

Although I work in IT, I look at the profession of a quality assurance engineer with the respect of an outsider. The development industry has its division between the role of the developer and the tester. The quality assurance testing folks have their own specific language, which isn't always clear to developers.

Of course, we have a lot in common since we are working towards the same goal of the end user satisfaction. For example, bugs unite us. In Agile methodologies, we are even the parts of the same cross-functional software development teams. But there are many differences too. The comprehensive software quality assurance checklist is an established tool. In development, this is not the case yet; only a few enthusiasts use them.

What is a Checklist in the Testing Process?

In his post, Alexander uses the following definition:

A QA checklist is a list of checks that help a specialist test an application, do functional testing or accomplish some other type of testing activities.

Any software product can be tested, be it operating systems, desktop applications or website quality assurance. However, these days, website QA testing is one of the most common QA teams' jobs. It's also usually accompanied by the mobile devices' QA process, as they tend to be an important source of visitors.

Long-time readers may remember the first post. It mentioned that, like any tool, every list has a purpose. This is especially true for software testing checklist.

The main purpose of a checklist is to ensure you don't forget to check everything you planned.

A Bit About Test Cases

In QA, checklists often interact with what is called a “test case”. The International Software Testing Qualifications Board (ISTQB) defines a test case as follows [1]:

A set of preconditions, inputs, actions (where applicable), expected results and postconditions, developed based on test conditions.

Sets of test cases are grouped into test suites. That is, tests united by something in common. This organization helps the tester better navigate through numerous tests. E.g., tests aimed at usability testing or tests created for integration testing.

There are test management tools that can help to keep test cases in order. However, the post is not about them, so let's move on.

Now that the main concepts are introduced, it's time to talk about two types of checklists. Both are related to test cases in some way.

Two Types of QA Checklists

There may be more types of checklists to maintain software quality, but this article will focus on two. The first type matches a set of tests or a part of it. This checklist contains the test cases that need to be completed. You can also view this type as a tester's work plan in the sophisticated test management system.

The second type is a simplified test case. With the advent of agile methodologies, meticulous work on documenting test cases sometimes no longer justifies the effort. However, rarely someone wants to completely skip the QA part, even if not aimed at comprehensive testing. This is where checklists come in, as they do in other industries. The written before doesn't mean that there is no such thing as a well-structured QA checklist. More on this in the dedicated section.

Checklists as a Test Plan

In their classic form, these QA checklists are tables with three columns:

1. Test case titles, structured by sections/functionality, or any points defined by the author;
2. A pass/fail mark;
3. A note if needed.

Here's what the table might look like:

In the first column, “TC01” is the test case identifier. It allows you to quickly locate the test case in other documents and get additional information. “Open product page” is the title, helping you quickly understand the specific case. The ✅ or ❌ symbol indicates whether the QA engineer successfully ran the test. “Note” speaks for itself. If something important was noticed during the test, it can be recorded for future reference.

By the way, the status doesn't have to be just ✅ or ❌. If required, this binary field can be expanded to better reflect the ongoing testing phase.

Here's how the test execution checkbox can be expanded:

• Not yet tested;
• Success (✅);
• Failure (❌);
• Skipped;
• Blocked.

If a project tracks QA checklist runs in the dedicated test reports, and we want to run only the part of the quality assurance checklist, the rest can be marked as skipped.

The “Blocked” status is useful to show the impossibility of completing certain points. For example, “TC01: Open product page” didn't work. The following steps cannot be tested since they depend on the user interface presence. And it is absent.

This enhanced type of software testing checklist ensures that the details of the testing phase are not lost.

Checklist as a Simplified Test Case

In one of its formats, a test scenario contains the following information [3]:

1. Test Suite ID;
2. Test Case ID;
3. Test Case Summary;
4. Related Requirement;
5. Prerequisite tasks;
6. Test Script / Procedure;
7. Test Data;
8. Expected Result;
9. Actual Result;
10. Status;
11. Remarks;
12. Created By;
13. Date of Creation;
14. Executed By;
15. Date of Execution;
16. Test Environment.

Even just listing 15 points looks daunting. And although all the information seems useful, it may be redundant when a small team is working on a small project. In this case, a simple QA checklist is suitable.

The test scenario, previously identified as “TC04: Add product to cart”, could be described by the following checklist:

☐ Open product page.
☐ Click the “Add to cart” button.
☐ Expect the number of items in the cart to increase by 1.

As you usually don't delete the checklist right after doing the manual test, it's possible to use it for regression testing. Regression testing is crucial for ensuring that new changes do not negatively impact existing functionality. The way they are named tell us that their job is to find regression in the application.

Such checklists are a good foundation for developing automated tests. For example, in the Gherkin language of the BDD* Cucumber framework, this checklist could be transformed into the following:

Scenario: Cart counter increases when adding an item
  When User goes to Product page
  And User clicks the "Add to cart" button
  Then The cart counter increases by 1

However, it should be noted that all these steps will still need to be described in the programming language you are using, but subsequently, they can be executed with the push of a button. Moving some stable features verification to automated E2E-tests might streamline the software development process and ease keeping up to user expectations over time. The much more famous unit testing can help you ensure the details, but typically at the start you are more interested in a general picture.

Other Applications of a QA Checklists

The examples given above provide a general idea but don't show all possible ways of using checklists to ensure the quality of software products and projects.

In addition to the named, Alexander provides several other examples of QA checklist usage:

1. Verifying criteria for the start and end of testing. This could be data validation or cleanup, environment configuration and shut down.
2. Conducting checks to ensure readiness for each upcoming stage and proper completion of each completed stage.
3. A basis for exploratory testing.
4. Documenting other processes to spread knowledge within the team. QA checklist can also keep knowledge for not-so-often types of quality assurance. This could be performance testing, security testing or, gaining the attention these days, accessibility testing. Once the necessary knowledge is gained, it's wise to keep it documented to avoid costly knowledge regathering the next time.

Software Testing Checklist. Conclusion

A QA checklist is a useful tool for testers. It helps test individual application functions. And a sufficiently large set of checklists can cover a small application quite accurately.

You should not rely on a checklist as the only quality assurance process. However, it is definitely worth considering it as a foundation on the path to quality. A comprehensive QA checklist can also serve as a communicative mean, both inside and outside your team. E.g., for user acceptance testing.

If you liked this post, subscribe to this blog. I hope to delight my guests with many more collaborative materials with representatives from various professions. Please reach out to me if lists or checklists are used in your profession!

* BDD is known as Behavior-Driven Development. It's a way to first write tests using natural language and then develop the desired features. Here is some extra information from the Agile Alliance [3]:

Behaviour-Driven Development (BDD) is a synthesis and refinement of practices stemming from Test Driven Development (TDD) and Acceptance Test Driven Development (ATDD).

List of Links

[1] “Test case” from the “ISTQB Glossary” website
[2] Archived version of the “Test Case” page from the “Software Testing Fundamentals” website
[3] “What is Behavior-Driven Development (BDD)?” from the “Agile Alliance” website