Software Quality Engineering: Techniques for Quality Assurance

When there is a book about how google does something then the topic must be at least somehow relevant. Quality Assurance in Software is a broad field with many processes, best practices, guidelines, workflows and of course techniques. Since early 2018 I am allowed to be a Software Quality Engineer using those techniques which I learned by completing some of the internationally recognized and high valued ISTQB Certifications. Following, I will provide an overview of three essential Quality Assurance testing techniques used in software testing to derive effective test cases: Equivalence Class Testing, Boundary Value Analysis, and Pairwise Testing.

Equivalence Class Testing

Starting with an Example. Lets say you have a input field and a output field in your Application. The output field displays one of 4 values depending on what the value of the input field is.

Requirements:

Alphabetic input -> output is 0
Numeric input (0-100) -> output is 1
Special character input -> output is 2
Invalid input → output is ERR

This now means that you have:

26 values to display 0
101 values to display 1
about 20 (probably way more) values to display 2
and countless invalid values to display ERR

You could test every possible value in the input field to check if the expected value in the output field occurs. But I hope by now you have come to the conclusion that this is just not the proper way to proceed in Quality Assurance. Exhaustive testing (testing all and everything) is just not applicable in the real world.

Therefore Equivalence Class testing exists where you cluster values, with the same expected output or where you know that those values will be treated similarly by the software, into classes. Then you can pick one representative value from each of those classes and test if the output is indeed the expected one.

In the above mentioned example you would now have 6 classes, 3 for valid values and 3 for invalid values. Which could look like the following:

Classes	Values	Representative
Letters	{A B C … X Y Z}	A
Numbers	{0 1 2 … 98 99 100}	100
Special Characters	{$ % & … ? ] !}	&
Negative Numbers	{-1 -2 -3 … -n}	-6
Numbers Greater Than 100	{101 102 103 … n}	999
Combinations	{!1n … 6?a}	!1n

Now you have 6 representative values to use in your test cases to check all the different behaviours. I’m sure there are more classes to find but you get the idea. This procedure makes testing more efficient, more manageable, reduces testing effort and ensuring comprehensive coverage.

Boundary Value Analysis

To explain Boundary Value Analysis we continue using the example from the Equivalence Class Testing part. There we created a class for valid numbers.

Classes	Values	Representative
Numbers	{0 1 2 … 98 99 100}	100

The Boundary Value Analysis is based on the premise that defects commonly occur at the edges (or boundaries) of input value ranges. When the Requirements say that the range is from 0 to 100, but as it happened to all of us, we used a wrong or incomplete compare operator in our condition, like less than (<) instead of less than or equal (<=). Therefore it is essential in quality assurance to test such edge cases.

Usually you would test:

Values just inside the boundaries
- min value + 1
- max value -1
Values just outside the boundaries
- min value – 1
- max value + 1
Values on the boundaries
- min
- max

Using the numbers class example those values then would be:

Values just inside the boundaries → 1 and 99
Values just outside the boundaries → -1 and 101
Values on the boundaries → 0 and 100

And there you have your next 6 values which you can use in your test cases. Even though Equivalence Class Testing and Boundary Value Analysis are not required to be used together they really do complement each other perfectly.

Pairwise Testing

There are situations where you have several input parameters with several different values and you need to test as much combinations with as much values of those input parameters as necessary. The same applies for Pairwise testing as it was with the Equivalence Class technique. Testing all possible combinations of all values for all those factors (Exhaustive Testing) is time consuming, not efficient and at some point you will have tests which are obsolete because you are starting to repeat some combinations just to test everything.

Therefore Pairwise Testing is a good technique to test all possible discrete combinations of each pair for those input parameters.

IT IS EXAMPLE TIME!

You have created a Website which shows some tables, images and text in different views based on which Role is logged in. It has to be tested that the correct views are shown depending on the logged in Role and of course that those views are displayed correctly on different Devices and in different Browsers.

Following are the parameters stated which we will have to test:

Browser
Edge
Chrome
Firefox

Role
Admin
User
Guest

Device
Mobile Phone
PC

Exhaustive Testing would mean that you have to test every combination. Testing every combination is the product of the amount of values for all parameters.

So this very simple example already would produce 3x3x2 = 18 test cases.

Browser	Role	Device
Edge	Admin	Mobile Phone
Edge	User	Mobile Phone
Edge	Guest	Mobile Phone
Edge	Admin	PC
Edge	User	PC
Edge	Guest	PC
Chrome	Admin	Mobile Phone
Chrome	User	Mobile Phone
Chrome	Guest	Mobile Phone
Chrome	Admin	PC
Chrome	User	PC
Chrome	Guest	PC
Firefox	Admin	Mobile Phone
Firefox	User	Mobile Phone
Firefox	Guest	Mobile Phone
Firefox	Admin	PC
Firefox	User	PC
Firefox	Guest	PC

With Pairwise testing this can be reduced to the product of the amount of values of the 2 parameters with the most values.

Applying pairwise testing would produce 3×3 = 9 test cases.

How to construct a Pairwise testing table:

List the parameter with the most values in the first column, repeating each value for each value of the second parameter.
In the second column, list each value once for each unique value in the first column.
Distribute the third parameter values in the third column to ensure each pair is covered.

Browser	Role	Device
Edge	Admin	PC
Edge	User	Mobile Phone
Edge	Guest	PC
Chrome	Admin	Mobile Phone
Chrome	User	PC
Chrome	Guest	Mobile Phone
Firefox	Admin	PC
Firefox	User	Mobile Phone
Firefox	Guest	PC

The table now shows:

First Column: Every Browser was written down 3 times, because Role has 3 Values.
Second Column: Every Browser uses every Role once
Thirst Column: Every Role is used on every Device at least once

As said before this is a simple example. Imagine having 9 parameters with 3 values each, which would result in approximately 20.000 rows in a table when performing exhaustive testing. With Pairwise testing this would be around 81 test cases or could even be reduced down to 17 when applying a pairwise combination of parameters, ensuring that each pair of parameters is covered at least once.

But of course there are plenty of tools out there to support you in creating pairwise testing test cases when you have loads of parameters to deal with.

Summary

Equivalence Class Testing: Reduces test cases by grouping inputs that should behave similarly.
Boundary Value Analysis: Focuses on testing the edges of input ranges where bugs are more likely.
Pairwise Testing: Ensures all pairs of input parameters are tested to catch interactions causing defects.

Conclusion

Employing these techniques helps ensure thorough testing without redundancy, that you don’t artificially blow up your test case repository and that you efficiently test the System under test while improving the overall quality of the Software and the test cases.

Want to know more about Software testing and get yourself a ISTQB Certificate? Then check out the book Software Testing Foundations.