Daniel Chelliah - What is Software Testing?: ISTQB Foundation Companion and Study Guide

What is Software Testing?

2012 iLoveTesting.com, Daniel Chelliah

Notice: This document is provided in confidentiality to you and is not to be distributed to anyone else unless permission is granted by Daniel Chelliah to do so.

Author(s): Daniel Chelliah, David Ross;

Editor: Sharon Campbell

B efore we look at what is testing, let's have a think about why we test? If you've ever used a really ancient computer then you have an intuitive sense of why software testing is necessary. As you grit your teeth through another reboot, you start to wonder why none of the people who made this hunk of junk thought to test something simple like installing a printer before they shipped it out to unwitting customers...

Software that doesn't work properly is a huge drain on your patience, money, and time. But the coders and software designers really aren't out to get you - they're just human. All it takes is for one person to make a mistake and then you get a defect, or in industry lingo a "bug", in the software. The bug might be in the code itself, in the instructions, or even in the requirements the coders were given in the first place (maybe no one told them to make it easy to install printers).

The software development process goes something like this:

Some guy or gal with spare cash thinks, "I'd really like a software game that lets me take out my aggression on green pigs." So they look up a developer or find a nerd on the street and say, "Hey! Hey you nerdy-looking person! Can you write me some code for a game that will let me throw stuff at green pigs?" to which the nerd excitedly replies, "Sure can! Gimme two hours." Eight hours later the nerd comes back with a really cool software game.

While this example of the software development process is a bit simplified and colourful hopefully you've caught on to the main points:

1. Certain requirements are given to the developer, e.g. throwing things at green pigs.
2. The developer writes an app that lets you throw things at green pigs.
3. The person who paid for the project ends up with code that actually does let you throw things at green pigs.

In this example a defect can occur at the requirements stage. For example, the buyer might want the game to let you skip levels, but never explain that to the developer. Or a defect can occur during the coding stage - the developer might write a bug in the code that stops the pigs from disappearing when they are hit.

How would you catch these bugs? Well, if you try to skip a level and find that you can't, you've just discovered a software failure. Same thing with the pigs - once you hit one and it doesn't disappear, the defect becomes noticeable as a software failure. Both of these bugs were discovered by executing the code, triggering the failure, and getting unexpected results. But what about a bug that causes the app to fail on a particular type of smartphone? If no one with that phone buys the app, the defect will never result in a failure and will never be noticed or corrected.

In reality it's frankly impossible to test a piece of software under every condition. What if it only fails while running the 39th level on an overclocked special edition R2D2 Droid phone in Arabic language mode?

Well, software testing is about risk management. The point of software testing is to run a wide variety of tests that will catch as many bugs(defects) as possible before releasing the code. Even though it will never catch 100% of the bugs, testing will still greatly improve your software and save your customers hours of frustration.

M any people think that testing is just about finding bugs, and maybe about giving developers' headaches (those poor developers). But as we've mentioned before, there's much more to testing than uncovering those creepy-crawlies.

In short, here are the objectives of testing:

1. Identification: Identify defects in the software.
   
2. Confidence: Enable stakeholders to gain confidence in the system's level of quality.
3. Decision-making: Provide information for decision-making.
4. Prevention: Prevent future bugs by improving the development process and standards.

These "whys" of testing can be remembered with the acronym ICDP (Identification, Confidence, Decision-making, Prevention). The other three are fairly self-explanatory, but decision-making merits a bit more discussion.

So how can testing help with decision-making?

Think about a software project that almost everyone has considered: building a website. You want something nice, so you hire a team of professionals to make one for you. When you get to testing the new site, everything looks great... but it takes 30 seconds to load. The results of the performance test now prompt you to do some decision-making. In this case, you decide to buy a dedicated server to host your website.

Take a popular social media site for example. They got as far as beta testing with their face-search feature. The reaction from initial users then provided the feedback necessary for them to pull the feature. Beta testing here lead directly to a decision to drop the new feature.

After those examples, hopefully you have a pretty good idea of how testing contributes to decision-making, the "D" of ICDP. And if you've got your head wrapped around ICDP, then you have a pretty good idea of the "why" of testing. In the next chapter we'll look at the "what".

When most people think about testing, they'll think of running tests on the software code. While that's part of the process, we need more to get the whole picture. Testing activities also include:

• Planning the approach to testing
• Manually reviewing project documents like specifications and code
• Using automated tools to scan for defects
• Setting up test conditions like sample data, Internet access, etc.
• Designing and coding test cases that try out particular features of the system
• Verifying results
• Evaluating results to determine whether the project can progress to the next phase, should stay where it is, or whether it even needs to go back a step or two
• Reporting test results about the software under scrutiny, and about the testing process itself (such as feedback on the appropriateness of testing, recommendations for future tests, etc.)
• Housekeeping activities like sign-offs that are part of the test phase finalization

You may have noticed that debugging is not on the list. Debugging is not actually a testing activity (although FINDING bugs and their causes is); it's a development activity. In a perfect world, testers would find the bugs and developers would fix them as part of the normal process for addressing defects. Testers just need to figure out what, exactly, needs to be fixed - which is often quite a tricky problem in itself.

With that we conclude our overview of the "why" and "what" of testing.

S oon we'll be digging deeper into the heart of testing and to put you on a good footing for this journey there are seven key testing principles for you to understand. You can remember these principles with the acronym "PEEC PAC", which refers to the words in bold below. The principles are:

1. Testing shows presence of defects

   The failure of a test indicates the presence of a defect related to that test. However, the success of one or more tests doesn't mean that the software you're testing doesn't have defects in other areas, any more than testing if one piece of meat on the barbecue is done can guarantee that they're all done. Although testing reduces the probability of undiscovered defects remaining in the software, it does not offer a proof of correctness for the software because not all possible features and uses of the software are tested.