Testing Your Software

Hemant

Goals of this Section

To learn what software testing means To learn what to test. To learn how to test software.

Testing
Kernighan & Pike Definition:
A determined, systematic attempt to break a program that you think is working.

Testing can demonstrate the presence of bugs, but not their absence. (Dijkstra)

Testing
Note:
Testing is not done only after the code is written. For testing to be effective, it should be considered as an ongoing process that is involved in software design, programming style, and debugging.

What to Test
Test all boundaries
Check upper & lower bounds of *ALL* buffers. Ensure that a loop iterates the correct number of times. Recursive functions: Consider all possible states of the control data. Be sure there is a terminal condition!! Ensure that all conditional statements branch in the right direction.

What to Test
Test all assumptions
Identify any assumptions that you make and test them. This is the rationale behind conditional statements. Are your functions dependent on the correctness or state of some data. If so, test it! Look for words like must, should and assumes in comments. These words identify assumptions that may be less obvious and which may not have been handled correctly (or at all).

What to Test
Test pre and post conditions
Assertions can be used to verify the necessary pre-conditions or postconditions. This is the same as testing assumptions. Pre and post conditions are simply assumptions that you are making about the state of required data in critical portions of your code. Using assertions is just another way to test some of these assumptions.

What to Test
Defensive Programming (built-in testing)
Write code to check the consistency of data and the stability of the program wherever possible. Test return values from all system calls and library functions. These values are returned for a reason. They indicate problems and error states that need to be handled. Deal with all errors and anomalies. You (or the designer(s)) must decide how to handle each type of error. If you can conceive of the error, it will probably occur.

How to Test
Test simple parts first
Write a small portion of code (or a function) and test it before coding anything else. This will reduce the amount of testing and debugging that you need when you finish writing the software. Testing and debugging should go hand-in-hand and be a continuous process intertwined with writing the code. Test the small portions first and gradually build your code step-by-step, testing and debugging at each and every step.

How to Test
Black Box testing
Here you are not concerned about the details of what a function or piece of code does. You should be concerned simply with whether it produces the correct results. Give the function controlled data and test the return values. Since you know what values you should receive you can test whether the function works with varying data. This is the type of testing you are doing when you use test harnesses.

How to Test
Test interfaces between functions
Each function provides an interface to another function and hides the details of what it does. This is one area that is susceptible to bugs because incorrect use may cause incorrect results. Black box testing would be appropriate to use here. Test all return values from functions and be sure to handle (and/or report) error conditions in a sane manner. Be sure that each function has the ability to test for applicable error

How to Test
Test incrementally
Write a function and test it before writing another or using it in another function. For larger functions, write a small portion of functional code and test it. When a function relies on other functions, combine the smaller functions into a larger function and test as a whole functional unit. Check for and handle error states. Black Box testing, again, would be appropriate.

Testing Tips
As you write your code, test it in manageable chunks. Write a function, then test it. Ensure that all functions are tested before calling them from another function. If your function is large, write part of it and test that part before continuing with the remainder. This will greatly reduce the amount of testing & debugging required in the end.