Program Testing for CS 115

How do you know if your program is correct? "Well, the interpreter runs it." This is not sufficient. If it runs, there are no syntax errors in the part of the program that you actually executed. There could be syntax errors in the part of the program you did NOT run, and there still could be semantic errors in any part. It could be that the program does not do what you intended! or the instructions in it do not have the meaning that you wanted.

You should run your program with different inputs and check the outputs to prove that the program did as you wanted it to.

You should think about what you need to test. Try large numbers and small, positives and negatives, zero, short strings and long ones, empty files and non-existent files. Try inputs that will cause all parts of your program to be executed at least once. Make each loop body be executed zero times if possible, just one time, and more than one time. Each branch of an if/then/else should be executed. ONE test case is almost always not enough!

Testing is one of the components that your program will be evaluated on. Specifically, Testing is some points on most grading sheets for your program assignments.

A test plan is something a programmer develops as they design and write their program. It specifies what cases they are testing, gives inputs for the case, the expected output and the actual output. A test plan is a collection of test cases which are intended to thoroughly test a program to detect errors. A test plan should include:

All cases in a test plan should be clearly labeled as to what is being tested.

Try to think about what is "normal" input and what is outside that range.

The quality of the test cases is more important than their quantity. Testing the same part of the code ten different times will probably give successful results if the first test does. It is better to make sure that all parts of the code are executed in different cases. If a program presents a menu, each one of the menu choices should be tested, as well as inputs which are not valid menu choices. If a program expects one type of data, try giving it a different type. Float instead of int, string instead of float, all are useful to know.

Each test case should be labeled as to what you were testing there. Labels are like "Normal case: negative number", "Error case: zero not allowed" or "Error case: Z is an illegal character".

An Example Test Plan

Here is an example problem:
Some code is supposed to input an integer number and test it to see if it is in the range 5 to 10 inclusive. If it is, the output should be "yes". If it is not, then the output should be "low" if it is lower than 5, and "high" if it is larger than 10.
Test Case Input Expected Output Actual Output
Normal case, number in range 7 "yes" "yes"
Normal case, number low 2 "low" "low"
Normal case, number high 14 "high" "high"
inside low Boundary case 5 "yes" "yes"
outside low Boundary case low 4 "low""low"
inside high Boundary case 10"yes""yes"
outside high Boundary case 11 "high" "high"
Error case (non-numeric instead of integer) ABC unpredictable program crashes