Program Testing for CS 115

How do you know if your program is correct? "Well, IDLE runs it." This is not sufficient. If it runs, there are no syntax errors in the part of the program that you actually executed. There could be syntax errors in the part of the program you did NOT run, and there still could be semantic errors. It could be that the program does not do what you intended! or the instructions in it do not have the meaning that you wanted.

You should run your program with different inputs, and check the output to prove that the program did as you wanted it to.

You should think about what you need to test. Try large numbers and small, positives and negatives, zero, short strings and long ones, empty files and non-existent files. Try inputs that will cause all parts of your program to be executed at least once. Make each loop body be executed zero times if possible, just one time, and more than one time. Each branch of an if/then/else should be executed. ONE test case is almost always not enough!

Testing is one of the components that your program will be evaluated on. Specifically, Testing is up to 16 points on most grading sheets for your program assignments.

Try to think about what is "normal" input and what is outside that range.

The quality of the test cases is more important than their quantity. Testing the same part of the code ten different times will probably give successful results if the first test does. It is better to make sure that all parts of the code are executed at different times.

Each test case should be labeled as to what you were testing there. Labels are like "Normal case: negative number", "Error case: zero not allowed" or "Error case: Z is an illegal character".

A test plan is something a programmer develops as they design and write their program. It specifies what cases they are testing, gives inputs for the case, the expected output and the actual output.

An Example Test Plan

Here is an example problem:
Some code is supposed to input an integer number and test it to see if it is in the range 5 to 10 inclusive. If it is, the output should be "yes". If it is not, then the output should be "low" if it is lower than 5, and "high" if it is larger than 10.
Test Case Input Expected Output Actual Output
Normal case, number in range 7 "yes" "yes"
Normal case, number too low 4 "low" "low"
Normal case, number too high 11 "high" "high"
Boundary case 5 "yes" "yes"
Boundary case 10 "yes" "yes"
Error case (non-numeric instead of integer) ABC unpredictable 29384131