Regression Testing

When you debug a program, you run it through each of the test cases that you have already developed as your test plan.

You probably work through the test cases in order, checking them off as your program passes them.

But then a test case does what it is supposed to and uncovers a bug in your program!

What's your immediate action?

Fix it!

That's normal. Ok, you went into your code and you fixed the bug.

The question that arises is what do you do next? The obvious answer seems to be "pick up in my test cases where I left off and continue through the rest of the list." But that is not the correct answer!

The fact is when you edit a program's code, you may be fixing one bug, but you may be either causing or uncovering OTHER bugs! One bug may be masking or covering up another bug. Or by fixing one bug incorrectly, you are actually causing a new bug which was not in the code before. This has been observed happening to all levels of programmers.

"Regression testing" is the principle that says you should go back to the start of your test plan (hence the "regression") and run all of your tests AGAIN any time that you make any change at all to your code. This makes sure that no new bugs were made and that no bugs were uncovered by the latest fix.

This seems redundant, but it is definitely not. It does mean that many of your early test cases will be tried many times. That's correct. You should try to put test cases in your lists in the order that you think they will "turn up" bugs the fastest, the best.

Note that "any change at all" means even putting in a comment, renaming a variable, in other words, small changes that you would not think would ever make a difference. They can! You can accidently delete code while you are editing. You can accidently comment out a line of code so it doesn't run any more. You can put stray characters into your file so that it won't even translate properly any more!

When you submit your program assignments, make sure that you run your test cases again, AFTER you submit. Make sure that what you turned in is indeed the version of the code that you used when you ran your testing. Strange to say, but many people have lost points because they did not do this step.

Wikipedia Article about Regression Testing