/* */

Sunday, December 30, 2007

Regression testing is not a phase

Time to vent. Probably past time.

One of my several careers has been as a software tester. I enjoyed it more than programming, probably due to the destructive side of my nature. I learned a few things and tried to share them with other developers and testers, but most have been disinterested in (or unable to) change. I finally figured out why.

I was reviewing some test plans at a client's site and came across some language that I had seen before at other companies and for other systems. Roughly, it goes "first we'll have the developers do unit testing, then we'll do the system, regression and integration testing."

The problem with that sentence is that regression testing isn't a phase -- it's a technique.

Regression testing is the re-running of tests from earlier releases to see if the changes made for the new release have damaged ("regressed") the software's existing features. There is no such thing as "regression testing", but there is regression unit testing, regression feature testing, there should be (but rarely is) regression user acceptance testing.

The tests written to exercise the new features are new tests, not regression tests. They're added to the regression test suite when the release is promoted to production and will be re-run for the next release.

Regression testing, when done properly, uses the same input data every time

Another testing technique that is rarely used but should be required is challenge testing.

Challenge testing is feeding the system bad orders, entering too many characters into the zip code field, logging on with the wrong password. Challenge testing is assaulting the system in an attempt to find things that don't work right.
Unfortunately, the users don't want to find things wrong with the system, they want to prove tht it works. That's not possible with 'happy path' testing alone and, in fact, isn't possible at