This paper describes a technique for regression testing a GUI-based software system. While this paper is not directly tied to acceptance-test driven development, it very much could be used to augment initial acceptance tests as they are transitions to automated regression tests. This paper aims at attempting to identify the useful characteristics of effective regression testing of GUI applications.
To facilitate this research, the authors created smoke tests for four different applications. 200 variations of each of these applications were created with random seeding of faults. 5000-8000 smoke test cases were created to test the GUI application, for a total of 4 million test runs. The authors were able to generate four contributions to regression smoke testing of GUI applications:
The regression smoke testing was fairly successful, with 60% of faults detected. The weakness in this fault detection was not the actual smoke tests, but the testing oracle. Faults seeded in dead code also were not caught with the smoke testing. Code coverage was also fairly good with this testing style, with roughly 50-70% coverage. Overall, the authors conclude that is a very promising technique and can be a very valuable and relatively cheap alternative to not testing in the long run.