Gregory M. Kapfhammer

Associate Professor of Computer Science

http://www.cs.allegheny.edu/~gkapfham/

  
View  Edit  Uploads  History  Top  Changes  Search

A Safe, Efficient Regression Test Selection Technique

  • Briefly review the process of selective regression testing. Why might this approach be appropriate for a large regression testing effort?
Selective regression testing is the process of reducing the time required to test a modified program by selectively picking which tests to reuse, and to expose where new tests may need to be implemented. This is a good idea on large suites, because it selects the most effective tests, so the integrity is not compromised, while still reducing time of testing.
Brandon Taylor
As everyone else has stated selective regression testing involves
picking a subset of tests from a pregenerated test suite and useing this subset to test modified versions of the original programs. We would like this subset to catch the most possible bugs for the amount of time we are given so we usually pick the tests that catch the most bugs. We can find these tests in several way such as mutation testing, Also we can not change the environment of the program when we modify the original program or we potentiall invalidate our test suite.
Jim Clause
The process of selective regression testing is determining which tests from a test suite T can be put into T' such that T' adequately tests a modified program P'. This approach is appropriate for any large testing effort in which the amount of time to determine T' is less than the amount of time it would take to test using T. For example, if a small part of the code was modified, it makes more sense to run a (potentially much) smaller test suite than to run the whole test suite again. bps
Selective regression testing is the regression testing process that reduces the cost of regression testing by taking pre-existing tests and reusing them. It is also capable of reducing the cost of regression testing by identifying parts of the changed program that need to be tested. Selective regression testing can be useful for large regression testing efforts because unlike other testing processes, it is more economical when the cost of selecting a reduced subset of tests to run is less than the cost of running the tests that the other test techniques do not consider.
Noosh Moussavi
  • What is controlled regression testing? Is it possible to apply selective regression testing techniques if you change the operating system or the Java virtual machine on which you execute P'?
Controled regression testing is the process of testing a modified program P' with the same tests that were used to test the original prgram P. Yes, as long as the test suite is still relavant for the change, otherwise a modified test suite would need to be created.
Brandon Taylor
Controlled regression testing determines how to test a new version of a program using a suite of tests that were run on the original
program. Changing the environment may or may bot invelidate the test suite but we can not assume that a test suite developed for a specific environment will be valid on a different environment to fix this we must run all of our test cases again and regernete the subset of these tests that we will use to test new versions of the program.
Jim Clause
Controlled regression testing is the process of determining how to test a modified program P' using the test suite T from the original program P. It may not be possible to use selective regression testing techniques when P's outside environment changes (in the case where P is ported to another operating system, for example, and the tests may need to be rewritten), but for the most part it is possible to use selective regression testing if P is platform independent. bps
Controlled regression testing tests the modified program P’ with the same tests and under the same conditions that were used with the original test P. Controlled regression testing uses the scientific method regression testing. In other words, it determines which code changes will cause errors in the new code and thus holds these factors constant when testing the modified program. I do not think that changing the operating system or language would allow the user to apply selective regression testing techniques when executing P’, such factors would alter the environment of the modified program too much when compared to the original. If other less environment specific factors are altered then selective regression testing would work.
Noosh Moussavi
  • What does it mean for a regression testing technique to be safe?
A technique is said to be safe when the tests are designed to demonstrate problems in the program which has been changed.
Brandon Taylor
A regression testing technique is safe when it can be shown that the tests chosen for T' by the technique can adequately show if faults have been introduced. bps

Like Brian has states a regression testing technique is safe when we can show that our subset of tests adaquatley show all faults that are introduced into a new version of a program. This makes me think that no testing techinque can be safe because these tests will not be able to detect faults of omission. It could be possible to leave out a new feature that was suppose to be added and still catch all errors in the code that was written but it can not catch the error of omitting other code.

Jim Clause
Regression testing techniques are considered safe when they pick every possible test from the original test suite that would be capable of exposing faults in the modified program.
Noosh Moussavi
  • Briefly summarize the conclusions and limitations of the empirical studies conducted by the authors. Can you propose any ways for eliminating some of the limitations listed by the authors?
The conclusions presented by the authors is that though their work is encouraging more work needs to still be down. They found that regression testing is a large part of the maintence section of software developement, and by improving its performance you can reduce the cost of the software. Another limitation is that there research has been focused on using an existing test suite, and more work needs to be donw on creating regression test suites.
Brandon Taylor
The authors drive at the fact that applications can cost less to be maintained if their method for applying selective regression testing can be used. They claim that their algorithm is safe, precise, efficient, and general. Their empirical results show that a great of time can be saved using their technique, and it was never shown to be slower than running T over P'. They list as a limitation the fact that they have only so far tested their method on a very small set of programs. The only way to get around this limitation is more testing. bps
I have a problem with the way the authors are presenting thier results. They state that running T' over P is never slower than running T over P. This is obvious since T' is a subset of T. They also say that it is possible that their method is faster even when considering the time necessary to calculate T'. This maybe true but they do not state how likely this is to happen in a real world situation. If you have a team of competent test writers it is possible that they will write very few extraneous test and then most likely the time needed to remove the excess tests will be greater than the time gained in execution. To over come these limitations they need to use more real world data to experiment on and then they will be able to answer these questions.

Jim Clause

1. It constructs a control flow graph for a procedure or program and its modified version and use these graphs to select tests that execute changed code from the original test suite.
This is appropriate for a large regression testing effort since they use more general techniques and also they opperate more effieciently. This inturns makes the cost of the regression testing less.

2. Controlled regression testing is the practice of testing P' under conditions equivalent to those that were used to test P. yes, since the algorithm is basic to all languages.

3. A regression testing technique is safe when it emphasizes selections of tests from T that can reveal faults in a modified program.

4. This will allow for better quality work that has been tested at a lower cost.
Some of the limitations were assumptions and estimating along with not exaclty enough testing since they can't test everything in the world.
By creating better tools to analyze the data would make the tests more relieable. This is the only one I can think of. sorry
Chris Howell
The authors conclusions on the empiricial studies that they conducted include the conclusion that there are some tests and test selection algorithms that were result in better savings when applied to different enviroments. They also conclude that their algorithm, specifically, can reduce the testing time required by the software regardless of the actual cost of analysis of the tests. They also conclude that there are factors that affect the results of regression testing. Their limitations include the testing of only a small group of possible programs and test suites, as a result there is still a lot more testing that remains undone. More testing will not only cut back on limitations but also eliminate setbacks that existed in their current system.
Noosh Moussavi

Link to this Page