Improving Automation in Developer Testing

Tao Xie
North Carolina State University
http://people/engr.ncsu.edu/txie

Abstract:

Developer testing, a common step in software development, involves
generating sufficient test inputs and checking the behavior of the
program under test during the execution of the test
inputs. Complicated logics inside a method make generating appropriate
arguments difficult. In testing object-oriented programs, generating
method sequences to put the receiver object or argument objects into
appropriate states further complicates test-input generation. Existing
industrial and academic tools incorporate various techniques to
address the challenges of generating sufficient test inputs. After the
generated test inputs are executed, program crashes or uncaught
exceptions can be used to indicate program problems, especially
robustness problems. However, some program problems such as producing
wrong program outputs do not crash the program. Since manually
inspecting the outputs of a large number of generated test inputs is
not feasible, existing industrial and academic tools incorporate
various techniques to address the challenges of checking the behavior
of the program under test during the execution of the test
inputs. This talk will present an overview of achievements and
challenges in improving automation in developer testing, especially on
test-input generation (i.e., generating sufficient test inputs) and
test oracles (i.e., checking the behavior of the program under test).