Problem Statement
Industry is constructing more and more complex software systems,
one consequence of this is that the costs associated with these system updates
and the verification of the implementation becomes increasingly difficult, and
expensive, to achieve. Software testing is an expensive component of software
development and maintenance. A particularly labour-intensive, and hence costly,
component of this test process is the generation of test data to satisfy the
testing requirements. Given a testing objective, test-data generation techniques
find a system input, which will satisfy the objective. The automation of test-data
generation is an important step in reducing the cost of both software development
and maintenance.
Approaches to be employed
A small number of test-data techniques have already been automated:
random, structural or path-oriented, goal-oriented, analysis-oriented test-data
generators. However, the limitations of these approaches have stopped their
general acceptance. Random generators, create large amounts of test data;
however, because no information exists about the testing objectives, the generators
often fail to find data that satisfy the stated objectives of the testing process.
A structural or path-oriented generator first identifies a path for which test data
is to be generated; unfortunately, the path is often infeasible causing the
generator to fail to find an input that will traverse the path. Analysis-oriented
generators have the ability to generate high quality test-data, but rely upon their
designer having a great insight upon the domain of operation, and hence are not
readily generalisable to arbitrary software systems. Hence, the project plans to
explore goal-oriented generators to provide an industrial-strength solution. Initial
work with goal-oriented approaches has achieved some limited success producing high
quality test-data for small programs. The approach to date can be characterized as:
· Find a numerical function, which measures the quality of a set
of test-data. This function tends to represent a small subset of the testing
objectives.
· Produce various sets of test-data, which seek to minimize this
function, using an optimization technique such as simulated annealing or genetic
algorithms.
Although the work has had some success, many of the articles report
that these approaches have serious problems associated with failing to find
global minima. The search space tends to lack features and consists on large
flat areas, which provide no information about the location or the direction
of the true local minima. Additionally, it appears that the success of the various
optimization approaches is strongly influenced by the nature and structure of
the program. This has resulted in many of the authors calling for comparative
studies about the effectiveness of computational intelligence approaches in
solving the test-case generation problem.