Predicting Testability of Concurrent Programs

Tingting Yu, Wei Wen, Xue Han and Jane Huffman Hayes


Abstract

Concurrent programs are difficult to test due to their inherent non-determinism. To address the nondetermin- ism problem, testing often requires the exploration of thread schedules of a program; this can be time-consuming for testing real-world programs. We believe that testing resources can be distributed more effectively if testability of concurrent programs can be estimated, so that developers can focus on exploring the low testable code. Voas introduces a notion of testability as the probability that a test case will fail if the program has a fault, in which testability can be measured based on fault-based testing and mutation analysis. Much research has been proposed to analyze testability and predict defects for sequential programs, but to date, no work has considered testability prediction for concurrent programs, with program characteristics distinguished from sequential programs. In this paper, we present an approach to predict testability of concurrent programs at the function level. We propose a set of novel static code metrics based on the unique properties of concurrent programs. To evaluate the performance of our approach, we build a family of testability prediction models combining both static metrics and a test suite metric and apply it to real projects. Our empirical study reveals that our approach is more accurate than existing sequential program metrics.

Experiment Settings


Supplemental Data

The following table shows sample data of two applications. We will provide full data by the submission of the final version.
Program data1 data2 data3
MySQL1 CPM SPM CPM+SPM
MySQL2 CPM SPM CPM+SPM