-
ODIN
Spectrum-based fault localization (SBFL) is a simple, effective and scalable approach, and, hence, bestowed with multiple fault localization metrics. Though there have been analytical and empirical studies, the relative effectiveness of these metrics is still unknown— while the theoretical studies were made under strong assumptions to keep the analysis tractable, the empirical studies were conducted on a small set of programs/faults and under a limited set of factors that impact the performance of SBFL metrics. We design a stochastic sampler for fault spectrums, ODIN, that allows us to sample fault spectrums based on a set of carefully selected control parameters and make statistically significant inferences. Armed with ODIN we undertake a study on 37 most popular SBFL metrics. Our study validate— and, in fact, generalize—inferences made in prior studies under strong assumptions. Moreover, we discover novel equivalence relations between SBFL metrics that hold under all experimental configurations.
GitHub: Yet to publish
-
AndroFL
We present our tool, ANDROFL, that provides an infrastructure for an evolutionary algorithm-based test-suite generation backed by a statistical fault localization module for diagnosing faults. ANDROFL’s evolutionary test-generator supports configurable fitness functions (e.g., coverage, diagnos- ability metrics like Ulysis). The statistical fault localization engine supports popular metrics like Ochiai, Tarantula and Barinel, and allows adding custom fault localization metrics. We evaluated ANDROFL on 20 open-sourced apps from F-Droid, and demonstrates significant efficiency gains: it reduces debugging effort by 74% (median EXAM score) compared to random testing— enabling developers to pinpoint faults ≈ 4× faster. Furthermore, ANDROFL localizes 25% and 50% more faults compared to random testing in the top-5 and top-10 ranked list in worst case ranking scenario.
-
Chiron
Program analysis, verification and testing are important topics in programming languages and software engineering. They aim to produce engineers who are not only capable of empirically evaluating but, also formally reasoning on the correctness of software systems. We propose a specialized framework, CHIRON, designed to teach graduate-level courses on these topics. CHIRON has a small code base for easy understanding, uses a unified intermediate representation across all its analysis modules, maintains a modular architecture for plugging in new algorithms and uses a “fun” programming language to provide a gamified experience. Currently, it packages a dataflow analysis engine for driving compiler optimizations, an abstract interpretation engine for verification, a symbolic execution engine, a fuzzer and an evolutionary test generator for program testing, and a spectrum based statistical bug localization module.
Within CHIRON, program analysis tasks are posed in an unconventional setting (as adventures of a turtle) to provide a gamified experience; the accompanying animations (showing the movements of the turtle) allow the student to understand the underlying concepts better, and the detailed logs allow the teaching assistants in their grading activities.
CHIRON has been used in two offerings of a graduate level course on program analysis, verification and testing. In response to our survey questionnaire, all the students unanimously held the opinion that CHIRON was extremely helpful in aiding their learning, and recommended its use in similar courses.