SPLASH 2021
Sun 17 - Fri 22 October 2021 Chicago, Illinois, United States
Wed 20 Oct 2021 10:50 - 11:05 at Zurich B - Testing Chair(s): Iulian Neamtiu
Wed 20 Oct 2021 18:50 - 19:05 at Zurich B - Testing - Mirror Chair(s): Steve Blackburn

Android apps are GUI-based event-driven software and have become
ubiquitous in recent years. Obviously, functional correctness is critical for an app's success.
However, in addition to crash bugs, \emph{non-crashing functional bugs} (in short as ``non-crashing bugs'' in this work) like inadvertent function failures, silent user data lost and incorrect display information are prevalent, even in popular, well-tested apps.
These non-crashing functional bugs are usually caused by program logic errors and manifest themselves on the graphic user interfaces (GUIs).
In practice, such bugs pose significant challenges in
effectively detecting them because (1) current practices heavily rely on
expensive, small-scale manual validation (\emph{the lack of
automation}); and (2) modern \emph{fully automated} testing has been limited to
crash bugs (\emph{the lack of test oracles}).

This paper fills this gap by introducing \emph{independent view fuzzing},
\emph{a novel, fully automated approach} for detecting non-crashing functional bugs in Android apps.
Inspired by metamorphic testing, our key insight is to leverage the commonly-held \emph{independent view
property} of Android apps to manufacture property-preserving mutant
tests from a set of seed tests that validate certain app properties.
The mutated tests help exercise the tested apps under additional, adverse
conditions. Any property violations indicate likely functional bugs for further manual confirmation.
We have realized our approach as an automated, end-to-end functional
fuzzing tool, Genie. Given an app, (1) Genie automatically
detects non-crashing bugs without
requiring human-provided tests and oracles (thus
\emph{fully automated}); and (2) the detected non-crashing bugs are
diverse (thus \emph{general and not limited to specific functional
properties}), which set Genie apart from prior work.

We have evaluated Genie on 12 real-world Android
apps and successfully uncovered 34 previously unknown non-crashing bugs
in their latest releases — all have been confirmed, and 22 have
already been fixed. Most of the detected bugs are nontrivial and have
escaped developer (and user) testing for at least one year and
affected many app releases, thus clearly demonstrating Genie's effectiveness.
According to our analysis, Genie achieves a reasonable true positive rate of 40.9%, while these 34 non-crashing bugs could not be detected by prior fully automated GUI testing tools (as our evaluation confirms).
Thus, our work complements and enhances
existing manual testing and fully automated testing for crash bugs.

Wed 20 Oct

Displayed time zone: Central Time (US & Canada) change

10:50 - 12:10
TestingOOPSLA at Zurich B +8h
Chair(s): Iulian Neamtiu New Jersey Institute of Technology
10:50
15m
Talk
Fully Automated Functional Fuzzing of Android Apps for Detecting Non-crashing Logic BugsVirtual
OOPSLA
Ting Su East China Normal University, Yichen Yan East China Normal University, Jue Wang Nanjing University, Jingling Sun East China Normal University, Yiheng Xiong East China Normal University, Geguang Pu East China Normal University, Ke Wang Visa Research, Zhendong Su ETH Zurich
DOI
11:05
15m
Talk
Permchecker: A Toolchain for Debugging Memory Managers with TypestateVirtual
OOPSLA
Karl Cronburg Tufts University, Sam Guyer Tufts University
DOI Pre-print
11:20
15m
Talk
Generative Type-Aware Mutation for Testing SMT SolversVirtual
OOPSLA
Jiwon Park École Polytechnique, Dominik Winterer ETH Zurich, Chengyu Zhang East China Normal University, Zhendong Su ETH Zurich
DOI
11:35
15m
Talk
Programming and Execution Models for Parallel Bounded Exhaustive TestingIn-Person
OOPSLA
Nader Al Awar University of Texas at Austin, Kush Jain University of Texas at Austin, Chris Rossbach University of Texas at Austin; Katana Graph, Milos Gligoric University of Texas at Austin
DOI
11:50
20m
Live Q&A
Discussion, Questions and Answers
OOPSLA

18:50 - 20:10
Testing - MirrorOOPSLA at Zurich B
Chair(s): Steve Blackburn Australian National University
18:50
15m
Talk
Fully Automated Functional Fuzzing of Android Apps for Detecting Non-crashing Logic BugsVirtual
OOPSLA
Ting Su East China Normal University, Yichen Yan East China Normal University, Jue Wang Nanjing University, Jingling Sun East China Normal University, Yiheng Xiong East China Normal University, Geguang Pu East China Normal University, Ke Wang Visa Research, Zhendong Su ETH Zurich
DOI
19:05
15m
Talk
Permchecker: A Toolchain for Debugging Memory Managers with TypestateVirtual
OOPSLA
Karl Cronburg Tufts University, Sam Guyer Tufts University
DOI Pre-print
19:20
15m
Talk
Generative Type-Aware Mutation for Testing SMT SolversVirtual
OOPSLA
Jiwon Park École Polytechnique, Dominik Winterer ETH Zurich, Chengyu Zhang East China Normal University, Zhendong Su ETH Zurich
DOI
19:35
15m
Talk
Programming and Execution Models for Parallel Bounded Exhaustive TestingIn-Person
OOPSLA
Nader Al Awar University of Texas at Austin, Kush Jain University of Texas at Austin, Chris Rossbach University of Texas at Austin; Katana Graph, Milos Gligoric University of Texas at Austin
DOI
19:50
20m
Live Q&A
Discussion, Questions and Answers
OOPSLA