LLM-Based Detection of Mobile App Bugs
Bachelor & Master Thesis
Description
Automated GUI testing tools find crashes, because a crash needs no knowledge of what the app was supposed to do. Most user-visible defects are not crashes: the app keeps running and renders a screen that looks plausible but is wrong. This thesis investigates whether multimodal large language models can serve as the missing test oracle for such defects in Android apps. Two bug families are in scope. GUI display bugs cover elements that are misaligned on screen, such as a button label that has lost its centering, and mixed-language screens in localized builds, where hard coded strings remain untranslated. Functional bugs cover behavior that is wrong without being fatal: a tap on "Notifications" returns to the home screen, an item added to the wishlist does not appear there, a sort by lowest price returns descending order, a newly added income of 100 is recorded twice or leaves the balance unchanged, an order detail page shows an unrelated order number, or a long amount is truncated, wrapped, or clipped by its container. Each of these is only visible when the observed screen is compared against what the user action should have produced, which is exactly the judgment the thesis asks an LLM to make from screenshots, view hierarchies, and interaction traces.
Project Goals
1. Detection: decide from a screen, or from an action-state pair, whether it contains a bug of the types above, and compare state representations (screenshot, view hierarchy, both, cropped elements).
2. Evaluation: build a labeled set from real bugs in open source apps and from seeded bugs, and measure precision, recall, per-screen false positive rate, cost, and verdict stability against existing tools.
Required Knowledge
- Android testing and GUI automation (Appium, UI Automator, or similar)
- Python programming and data processing
- Experience with (multimodal) LLMs, prompt and agent design
- Computer vision techniques (Image Segmentation, Object Detection, etc.)
References
Z. Liu et al., "Seeing is Believing: Vision-Driven Non-Crash Functional Bug Detection for Mobile Apps," in IEEE Transactions on Software Engineering, vol. 51, no. 12, pp. 3452-3466, Dec. 2025, doi: 10.1109/TSE.2025.3614469.
Liu, Zhe, Chunyang Chen, Junjie Wang, Yuekai Huang, Jun Hu, and Qing Wang. "Owl Eyes: Spotting UI Display Issues via Visual Understanding." ASE 2020, pp. 398–409.