Automated Discovery of Internet Reported Software Bugs and AI-Driven Test Case Generation

Abstract

Software bugs reported on platforms like GitHub, Google Play, and Stack Overflow represent a vast, untapped source of real-world failure data. While automated test case generation has advanced with large language models, existing approaches operate only on source code or specifications and ignore user-reported defects. This thesis presents an end to-end framework that automatically discovers, structures, and converts internet-reported bugs into executable test cases. The framework comprises of four integrated phases: multi source bug collection from GitHub Issues and Play Store reviews using label-based filtering; NLP analysis using TF-IDF and a Naïve Bayes classifier with rule-based extraction; test case generation via LLMs (GPT-4, GPT-3.5-turbo, LLaMA-2 7B) using engineered prompts with an abstention mechanism; and export to standard formats. A key feature is principled refusal to generate low-quality test cases, prioritizing precision over recall. The Framework is evaluated on 120 annotated bug reports from three open-source repositories and five mobile apps, results show collection precision of 0.86, classification F1 of 0.83, and GPT-4 composite test-case quality of 0.93. Generation quality correlates with input completeness, validating the abstention design. This research establishes internet-reported bugs as a viable input for AI-driven testing. The modular open-source implementation supports independent reuse of each pipeline phase