Generative AI in Automated Software Testing: A Comparative Study

Abstract

Software testing is a crucial phase in the software development lifecycle, ensuring quality, reliability, and performance. Traditional automated testing tools, such as Selenium and JUnit, have improved efficiency but often require extensive manual intervention for test case creation. Recent advancements in Generative AI, particularly models like GPT-4, Codex, and CodeT5, have introduced a new paradigm in test automation by generating intelligent, dynamic test cases with minimal human involvement. This paper presents a comparative study of Generative AI models in automated software testing, analyzing their effectiveness in terms of test coverage, accuracy, execution time, and false positive rates. We benchmark multiple AI-driven testing approaches against traditional methods and evaluate their strengths and limitations. Experimental results indicate that Generative AI significantly enhances test efficiency, with models like GPT-4 achieving up to 92% test coverage and a 95% accuracy rate. However, challenges such as AI hallucinations, dependency on training data, and ethical considerations remain critical.