[View PDF](/pdf/2507.17542)
    [HTML (experimental)](https://arxiv.org/html/2507.17542v2)

            Abstract:Bug reproduction is critical in the software debugging and repair process, yet the majority of bugs in open-source and industrial settings lack executable tests to reproduce them at the time they are reported, making diagnosis and resolution more difficult and time-consuming. To address this challenge, we introduce AssertFlip, a novel technique for automatically generating Bug Reproducible Tests (BRTs) using large language models (LLMs). Unlike existing methods that attempt direct generation of failing tests, AssertFlip first generates passing tests on the buggy behaviour and then inverts these tests to fail when the bug is present. We hypothesize that LLMs are better at writing passing tests than ones that crash or fail on purpose. Our results show that AssertFlip outperforms all known techniques in the leaderboard of SWT-Bench, a benchmark curated for BRTs. Specifically, AssertFlip achieves a fail-to-pass success rate of 43.6% on the SWT-Bench-Verified subset.





          Subjects:

            Software Engineering (cs.SE)

          Cite as:
          [arXiv:2507.17542](https://arxiv.org/abs/2507.17542) [cs.SE]



          (or
              [arXiv:2507.17542v2](https://arxiv.org/abs/2507.17542v2) [cs.SE] for this version)




                        [https://doi.org/10.48550/arXiv.2507.17542](https://doi.org/10.48550/arXiv.2507.17542)



                  arXiv-issued DOI via DataCite








## Submission history

 From: Noble Saji Mathews [[view email](/show-email/80d80b43/2507.17542)]
            **[v1]**
        Wed, 23 Jul 2025 14:19:55 UTC (323 KB)

    **[v2]**
        Wed, 7 Jan 2026 20:10:04 UTC (514 KB)