Why Your LLM-Generated Testbench Compiles But Doesn’t Verify: The Verification Gap Problem

The article discusses the issue of LLM-generated testbenches compiling successfully but failing to verify at the functional level, highlighting the Verification Gap problem. It explains that compile success does not guarantee functional correctness at the protocol level, as compilers focus on type consistency and syntax rather than protocol-specific details. The piece presents failures from a case study on an AHB2APB bridge, emphasizing the importance of metrics like Repair Efficiency Score (RES), Verification Gap (VG), and Specification Coverage Ratio (SCR) to measure the gap between compilation and verification. It suggests that improving formal specification schemas is more effective than increasing model complexity in LLM-based verification automation. The article concludes with insights on the importance of a well-designed testbench in detecting integration bugs and provides recommendations for verification teams using LLMs.