SolBehave: Evaluating the Behavioural Correctness of LLM-Generated Smart Contracts

Evidence record

The SolBehave paper tests 250 reproducible Ethereum contracts and seven open-weight models using compilation, differential fuzzing, historical replay and DCR-guided synthetic sequences. The 31.4% rate is no detected divergence under bounded fuzzing, not a mathematical equivalence proof.

Read the original source ↗

Claim-level citations (5)

• supportsSolBehave finds compiling AI-generated Solidity often diverges from its reference: SolBehave studies 250 real contracts with differential fuzzing, historical transaction replay and DCR-guided synthetic replay.

  Abstract and Section IV, Methodology

• supportsSolBehave finds compiling AI-generated Solidity often diverges from its reference: These results are controlled regeneration experiments rather than confirmed production attacks, with limited generalization to complex DeFi systems.

  Section VI, Threats to Validity

• supportsSolBehave finds compiling AI-generated Solidity often diverges from its reference: The benchmark selected 250 contracts with 1 to 20 state-changing functions after reproducibility checks, and its statistical equivalence rate is limited to the tested fuzzing inputs.

  IV-A Benchmark Construction, Size Filter; IV-C O1 Differential Fuzzing; V-F

• supportsSolBehave finds compiling AI-generated Solidity often diverges from its reference: DCR-guided replay reached 246 to 311 state-changing functions not reached by the historical transaction oracle, depending on the model extracting the DCR graph.

  V-E Coverage Explains the Complementarity; O2 and O3 coverage table

• supportsSolBehave finds compiling AI-generated Solidity often diverges from its reference: The authors report only 31.4% pooled function-body behavioral equivalence among compiling samples and contract replay divergence for six of seven models.

  Abstract; Section V-D and V-F