What earlier causal-inference evaluation actually measured
Before CausalVerify, LLM causal benchmarks concentrated on causal direction, graph reasoning, counterfactuals, correlation-to-causation judgments, and intervention design, while code benchmarks tested whether programs could be synthesized, executed, or pass tests [1]. Those are real capabilities, but they stop short of the applied workflow that matters in econometrics: interpret a study, choose a design, write analysis code, and report a treatment effect that recovers the intended estimate [1]. The gap is not hypothetical. A model can name the right design, produce code that runs without error, and still estimate the wrong coefficient, a failure mode that appears fluent and complete while the causal claim remains unverified [1].
The broader causal-ML evaluation literature had already recognized that fixed, hand-crafted datasets produce brittle conclusions. CausalProfiler argued for synthetic benchmark generation from an explicit space of interest, sampling causal models, queries, and ground truths so methods can be assessed under diverse conditions [5]. CausalVerify adopts a related logic for LLM workflows: fixed-seed synthetic data-generating processes supply executable targets, while real papers supply ecological context [1]. The difference is the endpoint. CausalProfiler evaluates causal ML methods against ground-truth queries [5]; CausalVerify evaluates whether an LLM-written R workflow agrees with a benchmark-fixed canonical estimator on the same realised dataset [1].
Execution success and estimate correctness diverge sharply
CausalVerify's Experiment B runs model-written R code and checks whether the extracted treatment-effect estimate matches a canonical estimator, a layer the authors call L2b+, distinct from L2b, which records only whether code executes [1]. Across seven evaluated LLMs, final L2b+ pass rates span 10% to 88% at the default 50% tolerance, and 66 of 426 executing workflows (15.5%) return a wrong estimate [1]. The execution cascade decomposes each model's 100 scenarios into correct estimates, executed-but-wrong coefficients, execution failures, and missing or unparseable code, making visible what a pass/fail execution metric would hide [1].
The ranking evidence sharpens the point. Execution ranking (L2b) agrees with L2b+ correctness at Kendall τ = 0.81 and Spearman ρ = 0.93, whereas text-direction agreement (L4) against consensus labels shows Kendall τ between −0.20 and 0.10 [1]. Leave-one-model-out stability keeps Kendall τ in [0.733, 0.867], and leave-one-design-out stability keeps the L2b+ ranking within τ ∈ [0.81, 1.00] of the full panel with top-two and bottom-two model sets unchanged [1]. Execution is therefore not sufficient, but it is a much stronger proxy for reference-agreement correctness than text-level direction [1].
This connects to a prior critique of LLM causal discovery. Wu and colleagues argued that LLMs' autoregressive, correlation-driven modeling lacks theoretical grounding for causal reasoning, that prompt engineering or information leakage can overstate performance, and that LLMs should be confined to non-decisional support such as search acceleration rather than determining causal existence or direction [3]. CausalVerify does not test causal discovery from observational data, but its finding that plausible text and runnable code can coexist with wrong estimates is consistent with that caution: surface fluency is not a guarantee of causal correctness [1][3].
Why real-paper text agreement cannot carry the correctness claim
Experiment A scores method-family and direction agreement against four-LLM consensus labels on 259 published economics papers [1]. L3 is scoreable for 187 of 259 papers and L4 for 92 of 259, with unscoreable cases predominantly those where the consensus did not converge on a single method-family or direction label [1]. GPT-4o leads L3 at 88.8% and Sonnet leads L4 at 85.9%, and the rankings do not collapse to one textual ability score [1]. A blinded 30-paper human ambiguity audit found agreement with the four-LLM consensus only fair for direction labels (Cohen's κ = 0.294), so L4 scores partly reflect reference-label noise rather than purely model behavior [1].
The authors therefore treat Experiment A as a real-context text diagnostic, not the primary correctness endpoint, and reserve executable verification for Experiment B [1]. This design choice matters for readers comparing benchmarks. A high text-agreement score does not license a claim that a model recovers causal estimates, and the weak or negative association between L4 and L2b+ ranking quantifies that disconnect [1]. The methodological lesson is that separating ecological realism from verifiable computation is not a compromise but a way to keep each claim within the evidence that supports it [1].
Confidence does not reliably flag wrong estimates
The calibration arm asks each model to rate its own previous Experiment B output without seeing the L2b+ pass/fail label [1]. Confidence gaps between correct and incorrect workflows remain small for the main models: Opus +0.075, GPT-5 −0.045, GPT-4o +0.016, Sonnet −0.011, o3 −0.010, Gemini +0.234, and Kimi +0.027 [1]. GPT-5 illustrates the asymmetry, ranking second on L2b+ while showing a slightly negative confidence gap [1]. Under this retrospective confidence prompt, self-assessment does not provide a reliable deployment triage signal [1].
This finding sits alongside earlier work showing that LLM causal reasoning can shift under sustained questioning. Cox's Socratic dialogue with ChatGPT on PM2.5 and mortality moved the model from a strongly worded causal claim to a more hedged conclusion acknowledging omitted confounders, suggesting that initial confident statements may not survive scrutiny [4]. CausalVerify's calibration result is narrower and more quantitative: even when models are asked to rate their own executed workflows, confidence does not cleanly separate correct from incorrect estimates [1]. The authors note this does not rule out stronger calibration interfaces, leaving the design of reliable uncertainty signals an open question [1].
Where the conclusion stops and what remains untested
CausalVerify's claims are confined to standardized single-shot workflows in four design families under the evaluated R backend and model panel, and the benchmark does not measure general causal-inference ability [1]. The canonical estimands are fixed benchmark references, not exhaustive comparisons of estimator choices: the DID template is not a full benchmark of staggered or heterogeneous-treatment estimators, the event-study window mapping is fixed, the IV reference tests implementation of the estimand without validating the exclusion restriction or weak-instrument diagnostics, and the RDD reference is not an exhaustive comparison of bandwidth or inference choices [1]. A separate blinded 50-cell human audit validated the coefficient-extraction instrument with 90.9% numeric agreement among human-scoreable effect-present cells and 88.6% agreement on the induced L2b+ pass/fail decision, but did not change the frozen headline rates [1].
The robustness check with Llama-3.3-70B-Instruct reproduces the same qualitative gap between runnable code and numerically correct code, reaching L2b = 41% and L2b+ = 20%, though the authors treat this as a single cross-vendor check rather than a complete benchmark of the open-source ecosystem [1]. For causal inference methodologists, the practical implication is that evaluating LLM workflows should validate the target quantity rather than rely on a proxy for producing it [1]. For LLM evaluation researchers, the open questions are whether execution-grounded correctness generalizes beyond these four design families and the R backend, and whether any confidence interface can supply a usable triage signal [1].
About These Sources
This research page is built on 5 studies (3 peer-reviewed, 2 preprints) — published from 2023 to 2026, 4 from 2024 or later — selected as the most relevant from 9 studies that passed quality screening, drawn from 78 papers retrieved from a database of over 500 million.
Sources used in this answer
CausalVerify: An Execution-Grounded Benchmark for LLM Causal Inference Workflows
CausalVerify introduces an execution-grounded benchmark separating real-paper text agreement from verifiable computation, finding that 66 of 426 executing workflows (15.5%) return wrong estimates and that execution ranking tracks reference-agreement correctness far better than text-direction scoring [1].
Design and Optimization of Domain-Specific Large Language Model Agents: Towards Automated Causal Inference
Jiang's work on domain-specific LLM agents for automated causal inference benchmarks end-to-end performance of sequential steps, establishing a foundational framing for agentic causal-inference pipelines that CausalVerify evaluates at the executed-estimate level [2].
LLM cannot discover causality, and should be restricted to non-decisional support in causal discovery
Wu and colleagues argue that LLMs cannot reliably discover causality and should be restricted to non-decisional support, providing limitation evidence that aligns with CausalVerify's finding that plausible text and runnable code can accompany wrong estimates [5].
Causal reasoning about epidemiological associations in conversational AI
Cox's Socratic dialogue shows ChatGPT shifting from a confident PM2.5-mortality causal claim to a hedged conclusion under sustained questioning, offering limitation evidence that initial LLM causal confidence may not survive scrutiny [6].
CausalProfiler: Generating Synthetic Benchmarks for Rigorous and Transparent Evaluation of Causal Machine Learning
CausalProfiler generates synthetic benchmarks for causal ML from an explicit space of interest, providing validation evidence that synthetic ground-truth benchmarks can expose method failure modes and supporting CausalVerify's use of fixed-seed synthetic scenarios [9].