Do Language Models Know Who Fails? Look-Ahead Bias in LLM Credit-Risk Assessment

RAIDS Lab Authors

Zhengyi Yang

Director

Details

Research Area

Tags

Resources

Abstract

Large language models (LLMs) are being adopted as low-cost analysts of corporate disclosures, and recent studies report that they forecast adverse outcomes directly from filing text. We show that this apparent forecasting power is substantially an artefact of look-ahead bias, because the models were trained on text that already reports the outcomes they are asked to predict. Using 443 U.S. public-company bankruptcies from 2015 to 2026, each paired with a surviving firm and represented by the Risk Factors section of its last annual report before failure, we score every filing twice, once with the issuer named and once with all identifying detail removed. Naming raises the estimated 24-month default probability by 20 percentage points for firms that failed and lowers it by 7 points for survivors. Direct memory tests confirm the source, since the model can state the bankruptcy outcome for 64% of failed firms with no document in front of it. De-identifying the filing cuts measured discrimination from an area under the ROC curve of 0.97 to 0.84, and the surviving skill no longer significantly beats a conventional accounting and dictionary model. We provide a simple de-identification and memory-probe procedure that lets researchers and risk managers measure and remove this bias.

@misc{fang2026language,
  title = {Do Language Models Know Who Fails? Look-Ahead Bias in LLM Credit-Risk Assessment},
  author = {Yiwen Fang and Xiaolu Hu and Yifu Tang and Mark Wallis and Zhengyi Yang},
  year = {2026},
  doi = {10.2139/ssrn.7429342},
  url = {https://ssrn.com/abstract=7429342},
  note = {SSRN preprint}
}