Causal Risk Evaluation.pdf

Causal structure learning algorithms are evaluated via risk estimation, using a decision-theoretic framework. The goal is to assess the practical efficacy of one or more causal structure learning methods in a given problem setting. A theoretical notion of causal risk is introduced, along with sample quantities that can be computed from data. The relationship between the two is studied, both theoretically and through an extensive simulation study. The results provide an assumptions-light framework for assessing causal structure learning methods, applicable in a range of practical use-cases.

Key contributions include:

1. Decision-theoretic view of causal structure learning: A new approach to assessing existing methods in a problem-specific manner.
2. Causal risk estimation: A notion of causal loss that allows assessment via expected loss or risk.
3. Assumptions-light risk estimation procedures: Procedures that can be used in practice using interventional data.
4. Extensive simulation study: Spanning over 40,000 data-generating regimes, investigating the practical performance of the proposed criteria.

The evaluation of causal structure learning algorithms is challenging due to the limitations of simulation studies and the difficulty of testing causal assumptions directly. The proposed approach addresses these challenges by providing a general framework for assessing causal structure learning methods.