A new research paper titled 'Faithfulness Without Anchors' challenges the foundation of explainable AI (XAI) metrics, arguing that current approaches to measuring explanation faithfulness are structurally flawed and fail to capture whether explanations accurately reflect model reasoning processes.
The paper contends that feature importance methods — widely used to explain AI model decisions in applications ranging from healthcare diagnostics to financial lending — suffer from fundamental misalignments that prevent them from accurately capturing whether an explanation reflects the model's true reasoning process. These methods attempt to identify which input features most influenced a model's output, but the researchers demonstrate that multiple contradictory explanations can be equally 'faithful' under current metrics.
Researchers identify several structural issues: the lack of ground truth anchors for comparison, meaning there is no objective standard against which to measure explanation accuracy; sensitivity to irrelevant features that can distort the importance rankings; and the tendency of explanation methods to produce plausible-sounding but misleading outputs that satisfy mathematical criteria without providing genuine insight into model behavior.
The findings have significant implications for AI regulation and deployment. As governments increasingly require explainability for high-stakes AI systems — including the EU AI Act and proposed US legislation — this research suggests current compliance approaches may provide false confidence in model transparency. Organizations relying on XAI tools to meet regulatory requirements may be operating under a misconception about what these tools actually demonstrate.
The research team calls for a fundamental rethinking of how the field approaches explainability, suggesting that current metrics may be measuring the wrong properties entirely. They propose alternative evaluation frameworks that focus on causal relationships rather than feature correlations, though they acknowledge this represents a significant methodological shift that will require extensive validation.