Threshold Invariant Fairness.pdf

Threshold invariant fairness is a new notion that enforces a stronger condition on classifiers to achieve consistent fairness levels across different groups, regardless of the decision threshold. Existing fairness definitions, such as demographic parity and equalized odds, are threshold-sensitive, meaning that fairness may not hold when the decision threshold is changed. To address this, two approximation methods are proposed to equalize the risk distributions among groups, which can be incorporated into various differentiable classifiers. The advantages of this approach include not requiring protected attributes in the test phase and allowing for threshold tuning while maintaining fairness.

Key aspects of threshold invariant fairness include:

Enforcing equitable performances across different groups independent of the decision threshold
Equalizing the distributions of risk scores over all groups
Allowing for threshold tuning to modify the positive rate of classifier predictions while maintaining fairness
Not requiring protected attributes in the test phase

The proposed methodology is effective in alleviating threshold sensitivity in machine learning models designed to achieve fairness, and experimental results demonstrate its effectiveness.