xpectra.metrics

Calibration and selective-prediction metrics not provided by sklearn.

Functions

aurc(y_true, y_pred, confidence)	Area under the risk-coverage curve (lower is better).
expected_calibration_error(y_true, proba[, ...])	Top-label ECE plus the per-bin table used for reliability diagrams.
multiclass_brier(y_true, proba)	Mean squared error between one-hot labels and predicted probabilities.
risk_coverage(y_true, y_pred, confidence)	Selective-risk curve: abstain on the least confident samples first.
selective_metrics_at_coverage(y_true, ...)	Accuracy/macro-F1/MCC on the retained fraction at a target coverage.

xpectra.metrics.expected_calibration_error(y_true, proba, n_bins=15, strategy='uniform')

Top-label ECE plus the per-bin table used for reliability diagrams.

Parameters:

• y_true (ndarray)

• proba (ndarray)

• n_bins (int)

• strategy (str)

Return type:

tuple[float, DataFrame]

xpectra.metrics.multiclass_brier(y_true, proba)

Mean squared error between one-hot labels and predicted probabilities.

Parameters:

• y_true (ndarray)

• proba (ndarray)

Return type:

float

xpectra.metrics.risk_coverage(y_true, y_pred, confidence)

Selective-risk curve: abstain on the least confident samples first.

Parameters:

• y_true (ndarray)

• y_pred (ndarray)

• confidence (ndarray)

Return type:

DataFrame

xpectra.metrics.aurc(y_true, y_pred, confidence)

Area under the risk-coverage curve (lower is better).

Parameters:

• y_true (ndarray)

• y_pred (ndarray)

• confidence (ndarray)

Return type:

float

xpectra.metrics.selective_metrics_at_coverage(y_true, y_pred, confidence, coverage)

Accuracy/macro-F1/MCC on the retained fraction at a target coverage.

Parameters:

• y_true (ndarray)

• y_pred (ndarray)

• confidence (ndarray)

• coverage (float)

Return type:

dict[str, float]