QMIA Attack#

Quantile Membership Inference Attack (QMIA).

Scalable Membership Inference Attacks via Quantile Regression. Bertran et al., NeurIPS 2023. https://arxiv.org/abs/2307.03694

Trains a histogram-based quantile regressor on non-member hinge scores to learn per-sample membership thresholds. A sample is predicted as a member when its observed score exceeds the predicted threshold.

Uses HistGradientBoostingRegressor rather than GradientBoostingRegressor for its histogram-based splitting algorithm, which is faster on large datasets.

Key Features#

Multiclass support via full hinge score: logit(p_y) - max_{y'!=y} logit(p_{y'})
Q conditioned on (x, y): The regressor learns thresholds per sample and label.
FPR control: The quantile level (1 - alpha) calibrates the false-positive rate on non-members.

Benchmarking#

Run the full benchmark comparing QMIA against WorstCase and LiRA:

python examples/sklearn/benchmark_qmia_full.py

Example#

from sacroml.attacks.qmia_attack import QMIAAttack
from sacroml.attacks.target import Target

target = Target(
    model=model, X_train=X_train, y_train=y_train, X_test=X_test, y_test=y_test
)
attack = QMIAAttack(alpha=0.01, output_dir="output_qmia")
attack.attack(target)

class sacroml.attacks.qmia_attack.QMIAAttack(output_dir: str = 'outputs', write_report: bool = True, alpha: float = 0.01, p_thresh: float = 0.05, max_iter: int = 100, random_state: int = 0, report_individual: bool = False)[source]#

Paper-faithful tabular QMIA attack.

This implementation focuses on tabular classification. It fits a quantile regressor on public non-member examples (X_test, y_test) to predict a sample-dependent threshold for the hinge score. Membership evidence is then the margin between the observed score and the predicted threshold.

Methods

`attack`(target)	Check whether an attack can be performed and run the attack.
`attackable`(target)	Return whether a target can be assessed with QMIA.
`get_params`()	Get parameters for this attack.

classmethod attackable(target: Target) → bool[source]#

Return whether a target can be assessed with QMIA.

Parameters:

targetTarget: The target to assess.

Returns:

bool: True if the target has a model and data.

__init__(output_dir: str = 'outputs', write_report: bool = True, alpha: float = 0.01, p_thresh: float = 0.05, max_iter: int = 100, random_state: int = 0, report_individual: bool = False) → None[source]#

Construct a QMIA attack.

Parameters:

output_dirstr: Name of the directory where outputs are stored.
write_reportbool: Whether to generate a JSON and PDF report.
alphafloat: Target false-positive rate for the public non-member distribution.
p_threshfloat: P-value threshold for AUC significance reporting.
max_iterint: Maximum number of boosting iterations for the quantile regressor.
random_stateint: Random seed for the QMIA regressor.
report_individualbool: Whether to include per-record QMIA outputs in the report.

attack(target: Target) → dict#: Check whether an attack can be performed and run the attack.

get_params() → dict#

Get parameters for this attack.

Returns:

paramsdict: Parameter names mapped to their values.

attack_metrics: dict | list#

metadata: dict#

output_dir: str#

shadow_path: str#

write_report: bool#