Meta Attack

Meta-attack: aggregate per-record vulnerability across multiple privacy attacks.

Runs multiple privacy attacks (LiRA, QMIA, Structural) on the same Target, extracts per-record vulnerability scores from each, and aggregates them into a unified pandas DataFrame with two-level aggregation:

• Level 1 — within-attack: Mean, std, and consistency across repeated runs.

• Level 2 — cross-attack: Arithmetic/geometric mean of MIA scores, binary structural flag, and total vulnerability count.

Supports three operating modes via the behaviour parameter:

'run_all' (default)

Run every specified attack from scratch.

'use_existing_only'

Read per-record scores from existing report.json files in report_dir; no new attacks are executed. Use when attacks were already run (possibly at great computational cost) and you only want to collate their results.

'fill_missing'

Load any attacks already present in report_dir and run only those not yet found. Saves redundant computation when some attacks have been run but others have not.

The vulnerability matrix is saved as vulnerability_matrix.csv in output_dir.

Example

from sacroml.attacks.meta_attack import MetaAttack
from sacroml.attacks.target import Target

target = Target(
    model=model, X_train=X_train, y_train=y_train, X_test=X_test, y_test=y_test
)
meta = MetaAttack(
    attacks=[("lira", {}), ("qmia", {}), ("structural", {})],
    behaviour="run_all",  # alternatives: "use_existing_only", "fill_missing"
    output_dir="output_meta",
)
meta.attack(target)

class sacroml.attacks.meta_attack.MetaAttack(attacks: list[tuple | list], behaviour: str = 'run_all', report_dir: str | None = None, mia_threshold: float = 0.5, k_threshold: int | None = None, output_dir: str = 'outputs', write_report: bool = True, keep_separate: bool = False)

Aggregate per-record vulnerability across multiple privacy attacks.

Parameters:

attackslist[tuple]

Each entry is (name, params) or (name, params, n_reps). name must be one of SUPPORTED_ATTACKS. params is a dict of keyword arguments forwarded to the sub-attack constructor. n_reps (default 1) is the number of independent repetitions; useful for stochastic attacks like LiRA.

behaviourstr

'run_all' (default), 'use_existing_only', or 'fill_missing'. See module docstring for details.

report_dirstr or None

Directory to scan for existing attack report.json files when behaviour is 'use_existing_only' or 'fill_missing'. Defaults to output_dir when not provided.

mia_thresholdfloat

Score above which a record is flagged as MIA-vulnerable.

k_thresholdint or None

k-anonymity value below which a record is structurally vulnerable. None reads the default from the ACRO risk-appetite config.

output_dirstr

Directory for all outputs (sub-attack subdirectories, report, CSV).

write_reportbool

Whether to write JSON report and CSV to disk.

keep_separatebool

Controls JSON output location. False (default) appends the MetaAttack section to {report_dir}/report.json so it joins any sub-attack reports already there, matching the project convention. True writes a separate {output_dir}/report.json like the base class. The CSV (vulnerability_matrix.csv) and PDF always follow the JSON output location.

Methods

attack(target)	Check whether an attack can be performed and run the attack.
attackable(target)	Return whether target can be assessed with the meta-attack.
get_params()	Get parameters for this attack.

classmethod attackable(target: Target) → bool

Return whether target can be assessed with the meta-attack.

__init__(attacks: list[tuple | list], behaviour: str = 'run_all', report_dir: str | None = None, mia_threshold: float = 0.5, k_threshold: int | None = None, output_dir: str = 'outputs', write_report: bool = True, keep_separate: bool = False) → None

Instantiate an attack.

Parameters:

output_dirstr

name of the directory where outputs are stored

write_reportbool

Whether to generate a JSON and PDF report.

attack(target: Target) → dict

Check whether an attack can be performed and run the attack.

get_params() → dict

Get parameters for this attack.

Returns:

paramsdict

Parameter names mapped to their values.

attacks: list[tuple[str, dict, int]]

behaviour: str

k_threshold: int

keep_separate: bool

mia_threshold: float

report_dir: str

vulnerability_df: DataFrame | None

sacroml.attacks.meta_attack.MIA_ATTACKS: set[str] = {'lira', 'qmia'}

Subset of supported attacks that produce membership-inference scores.

sacroml.attacks.meta_attack.SUPPORTED_ATTACKS: set[str] = {'lira', 'qmia', 'structural'}

Attacks that expose per-record vulnerability scores.