riskcal.calibration

Core calibration interface

Generic calibration framework.

riskcal.calibration.calibrate_parameter(evaluator, target, config=None, parameter_name='noise_multiplier')[source]

Generic privacy parameter calibration algorithm.

Finds the parameter value such that the privacy guarantee (as measured by the evaluator) meets or exceeds the specified target.

This is the core calibration algorithm that works with any privacy mechanism, provided you can supply an evaluator function that computes privacy metrics for a given parameter value.

Parameters:

  • evaluator (PrivacyEvaluator) – Callable that maps parameter_value → PrivacyMetrics. Should compute the relevant privacy metrics (advantage, beta, epsilon) for a given parameter value.

  • target (CalibrationTarget) – Target privacy level to calibrate to. Specifies what metric to optimize (advantage, err_rates, or epsilon_delta) and the target value.

  • config (CalibrationConfig | None) – Calibration configuration (search bounds, tolerances, etc.). If None, uses default CalibrationConfig().

  • parameter_name (str) – Name of the parameter being calibrated (for documentation). Default: “noise_multiplier”

Returns:

  • parameter_value: Calibrated parameter value

  • parameter_name: Name of the calibrated parameter

  • achieved_*: Actually achieved privacy metrics at this parameter value

  • converged: Whether calibration converged

  • iterations: Number of search iterations

Return type:

CalibrationResult containing

Raises:

  • ValueError – If target specification is invalid or inconsistent

  • RuntimeError – If calibration fails to converge

Example

>>> # Define evaluator for DP-SGD
>>> def evaluate_dpsgd(noise_mult):
...     pld = create_dpsgd_pld(noise_mult, sample_rate=0.002, num_steps=10000)
...     advantage = get_advantage_from_pld(pld)
...     return PrivacyMetrics(advantage=advantage)
>>>
>>> # Calibrate to advantage target
>>> target = CalibrationTarget(kind='advantage', advantage=0.1)
>>> result = calibrate_parameter(evaluate_dpsgd, target)
>>> print(f"Noise: {result.parameter_value:.3f}")

Notes

  • Uses binary search for monotonic objectives (advantage, epsilon, beta)

  • Assumes higher parameter values → stronger privacy (lower advantage/epsilon/beta) For parameters where lower is better, adjust bounds accordingly

class riskcal.calibration.CalibrationTarget(kind, advantage=None, alpha=None, beta=None, epsilon=None, delta=None)[source]

Bases: object

Specification of target privacy level to calibrate to.

kind: Literal['advantage', 'err_rates', 'epsilon_delta']
advantage: float | None = None
alpha: float | None = None
beta: float | None = None
epsilon: float | None = None
delta: float | None = None
__init__(kind, advantage=None, alpha=None, beta=None, epsilon=None, delta=None)
class riskcal.calibration.CalibrationConfig(increasing=True, param_min=0.0, param_max=100.0, target_tol=0.001, param_tol=0.001, max_iterations=100)[source]

Bases: object

Configuration for calibration search.

increasing: bool = True
param_min: float = 0.0
param_max: float = 100.0
target_tol: float = 0.001
param_tol: float = 0.001
max_iterations: int = 100
__init__(increasing=True, param_min=0.0, param_max=100.0, target_tol=0.001, param_tol=0.001, max_iterations=100)
class riskcal.calibration.CalibrationResult(parameter_value, parameter_name='noise_multiplier', achieved_advantage=None, achieved_alpha=None, achieved_beta=None, achieved_epsilon=None, achieved_delta=None, converged=True, iterations=0, method='generic', metadata=None)[source]

Bases: object

Result of parameter calibration.

parameter_value: float
parameter_name: str = 'noise_multiplier'
achieved_advantage: float | None = None
achieved_alpha: float | None = None
achieved_beta: float | None = None
achieved_epsilon: float | None = None
achieved_delta: float | None = None
converged: bool = True
iterations: int = 0
method: str = 'generic'
metadata: dict | None = None
property noise_multiplier: float

return parameter_value as noise_multiplier.

Type:

Backward compatibility

__init__(parameter_value, parameter_name='noise_multiplier', achieved_advantage=None, achieved_alpha=None, achieved_beta=None, achieved_epsilon=None, achieved_delta=None, converged=True, iterations=0, method='generic', metadata=None)
class riskcal.calibration.PrivacyMetrics(advantage=None, alpha=None, beta=None, epsilon=None, delta=None, metadata=None)[source]

Bases: object

Privacy metrics computed for a given parameter value.

advantage: float | None = None
alpha: float | None = None
beta: float | None = None
epsilon: float | None = None
delta: float | None = None
metadata: dict | None = None
__init__(advantage=None, alpha=None, beta=None, epsilon=None, delta=None, metadata=None)
class riskcal.calibration.PrivacyEvaluator(*args, **kwargs)[source]

Bases: Protocol

Protocol for functions that evaluate privacy metrics for given parameter value.

__call__(parameter_value)[source]

Evaluate privacy metrics for given parameter value.

Parameters:

parameter_value (float) – Value of the parameter being calibrated (e.g., noise_multiplier, epsilon, sample_rate)

Returns:

PrivacyMetrics containing computed metrics

Return type:

PrivacyMetrics

__init__(*args, **kwargs)

DP-SGD calibration

Specialized functions for calibrating DP-SGD noise multipliers.

riskcal.calibration.find_noise_multiplier_for_advantage_dpsgd(advantage, sample_rate, num_steps, grid_step=0.0001, advantage_tol=0.001, noise_min=0.1, noise_max=50.0, advantage_error=None, mu_error=None, mu_min=None, mu_max=None)

Calibrate DP-SGD noise to target advantage (legacy interface).

Deprecated since version 1.2.0: Legacy parameter names (advantage_error, mu_error, mu_min, mu_max) will be removed in version 2.0.0.

Finds minimum noise_multiplier such that advantage ≤ target.

Parameters:

  • advantage (float) – Target advantage bound in [0, 1].

  • sample_rate (float) – Poisson sampling rate.

  • num_steps (int) – Number of DP-SGD steps.

  • grid_step (float) – PLD discretization interval.

  • advantage_tol (float) – Convergence tolerance for advantage.

  • noise_min (float) – Lower bound for search.

  • noise_max (float) – Upper bound for search.

Returns:

Calibrated noise_multiplier (float).

Return type:

float

Example

>>> noise = find_noise_multiplier_for_advantage(
...     advantage=0.1,
...     sample_rate=0.002,
...     num_steps=10000
... )
>>> print(f"Use noise_multiplier: {noise:.3f}")

Note

For new code, consider using the generic interface: >>> from riskcal.calibration import calibrate_parameter, CalibrationTarget >>> evaluator = create_dpsgd_evaluator(sample_rate=0.002, num_steps=10000) >>> target = CalibrationTarget(kind=’advantage’, advantage=0.1) >>> result = calibrate_parameter(evaluator, target)

riskcal.calibration.find_noise_multiplier_for_err_rates_dpsgd(alpha, beta, sample_rate, num_steps, grid_step=0.0001, beta_tol=0.001, noise_min=0.1, noise_max=50.0, beta_error=None, mu_error=None, mu_min=None, mu_max=None)

Calibrate DP-SGD noise to target error rates (legacy interface).

Deprecated since version 1.2.0: Legacy parameter names (beta_error, mu_error, mu_min, mu_max) will be removed in version 2.0.0.

Finds minimum noise_multiplier such that beta(alpha) ≤ target_beta.

Parameters:

  • alpha (float) – Target false positive rate (FPR) in [0, 1].

  • beta (float) – Target false negative rate (FNR) in [0, 1].

  • sample_rate (float) – Poisson sampling rate.

  • num_steps (int) – Number of DP-SGD steps.

  • grid_step (float) – PLD discretization interval.

  • beta_tol (float) – Convergence tolerance for beta.

  • noise_min (float) – Lower bound for search.

  • noise_max (float) – Upper bound for search.

Returns:

Calibrated noise_multiplier (float).

Return type:

float

riskcal.calibration.get_advantage_for_dpsgd(noise_multiplier, sample_rate, num_steps, grid_step=0.0001)[source]

Compute advantage for DP-SGD with given parameters.

Parameters:

  • noise_multiplier (float) – Noise scale parameter.

  • sample_rate (float) – Poisson sampling rate.

  • num_steps (int) – Number of DP-SGD steps.

  • grid_step (float) – PLD discretization interval.

Returns:

Attack advantage value.

Return type:

float

riskcal.calibration.get_beta_for_dpsgd(noise_multiplier, sample_rate, num_steps, alpha, grid_step=0.0001)[source]

Compute beta (FNR) for DP-SGD at given alpha (FPR).

Parameters:

  • noise_multiplier (float) – Noise scale parameter.

  • sample_rate (float) – Poisson sampling rate.

  • num_steps (int) – Number of DP-SGD steps.

  • alpha (float | ndarray) – False positive rate(s) in [0, 1]. Can be scalar or array.

  • grid_step (float) – PLD discretization interval.

Returns:

False negative rate(s) corresponding to input alpha.

Return type:

float | ndarray

riskcal.calibration.create_dpsgd_evaluator(sample_rate, num_steps, grid_step=0.0001, target_alpha=None)[source]

Create a privacy evaluator for DP-SGD.

Returns a function that maps noise_multiplier → PrivacyMetrics for DP-SGD with the specified parameters.

Parameters:

  • sample_rate (float) – Poisson sampling rate (typically batch_size / dataset_size).

  • num_steps (int) – Number of DP-SGD steps (typically num_epochs * steps_per_epoch).

  • grid_step (float) – Discretization interval for PLD computation.

  • target_alpha (float | None) – If provided, evaluator will compute beta at this alpha value. Required for err_rates calibration.

Returns:

PrivacyEvaluator function that computes metrics for given noise_multiplier.

Return type:

PrivacyEvaluator

Example

>>> evaluator = create_dpsgd_evaluator(sample_rate=0.002, num_steps=10000)
>>> metrics = evaluator(noise_multiplier=1.0)
>>> print(f"Advantage: {metrics.advantage:.4f}")
riskcal.calibration.create_dpsgd_epsilon_evaluator(sample_rate, num_steps, target_delta, grid_step=0.0001)[source]

Create an epsilon-delta evaluator for DP-SGD.

Returns a function that maps noise_multiplier → PrivacyMetrics with epsilon for the specified delta.

Parameters:

  • sample_rate (float) – Poisson sampling rate.

  • num_steps (int) – Number of DP-SGD steps.

  • target_delta (float) – Delta parameter for epsilon computation.

  • grid_step (float) – Discretization interval for PLD computation.

Returns:

PrivacyEvaluator that computes epsilon at target_delta.

Return type:

PrivacyEvaluator

Blackbox calibration

Calibration using privacy profiles.

riskcal.calibration.find_noise_multiplier_for_epsilon_delta(accountant, sample_rate, num_steps, epsilon, delta, eps_error=0.001, mu_error=0.1, mu_min=0.05, mu_max=100.0, **accountant_kwargs)[source]

Find a noise multiplier that satisfies a given target epsilon.

Adapted from https://github.com/microsoft/prv_accountant/blob/main/prv_accountant/dpsgd.py

Parameters:

  • accountant (Type) – Opacus-compatible accountant class.

  • sample_rate (float) – Probability of a record being in batch for Poisson sampling.

  • num_steps (int) – Number of optimization steps.

  • epsilon (float) – Desired target epsilon.

  • delta (float) – Value of DP delta.

  • eps_error (float) – Numeric threshold for convergence in epsilon.

  • mu_error (float) – Numeric threshold for convergence in mu / noise multiplier.

  • mu_min (float) – Minimum value of noise multiplier of the search.

  • mu_max (float) – Maximum value of noise multiplier of the search.

  • **accountant_kwargs – Parameters passed to the accountant’s get_epsilon.

Returns:

Calibrated noise_multiplier (float).

Return type:

float

riskcal.calibration.find_noise_multiplier_for_advantage_blackbox(accountant, advantage, sample_rate, num_steps, eps_error=0.001, mu_error=0.1, mu_min=0.05, mu_max=100.0, **accountant_kwargs)

Find a noise multiplier that satisfies given levels of attack advantage.

Parameters:

  • accountant (Type) – Opacus-compatible accountant class.

  • advantage (float) – Attack advantage bound.

  • sample_rate (float) – Probability of a record being in batch for Poisson sampling.

  • num_steps (int) – Number of optimization steps.

  • eps_error (float) – Numeric threshold for convergence in epsilon.

  • mu_error (float) – Numeric threshold for convergence in mu / noise multiplier.

  • mu_min (float) – Minimum value of noise multiplier of the search.

  • mu_max (float) – Maximum value of noise multiplier of the search.

  • **accountant_kwargs – Parameters passed to the accountant’s get_epsilon.

Returns:

Calibrated noise_multiplier (float).

Return type:

float

riskcal.calibration.find_noise_multiplier_for_err_rates_blackbox(accountant, alpha, beta, sample_rate, num_steps, delta_error=0.01, eps_error=0.001, mu_min=0.05, mu_max=100.0, method='bounded', **accountant_kwargs)

Find a noise multiplier that limits attack FPR/FNR rates.

Requires minimizing the function find_noise_multiplier(delta) over all delta. Currently, only the bounded method is supported to do this minimization.

Parameters:

  • accountant (Type) – Opacus-compatible accountant class.

  • alpha (float) – Attack FPR bound.

  • beta (float) – Attack FNR bound.

  • sample_rate (float) – Probability of a record being in batch for Poisson sampling.

  • num_steps (int) – Number of optimization steps.

  • delta_error (float) – Error allowed for delta used for calibration.

  • eps_error (float) – Error allowed for final epsilon.

  • mu_min (float) – Minimum value of noise multiplier of the search.

  • mu_max (float) – Maximum value of noise multiplier of the search.

  • method (str) – Optimization method. Only [‘bounded’] supported for now.

  • **accountant_kwargs – Parameters passed to the accountant’s get_epsilon.

Returns:

CalibrationResult with noise_multiplier, calibration_epsilon, and calibration_delta.

Return type:

CalibrationResult

Note

This is slower than DP-SGD direct method as it requires optimization over delta parameter and conversion between representations.

riskcal.calibration.create_accountant_evaluator(accountant_class, sample_rate, num_steps, target_delta=None, target_alpha=None, **accountant_kwargs)[source]

Create a privacy evaluator from an Opacus-compatible accountant.

Returns a function that uses the accountant’s epsilon-delta interface to evaluate privacy for a given noise level.

Parameters:

  • accountant_class (Type) – Opacus-compatible accountant class (e.g., RDPAccountant).

  • sample_rate (float) – Poisson sampling rate.

  • num_steps (int) – Number of steps.

  • target_delta (float | None) – Delta for epsilon computation (required for epsilon_delta calibration).

  • target_alpha (float | None) – Alpha for beta computation (required for err_rates calibration).

  • **accountant_kwargs – Additional arguments passed to accountant’s get_epsilon.

Returns:

PrivacyEvaluator function.

Return type:

PrivacyEvaluator

Note

For err_rates calibration via accountants, this uses conversion from (epsilon, delta) to (alpha, beta), which may be slower than direct PLD methods.