meta-pytorch / opacus / 22792141746 / 3
80%
main: 80%

DEFAULT BRANCH: main
Ran
Files 135
Run time 3s
Badge
Embed ▾
README BADGES
x

If you need to use a raster PNG badge, change the '.svg' to '.png' in the link

Markdown

Textile

RDoc

HTML

Rst
coverage: 79.035% (+0.9%) from 78.103%
22792141746.3

push

github

meta-codesync[bot] 
Fix adaptive noise accounting (#807)

Summary:
## Types of changes

- [x] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
- [ ] Docs change / refactoring / dependency upgrade

## Motivation and Context / Related issue
This PR fixes incorrect privacy accounting for `AdaClipDPOptimizer`.

The AdaClip optimizer internally adjusts the `noise_multiplier` based on Theorem 1 from the [AdaClip paper](https://arxiv.org/pdf/1905.03871.pdf):
`σ_eff = (σ^-2 - (2σ_u)^-2)^(-1/2)`

where `σ` is the user-provided noise multiplier and `σ_u` is the `unclipped_num_std` parameter.

The adjusted `σ_eff` is used internally for noise generation during training. However, privacy accountants (rdp, prv,...) were using this adjusted value instead of the original user-provided `σ` for privacy budget calculations, resulting in **incorrect epsilon values**.

Introduced an `accounting_noise_multiplier` property that:
- Returns `noise_multiplier` for standard `DPOptimizer` (backward compatible)
- Returns the original user-provided value for `AdaClipDPOptimizer` (before adjustment)
- Is used by the accountant hook for correct privacy accounting

The internal noise generation continues to use the adjusted value as intended by the AdaClip algorithm, while privacy accounting now uses the correct original value.
## Update!!
As of commit [1f9283a](https://github.com/meta-pytorch/opacus/pull/807/commits/1f9283ab3)
Refactored the noise adjustment to be calculated **locally within the `add_noise()` method** where it's actually needed, rather than modifying `self.noise_multiplier` at initialization.

Now:
- `self.noise_multiplier` always holds the **original user-provided value** (σ)
- The adjusted value (σ_eff) is calculated on-the-fly only when adding noise to gradients
- Privacy accountants automatically ... (continued)

5798 of 7336 relevant lines covered (79.03%)

0.79 hits per line

Source Files on job run-2 - 22792141746.3