meta-pytorch / opacus / 22513398481 / 3
80%
main: 80%

DEFAULT BRANCH: main
Ran
Files 72
Run time 3s
Badge
Embed ▾
README BADGES
x

If you need to use a raster PNG badge, change the '.svg' to '.png' in the link

Markdown

Textile

RDoc

HTML

Rst
coverage: 47.765% (-0.2%) from 47.934%
22513398481.3

push

github

meta-codesync[bot] 
Add arithmetic operations support to `DPTensorFastGradientClipping` (#805)

Summary:
Based on the latest instructions and the `.github/PULL_REQUEST_TEMPLATE.md`, here is the revised pull request text:

 ---

## Types of changes

- [ ] Bug fix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
- [ ] Docs change / refactoring / dependency upgrade

## Motivation and Context / Related issue

This PR adds support for arithmetic operations on `DPTensorFastGradientClipping` and improves the compatibility of `DPLossFastGradientClipping` with external frameworks and custom loss functions.

**Technical Changes:**
- **Arithmetic Operator Overloading**: Implemented `__add__`, `__radd__`, `__sub__`, `__rsub__`, `__mul__`, `__rmul__`, `__truediv__`, `__neg__`, and `detach()` for `DPTensorFastGradientClipping`. These changes allow for loss manipulation (e.g., weighting or combining multiple losses) while ensuring the two-pass backward mechanism required for Ghost Clipping remains intact.
- **Loss Reduction Management**:
    - Updated `DPLossFastGradientClipping` to save and restore the `reduction` attribute of the criterion during the per-sample loss computation in `__call__`.
    - Explicitly set the `reduction` attribute on the criterion if it is missing during initialization. This addresses cases in frameworks like `transformers` where custom loss implementations may not define a `reduction` attribute.
- **Verification**: The implementation has been verified to work with the HuggingFace `Trainer` in small-scale training runs using Fast Gradient Clipping.

## How Has This Been Tested (if it applies)

- **Unit Tests**: Added 9 tests to `opacus/tests/grad_sample_module_fast_gradient_clipping_test.py` covering arithmetic operations, `item()`, and `detach()` for `DPTensorFastGradientClipping`.
- **Regression Testing**: Confi... (continued)

1496 of 3132 relevant lines covered (47.77%)

0.48 hits per line

Source Files on job run-3 - 22513398481.3