meta-pytorch / opacus / 24873358800 / 1
80%
main: 80%

DEFAULT BRANCH: main
Ran
Files 72
Run time 2s
Badge
Embed ▾
README BADGES
x

If you need to use a raster PNG badge, change the '.svg' to '.png' in the link

Markdown

Textile

RDoc

HTML

Rst
coverage: 47.427% (+0.02%) from 47.41%
24873358800.1

push

github

meta-codesync[bot] 
Non-wrapping mode for better Transformers compatibility (#794)

Summary:
## Types of changes

- [ ] Bug fix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
- [ ] Docs change / refactoring / dependency upgrade

## Motivation and Context / Related issue

The primary goal of this PR is to introduce a **Non-wrapping mode** (controller-based) to Opacus. This feature allows for the computation of per-sample gradients without wrapping the original `nn.Module` in a `GradSampleModule` subclass.

This change addresses critical compatibility issues with third-party libraries—most notably HuggingFace `Transformers` and `Accelerate`—which often rely on strict module hierarchies or perform type checks that are disrupted by standard Opacus wrappers.

**Key Technical Changes:**
- **`GradSampleHooks` Controller**: Introduced a new architecture where gradient sampling hooks are managed by a standalone `GradSampleHooks` object instead of being embedded in a `GradSampleModule` wrapper.
- **`PrivacyEngine` Updates**: Modified `make_private` and `make_private_with_epsilon` to support a `wrap_model` parameter (defaults to `True` for backward compatibility). When `wrap_model=False`, Opacus attaches hooks to the original model and returns it along with a `GradSampleHooks` controller.
- **Enhanced Loss Compatibility**: Updated `DPLossFastGradientClipping` to handle custom loss implementations that lack a `reduction` attribute (a common occurrence in the `transformers` library). The implementation now explicitly sets the expected reduction if missing and restores the previous state after the backward pass.
- **HuggingFace Trainer Integration**: Verified the new mode via successful small-scale training runs using the HuggingFace `Trainer` with Fast Gradient Clipping. A new example (`examples/huggingface_trainer.py`) is included to d... (continued)

1539 of 3245 relevant lines covered (47.43%)

0.47 hits per line

Source Files on job run-3 - 24873358800.1