grpc / grpc-java / #20509
89%
master: 89%

LAST BUILD BRANCH: v1.84.x
DEFAULT BRANCH: master
Ran
Jobs 1
Files 687
Run time 2min
Badge
Embed ▾
README BADGES
x

If you need to use a raster PNG badge, change the '.svg' to '.png' in the link

Markdown

Textile

RDoc

HTML

Rst
coverage: 89.187% (+0.02%) from 89.164%
#20509

push

github

web-flow 
xDS: fix DistributorWatcher thread safety in CertificateProvider (v1.84.x backport) (#13099)

Backport of #13087 to v1.84.x.
---
A gRPC client using xDS-driven mTLS does two things concurrently:

1. Certificate provider: watches cert files on disk, periodically
reloads them, and notifies subscribers when certs change.
2. Channel subscriptions: each gRPC channel that needs mTLS registers a
watcher on the cert provider to get the TLS credentials.

The `DistributorWatcher` sits in the middle and fans out: one cert
provider pushes updates to it, and it distributes to N channel watchers.

I think there are two conditions in this that can lead to a
`ConcurrentModificationException`:

1. `updateSpiffeTrustMap` isn't synchronized (unlike `updateCertificate`
and `updateTrustedRoots`), so it can race against
`addWatcher`/`removeWatcher` from another thread.
2. if callbacks mutate their own subscription. A callchain (line numbers
per 1.83.0 / 1084e9707) for that is:

```
  DistributorWatcher.onError()                              CertificateProvider.java:126
    → for (watcher : downstreamWatchers)                    CertificateProvider.java:127
      → watcher.onError(errorStatus)                        CertificateProvider.java:128
        → DynamicSslContextProvider.onError()               DynamicSslContextProvider.java:143
          → callback.onException(error)                     DynamicSslContextProvider.java:145
            → SslContextProviderSupplier.onException()      SslContextProviderSupplier.java:79
              → releaseSslContextProvider(toRelease)        SslContextProviderSupplier.java:81
                → TlsContextManagerImpl.releaseClient...()  TlsContextManagerImpl.java:78
                  → ReferenceCountingMap.releaseInternal()  ReferenceCountingMap.java:89
                    → if refcount == 0: value.close()       ReferenceCountingMap.java:95
                      → CertProviderSslContextProvider.close()  CertProviderSslContextProvider.... (continued)

38412 of 43069 relevant lines covered (89.19%)

0.89 hits per line

Coverage Regressions

Lines Coverage ∆ File
5
75.0
-3.45% ../servlet/src/main/java/io/grpc/servlet/ServletServerStream.java
2
88.89
-11.11% ../netty/src/main/java/io/grpc/netty/UdsNameResolver.java
1
93.83
-0.62% ../opentelemetry/src/main/java/io/grpc/opentelemetry/OpenTelemetryMetricSink.java
1
92.99
0.93% ../xds/src/main/java/io/grpc/xds/client/ControlPlaneClient.java
1
98.68
2.13% ../xds/src/main/java/io/grpc/xds/internal/security/certprovider/CertificateProvider.java
Jobs
ID Job ID Ran Files Coverage
1 #20509.1 687
89.19
Source Files on build #20509