cilium / cilium / 9729
39%

DEFAULT BRANCH: master
Ran
Jobs 1
Files 436
Run time 36s
Badge
Embed ▾
README BADGES
x

If you need to use a raster PNG badge, change the '.svg' to '.png' in the link

Markdown

Textile

RDoc

HTML

Rst
coverage: 44.114% (-0.04%) from 44.157%
9729

push

travis-ci-com

ianvernon 
daemon: get list of frontends from ServiceCache before acquiring BPFMapMu

The `ServiceCache` mutex should always be acquired before acquiring the
`BPFMapMu` of the `LoadBalancer`.

The following deadlock was observed in a cilium instance:

We are trying to acquire the `BPFMapMu` in the daemon's `LoadBalancer` to add a
Kubernetes service as observed by the Kubernetes watcher:

```
goroutine 150 [semacquire, 225 minutes]:
sync.runtime_SemacquireMutex(0xc000401d44, 0x0)
        /usr/local/go/src/runtime/sema.go:71 +0x3d
sync.(*Mutex).Lock(0xc000401d40)
        /usr/local/go/src/sync/mutex.go:134 +0x109
sync.(*RWMutex).Lock(0xc000401d40)
        /usr/local/go/src/sync/rwmutex.go:93 +0x2d
main.(*Daemon).svcAdd(0xc0001b83c0, 0xc002d96eb0, 0x10, 0x10, 0xc001afa069, 0x3, 0x2474, 0x38f, 0x4d8bf28, 0x0, ...)
        /go/src/github.com/cilium/cilium/daemon/loadbalancer.go:132 +0x630
main.(*Daemon).addK8sSVCs(0xc0001b83c0, 0xc001badec0, 0x2d, 0xc001bbb000, 0xb, 0x0, 0xc000e73130, 0xc003e5e8a0, 0x0, 0x0)
        /go/src/github.com/cilium/cilium/daemon/k8s_watcher.go:1288 +0xf37
main.(*Daemon).k8sServiceHandler(0xc0001b83c0)
        /go/src/github.com/cilium/cilium/daemon/k8s_watcher.go:1064 +0xa29
created by main.(*Daemon).runK8sServiceHandler
        /go/src/github.com/cilium/cilium/daemon/k8s_watcher.go:1118 +0x3f
```

The controller which makes sure that the daemon's LoadbLanacer maps are in sync
with Kubernetes has possession of the aforementioned `BPFMapMu` mutex, but is
trying to acquire the mutex for the `ServiceCache` after having acquired the
`BPFMapMu`:

```
goroutine 401 [semacquire, 225 minutes]:
sync.runtime_SemacquireMutex(0xc0001b8514, 0x0)
        /usr/local/go/src/runtime/sema.go:71 +0x3d
sync.(*RWMutex).RLock(...)
        /usr/local/go/src/sync/rwmutex.go:50
github.com/cilium/cilium/pkg/k8s.(*ServiceCache).UniqueServiceFrontends(0xc0001b8500, 0x0)
        /go/src/github.com/cilium/cilium/pkg/k8s/service_cache.go:427 +0x37d
main.(*Daemon).syncLBM... (continued)

25287 of 57322 relevant lines covered (44.11%)

1179.85 hits per line

Jobs
ID Job ID Ran Files Coverage
1 9729.1 0
44.11
 Travis Job 9729.1
Source Files on build 9729
Detailed source file information is not available for this build.