1
66%
main: 66%

Ran 30 May 2023 12:58AM UTC

Files 250

Run time 4s

Badge

Embed ▾

Committed 29 May 2023 01:50PM UTC coverage: 67.314% (+0.09%) from 67.225%

Job # 5116183077.1

Build Type

push

Committed by

web-flow

Commit Message

fix(bulk): removed buffer max size (#8841)

## Problem

On huge datasets, bulk import crashes with this stack trace:

```
panic: z.Buffer max size exceeded: 68719476736 offset: 68719476718 grow: 51

goroutine 421 [running]:
github.com/dgraph-io/ristretto/z.(*Buffer).Grow(0xc050d17c00, 0x33)
        /home/runner/go/pkg/mod/github.com/dgraph-io/ristretto@v0.1.1/z/buffer.go:180 +0x55b
github.com/dgraph-io/ristretto/z.(*Buffer).SliceAllocate(0xc050d17c00, 0x2f)
        /home/runner/go/pkg/mod/github.com/dgraph-io/ristretto@v0.1.1/z/buffer.go:266 +0x2e
github.com/dgraph-io/dgraph/dgraph/cmd/bulk.(*mapIterator).Next(0xc00e56f410, 0x3?, {0xc034740c40, 0x1d, 0x20})
        /home/runner/work/dgraph/dgraph/dgraph/cmd/bulk/reduce.go:206 +0x126
github.com/dgraph-io/dgraph/dgraph/cmd/bulk.(*reducer).reduce.func2()
        /home/runner/work/dgraph/dgraph/dgraph/cmd/bulk/reduce.go:486 +0x5aa
created by github.com/dgraph-io/dgraph/dgraph/cmd/bulk.(*reducer).reduce
        /home/runner/work/dgraph/dgraph/dgraph/cmd/bulk/reduce.go:476 +0x452
```

## Solution

Removing hard limit of 64GB to buffer sizes avoided panic and
successfully completed the import.

Run Details

58444 of 86823 relevant lines covered (67.31%)

2266339.51 hits per line

dgraph-io / dgraph / 5116183077 / 1
66%
main: 66%

README BADGES
x

Markdown

Textile

RDoc

HTML

Rst

Source Files on job 5116183077.1

dgraph-io / dgraph / 5116183077 / 1 66% main: 66%

README BADGES x

Markdown

Textile

RDoc

HTML

Rst

Source Files on job 5116183077.1

dgraph-io / dgraph / 5116183077 / 1
66%
main: 66%

README BADGES
x