dgraph-io / dgraph / 5117788574

67%

main: 66%

push

fix(bulk): removed buffer max size (#8841)

## Problem

On huge datasets, bulk import crashes with this stack trace:

```
panic: z.Buffer max size exceeded: 68719476736 offset: 68719476718 grow: 51

goroutine 421 [running]:
github.com/dgraph-io/ristretto/z.(*Buffer).Grow(0xc050d17c00, 0x33)
        /home/runner/go/pkg/mod/github.com/dgraph-io/ristretto@v0.1.1/z/buffer.go:180 +0x55b
github.com/dgraph-io/ristretto/z.(*Buffer).SliceAllocate(0xc050d17c00, 0x2f)
        /home/runner/go/pkg/mod/github.com/dgraph-io/ristretto@v0.1.1/z/buffer.go:266 +0x2e
github.com/dgraph-io/dgraph/dgraph/cmd/bulk.(*mapIterator).Next(0xc00e56f410, 0x3?, {0xc034740c40, 0x1d, 0x20})
        /home/runner/work/dgraph/dgraph/dgraph/cmd/bulk/reduce.go:206 +0x126
github.com/dgraph-io/dgraph/dgraph/cmd/bulk.(*reducer).reduce.func2()
        /home/runner/work/dgraph/dgraph/dgraph/cmd/bulk/reduce.go:486 +0x5aa
created by github.com/dgraph-io/dgraph/dgraph/cmd/bulk.(*reducer).reduce
        /home/runner/work/dgraph/dgraph/dgraph/cmd/bulk/reduce.go:476 +0x452
```

## Solution

Removing hard limit of 64GB to buffer sizes avoided panic and
successfully completed the import.

1 of 1 new or added line in 1 file covered. (100.0%)

58165 of 86823 relevant lines covered (66.99%)

2232922.88 hits per line