m3db / m3collector / 173 / 1
97%
master: 97%

DEFAULT BRANCH: master
Ran
Files 10
Run time 21s
Badge
Embed ▾
README BADGES
x

If you need to use a raster PNG badge, change the '.svg' to '.png' in the link

Markdown

Textile

RDoc

HTML

Rst
coverage: 95.125% (+0.1%) from 94.994%
TEST_TIMEOUT_SCALE=20

push

travis-ci

web-flow 
[instance writer] Ensure large batch timers are broken up into multiple buffers (#18)

In a previous diff we added a maxTimerBatchSize field to the instance writer. This field caused the instance writer to break up batch timers into multiple metrics with a maximum of maxTimerBatchSize values. However, while we broke up large timers into multiple metrics, we only checked to see if we should enqueue them after all the smaller metrics had been encoded. This can cause the size of the underlying buffer in the encoder to grow much larger than the flush size though since the size of the buffer will still need to be large enough to support all the timer values in a large batch timer. Furthermore, since these encoders, and therefore their buffers, are pooled, the size of the buffers in the pool would grow over time leading to a large increase in memory usage.

This diff addresses this problem by checking if we should enqueue an encoder in the instance writer after each timer batch has been encoded, instead of after all timer values have been encoded. As a result, the size of the underlying buffer will never grow much larger than the flush size or the size of a timer metric batch. Thus, since we set maxTimerBatchSize to create metrics whose size is roughly that of the flush size, the buffers will not grow much larger than the flush size, curbing the memory growth we saw previously.

800 of 841 relevant lines covered (95.12%)

826.29 hits per line

Source Files on job 173.1 (TEST_TIMEOUT_SCALE=20)