apache / bookkeeper / 62
72%

DEFAULT BRANCH: master
Ran
Jobs 1
Files 404
Run time 16s
Badge
Embed ▾
README BADGES
x

If you need to use a raster PNG badge, change the '.svg' to '.png' in the link

Markdown

Textile

RDoc

HTML

Rst
62

push

jenkins

Enrico Olivelli 
Issue #791: Eliminate byte[] copies in AddEntry code path

The AddEntry code path in Bookkeeper Client performs several copies of the send buffer before sending it out to bookies, which not only costs CPU cycles for array copying, but also drives up object allocation rate and thus increases young GC frequency.

Currently, the send buffer is prepared by DigestManager.computeDigestAndPackageForSending(), which wraps the netty ByteBuf containing the data together with a (direct) header buffer into a ByteBufList. So far, we haven't yet allocated or copied any new buffers. But in order to send the buffer to the bookies, PendingAddOp.safeRun() will invoke sendWriteRequest() for each bookie in the write set, which will eventually invoke PerChannelBookieClient.addEntry() for each bookie. In here, we are now allocating two new byte[] and copying the buffer twice per bookie for V3 protocol with protobuf:
    byte[] toSendArray = toSend.toArray();      // 1st byte[] allocation and copy
    .setBody(ByteString.copyFrom(toSendArray)); // 2nd byte[] allocation and copy
If for example we're using a write set size of 3, then we allocate 6 byte arrays and copy the same buffer 6 times.

All 6 copies can be eliminated if DigestManager.computeDigestAndPackageForSending() would return a ByteBuf (or ByteBufList) that is backed by an array, which we can access and use without further copying it. PerChannelBookieClient.addEntry() could then wrap this array into a protobuf ByteString without performing any additional copies.

This change eliminates all byte[] allocations and copies in PCBC by:
1. allocating a new (pooled) array-backed heap ByteBuf for header, digest, and data in DigestManager.computeDigestAndPackageForSending()
2. copying the data payload into this newly allocated ByteBuf and returning it wrapped into a ByteBufList of size 1 (instead of header + body separately)
3. enhancing ByteBufList to give access to the underlying array if it consists of only a single By... (continued)

23259 of 32204 relevant lines covered (72.22%)

0.72 hits per line

Jobs
ID Job ID Ran Files Coverage
1 62.1 0
72.22
Source Files on build 62
Detailed source file information is not available for this build.