tarantool / tarantool / 18744257466
88%

DEFAULT BRANCH: master
Ran
Jobs 1
Files 514
Run time 2min
Badge
Embed ▾
README BADGES
x

If you need to use a raster PNG badge, change the '.svg' to '.png' in the link

Markdown

Textile

RDoc

HTML

Rst
coverage: 87.664% (+0.04%) from 87.625%
18744257466

push

github

locker 
box: read snapshot in a distinct thread to improve recovery performance

Validating requests MsgPack and parsing the requests body turns out to
be significant part of recovery in terms of CPU. Let's move it to
a distinct thread. This is similar to iproto and applier where we have
a distinct thread(s) for that.

Snapshot is written in batches of about 128k
(`XLOG_TX_AUTOCOMMIT_THRESHOLD`). The recovery thread reads the batch,
parses its requests and send to TX to process. The maximum number of
read but not processed batches is limited to 2 to make a flow control.

Closes #11888
Part of #11452

============================
= Performance test results =
============================

Recovery performance is evaluated using below perf test. The 1.10.5
performance is taken from the previous evaluations for the issue.

```
tarantool perf/lua/recovery.lua \
  --row_count 10000000 --column_count 10 --recovery_count 5 \
  --preload_script "require('compat').box_recovery_triggers_deprecation = 'new'"
```

Run                 | Perf, krps |
---------------------------------|
master, current     |    1060    |
---------------------------------|
master, patched     |    1800    |
---------------------------------|
1.10.5              |    1315    |

So introducing snapshot reader thread gives 70% boost for recovery from
snapshot. Now master is 37% faster than 1.10.5.

If `box_recovery_triggers_deprecation = 'old'` then performance is only
990 krps.

NO_TEST=performance
NO_DOC=performance

69963 of 117797 branches covered (59.39%)

297 of 310 new or added lines in 4 files covered. (95.81%)

415 existing lines in 19 files now uncovered.

105048 of 119830 relevant lines covered (87.66%)

1378961.82 hits per line

New Missed Lines in Diff

Lines Coverage ∆ File
3
84.89
 0.16% src/box/xlog.c
5
91.03
 -0.3% src/box/memtx_engine.cc
5
96.86
 src/box/xlog_reader.c

Uncovered Existing Lines

Lines Coverage ∆ File
1
90.91
 0.0% src/box/txn_checkpoint.c
1
97.41
 -0.85% src/lib/vclock/vclock.c
1
75.06
 -0.25% src/main.cc
2
95.84
 -0.11% src/box/iproto.cc
2
92.54
 0.09% src/box/vinyl.c
2
85.61
 -0.72% src/lib/core/coio.c
2
91.56
 -0.22% src/lib/swim/swim.c
3
89.47
 0.38% src/box/xlog.h
3
99.23
 -0.0% src/lib/raft/raft.c
4
97.05
 -0.41% src/lib/msgpuck/msgpuck.h
5
97.72
 -0.81% src/lib/salad/rtree.c
12
85.37
 0.0% src/box/xrow.h
15
95.2
 0.03% src/box/raft.c
20
96.12
 -0.21% src/box/relay.cc
34
84.89
 0.16% src/box/xlog.c
41
94.48
 0.01% src/box/txn_limbo.c
71
90.5
 -0.01% src/box/applier.cc
90
91.03
 -0.3% src/box/memtx_engine.cc
106
89.37
 -0.01% src/box/xrow.c
Jobs
ID Job ID Ran Files Coverage
1 18744257466.1 514
87.66
 GitHub Action Run
Source Files on build 18744257466