tarantool / luajit
88%
tarantool/master: 93%

LAST BUILD BRANCH: skaplun/gh-noticket-perf-compare
DEFAULT BRANCH: tarantool/master
Repo Added
Files 89
Badge
Embed ▾
README BADGES
x

If you need to use a raster PNG badge, change the '.svg' to '.png' in the link

Markdown

Textile

RDoc

HTML

Rst

LAST BUILD ON BRANCH skaplun/gh-noticket-perf-compare
branch: SELECT
CHANGE BRANCH
x

coverage: 88.195% (-5.2%) from 93.43%
32949892471

push

github

Buristan 
perf: add helper for benchmark results comparison

This patch adds the helper, which is similar to the Google-benchmark
compare.py script [1]. Nevertheless, it has the following semantic
changes:

* compare.py compares the absolute time of the benchmark, which may lead
  to confusion when the time is autotuned to be sure that the benchmark
  runs the minimum required time amount. Hence, this script compares the
  items_per_second metric to check the performance difference even for
  the same absolute times of the benchmarks.
* compare.py compares only 2 files, while this script allows comparing
  directories filled with the same benchmarks (checks intersection with
  warning of unmatched benchmarks), which allows to see the full
  statistic of the patch for the suite.
* This helper doesn't allow running the benchmarks to compare. It
  proceeds only with the given results.
* There is no support for U test.
* --alpha flag allows setting the relative difference threshold from
  which results are considered as changed, instead of the p-value for
  U test.
* The geomean of the benchmarks can be hidden by the flag
  --hide_aggregates, -i.
* The output may be filtered to contain only changed benchmarks by the
  option --changes_only, -c.

For more details, see help in the script.

[1]: https://github.com/google/benchmark/blob/267a11154/tools/compare.py

12756 of 15761 branches covered (80.93%)

Branch coverage included in aggregate %.

21923 of 23560 relevant lines covered (93.05%)

3984732.76 hits per line

Relevant lines Covered
23560 RELEVANT LINES 21923 COVERED LINES
3984732.76 HITS PER LINE
Source Files on tarantool/master

Recent builds

Builds Branch Commit Type Ran Committer Via Coverage
32949892471 skaplun/gh-noticket-perf-compare perf: add helper for benchmark results comparison This patch adds the helper, which is similar to the Google-benchmark compare.py script [1]. Nevertheless, it has the following semantic changes: * compare.py compares the absolute time of the ben... push Buristan github
88.19
32345955820 tmr_g/lj-noticket-fix-test-build-mac ci: added a common way to disable perf workflow This allows a quick disable of all but one workflow to debug a new CI behavior. The way is to substitue '**-notest' with 'your-branch-name' temporarily on other workflows while debugging the target... push tmr-g github
88.26
32254758341 tmr_g/lj-noticket-fix-test-build-mac wip push tmr-g github
88.19
32252517825 tmr_g/lj-noticket-fix-test-build-mac wip push tmr-g github
88.13
32173923370 tmr_g/lj-noticket-ci-build-mac ci: unused gcc removed from macOS setup The brew gcc installation is useless, because cmake uses the clang compiler, which is already present in the system. Also, the clang is better than gcc for macOS. push tmr-g github
93.41
31216599541 tmr_g/lj-noticket-ci-build-mac ci: unused gcc removed from macOS setup The brew gcc installation is useless, because cmake use the clang compiler which is already present in the system. Also, the clang is better than gcc for macOS. push tmr-g github
93.36
31112893780 skaplun/lj-noticket-stitch-growbot-errmem WIP: check stitch growbot push Buristan github
93.44
31103628623 tmr_g/lj-noticket-ci-build-mac wip push tmr-g github
93.4
31041410074 tmr_g/lj-noticket-fix-test-build-mac ci: added a common way to disable perf workflow This allows a quick disable of all but one workflow to debug a new CI behavior. The way is to substitue '**-notest' with 'your-branch-name' temporarily on other workflows while debugging the target... push tmr-g github
93.39
30653519284 tmr_g/lj-noticket-fix-test-build-mac ci: added a common way to disable perfomance workflow This allow a quick disable of all but one workflow to debug new ci behavior. The way is to substitue '**-notest' with 'your-branch-name' temporarily on other workflows while debugging the tar... push tmr-g github
93.42
See All Builds (2280)