2
23%
master: 23%

Ran 23 Mar 2020 12:09PM UTC

Files 206

Run time 13s

Badge

Embed ▾

Committed 23 Mar 2020 12:03PM UTC coverage: 22.837%. Remained the same

Job # 64.2

Build Type

push

travis-ci-com

Committed by

michelole

Commit Message

Add support for paired statistical tests

The current [Python code for statistical testing](https://github.com/JULIELab/trec-pm/blob/master/notebooks/sigir2020-notebooks/testing/StatisticalTests.ipynb) uses the `randtest` library for approximate randomization, which assumes groups are independent.

However, our experimental units (topics) are paired (i.e., we run all experiments on all of them) and therefore there is a dependence among runs.

This commit reimplements statistical testing using the popular R `coin` library on top of the existing code for parsing `trec_eval` and `sampleval` files.

Run Details

1729 of 7571 relevant lines covered (22.84%)

0.23 hits per line

michelole / trec-pm / 64 / 2
23%
master: 23%

README BADGES
x

Markdown

Textile

RDoc

HTML

Rst

Source Files on job 64.2

michelole / trec-pm / 64 / 2 23% master: 23%

README BADGES x

Markdown

Textile

RDoc

HTML

Rst

Source Files on job 64.2

michelole / trec-pm / 64 / 2
23%
master: 23%

README BADGES
x