64
23%
master: 23%

Ran 23 Mar 2020 12:09PM UTC

Jobs 2

Files 206

Run time 53s

Badge

Embed ▾

pending completion

Build # 64

Build Type

push

travis-ci-com

Committed by

michelole

Commit Message

Add support for paired statistical tests

The current [Python code for statistical testing](https://github.com/JULIELab/trec-pm/blob/master/notebooks/sigir2020-notebooks/testing/StatisticalTests.ipynb) uses the `randtest` library for approximate randomization, which assumes groups are independent.

However, our experimental units (topics) are paired (i.e., we run all experiments on all of them) and therefore there is a dependence among runs.

This commit reimplements statistical testing using the popular R `coin` library on top of the existing code for parsing `trec_eval` and `sampleval` files.

Run Details

1729 of 7571 relevant lines covered (22.84%)

0.46 hits per line

Jobs

ID	Job ID	Ran	Files	Coverage
1	64.1	23 Mar 2020 12:09PM UTC	0	22.84	Travis Job 64.1
2	64.2	23 Mar 2020 12:09PM UTC	0	22.84	Travis Job 64.2

Source Files on build 64

Detailed source file information is not available for this build.

michelole / trec-pm / 64
23%
master: 23%

README BADGES
x

Markdown

Textile

RDoc

HTML

Rst

Jobs

Source Files on build 64

michelole / trec-pm / 64 23% master: 23%

README BADGES x

Markdown

Textile

RDoc

HTML

Rst

Jobs

Source Files on build 64

michelole / trec-pm / 64
23%
master: 23%

README BADGES
x