• Home
  • Features
  • Pricing
  • Docs
  • Announcements
  • Sign In

michelole / trec-pm
23%
master: 23%

Build:
Build:
LAST BUILD BRANCH: stats
DEFAULT BRANCH: master
Repo Added 28 Jun 2019 04:57PM UTC
Files 206
Badge
Embed ▾
README BADGES
x

If you need to use a raster PNG badge, change the '.svg' to '.png' in the link

Markdown

Textile

RDoc

HTML

Rst

LAST BUILD ON BRANCH stats
branch: stats
CHANGE BRANCH
x
Reset
  • stats
  • issue-13
  • issue-28
  • issue-48
  • issue-50
  • issue-52
  • issue-56
  • master
  • plots

pending completion
64

push

travis-ci-com

michelole
Add support for paired statistical tests

The current [Python code for statistical testing](https://github.com/JULIELab/trec-pm/blob/master/notebooks/sigir2020-notebooks/testing/StatisticalTests.ipynb) uses the `randtest` library for approximate randomization, which assumes groups are independent.

However, our experimental units (topics) are paired (i.e., we run all experiments on all of them) and therefore there is a dependence among runs.

This commit reimplements statistical testing using the popular R `coin` library on top of the existing code for parsing `trec_eval` and `sampleval` files.

1729 of 7571 relevant lines covered (22.84%)

0.46 hits per line

Relevant lines Covered
Build:
Build:
7571 RELEVANT LINES 1729 COVERED LINES
0.46 HITS PER LINE
Source Files on stats
Detailed source file information is not available for this build.

Recent builds

Builds Branch Commit Type Ran Committer Via Coverage
64 stats Add support for paired statistical tests The current [Python code for statistical testing](https://github.com/JULIELab/trec-pm/blob/master/notebooks/sigir2020-notebooks/testing/StatisticalTests.ipynb) uses the `randtest` library for approximate r... push 23 Mar 2020 12:09PM UTC michelole travis-ci-com pending completion  
63 stats Add support for paired statistical tests push 23 Mar 2020 12:05PM UTC michelole travis-ci-com pending completion  
62 stats Add support for statistical tests push 23 Mar 2020 12:04PM UTC michelole travis-ci-com pending completion  
See All Builds (45)
  • Repo on GitHub
STATUS · Troubleshooting · Open an Issue · Sales · Support · CAREERS · ENTERPRISE · START FREE · SCHEDULE DEMO
ANNOUNCEMENTS · TWITTER · TOS & SLA · Supported CI Services · What's a CI service? · Automated Testing

© 2026 Coveralls, Inc