• Home
  • Features
  • Pricing
  • Docs
  • Announcements
  • Sign In

michelole / trec-pm
23%
master: 23%

Build:
Build:
LAST BUILD BRANCH: stats
DEFAULT BRANCH: master
Repo Added 28 Jun 2019 04:57PM UTC
Files 206
Badge
Embed ▾
README BADGES
x

If you need to use a raster PNG badge, change the '.svg' to '.png' in the link

Markdown

Textile

RDoc

HTML

Rst

LAST BUILD ON BRANCH stats
branch: stats
CHANGE BRANCH
x
Reset
  • stats
  • issue-13
  • issue-28
  • issue-48
  • issue-50
  • issue-52
  • issue-56
  • master
  • plots

pending completion
64

push

travis-ci-com

michelole
Add support for paired statistical tests

The current [Python code for statistical testing](https://github.com/JULIELab/trec-pm/blob/master/notebooks/sigir2020-notebooks/testing/StatisticalTests.ipynb) uses the `randtest` library for approximate randomization, which assumes groups are independent.

However, our experimental units (topics) are paired (i.e., we run all experiments on all of them) and therefore there is a dependence among runs.

This commit reimplements statistical testing using the popular R `coin` library on top of the existing code for parsing `trec_eval` and `sampleval` files.

1729 of 7571 relevant lines covered (22.84%)

0.46 hits per line

Relevant lines Covered
Build:
Build:
7571 RELEVANT LINES 1729 COVERED LINES
0.46 HITS PER LINE
Source Files on stats
  • List 0
  • Changed 0
  • Source Changed 0
  • Coverage Changed 0
Coverage ∆ File Lines Relevant Covered Missed Hits/Line

Recent builds

Builds Branch Commit Type Ran Committer Via Coverage
64 stats Add support for paired statistical tests The current [Python code for statistical testing](https://github.com/JULIELab/trec-pm/blob/master/notebooks/sigir2020-notebooks/testing/StatisticalTests.ipynb) uses the `randtest` library for approximate r... push 23 Mar 2020 12:09PM UTC michelole travis-ci-com pending completion  
63 stats Add support for paired statistical tests push 23 Mar 2020 12:05PM UTC michelole travis-ci-com pending completion  
62 stats Add support for statistical tests push 23 Mar 2020 12:04PM UTC michelole travis-ci-com pending completion  
See All Builds (45)
  • Repo on GitHub
STATUS · Troubleshooting · Open an Issue · Sales · Support · CAREERS · ENTERPRISE · START FREE · SCHEDULE DEMO
ANNOUNCEMENTS · TWITTER · TOS & SLA · Supported CI Services · What's a CI service? · Automated Testing

© 2025 Coveralls, Inc