ssoloff / dice-server-js / 421
100%
master: 100%

LAST BUILD BRANCH: develop
DEFAULT BRANCH: master
Ran
Jobs 1
Files 21
Run time 1s
Badge
Embed ▾
README BADGES
x

If you need to use a raster PNG badge, change the '.svg' to '.png' in the link

Markdown

Textile

RDoc

HTML

Rst
421

push

travis-ci

ssoloff 
Do not run coverage during dev task

There is a problem, specific to our setup, running coverage
continuously, as occurs with the dev task.  During the first coverage
run, everything is fine.  However, on second and subsequent runs,
coverage decreases below the acceptable threshold because the
dice-expression-parser-utils module is no longer covered.  This change
simply runs the unit tests without coverage whenever the unit tests
would be run continuously within a single Gulp process.

After researching the issue, I discovered two problems: one unrelated to
the above issue, and the other that I believe to be the cause.

First, when running Istanbul continuously, we must use a different
coverage variable for each run.  If we don't, old coverage information
is carried from run to run.  This would have the effect that if a change
was made that caused previously-covered code to no longer be covered,
Istanbul would still report the code as covered due to previous runs.
This has nothing to do with the issue described in this commit, but if
we ever do run coverage continuously in the future, specifying a custom
coverage variable (see the gulp-istanbul docs) will be required.

The second problem is, I believe, the cause of the decrease in coverage
on second and subsequent runs when Istanbul is run continuously.

While debugging the issue, I observed that dice-expression-parser-utils
gets instrumented and cached by gulp-istanbul on the first run.  However,
on the second and subsequent runs, istanbul no longer asks gulp-istanbul
to provide the instrumented code.  I believe this may have something to
do with the require() cache, such that istanbul won't ask gulp-istanbul
to transform the code if the require() implementation is never called.
I'm not sure what causes this because gulp-istanbul does clear the
require() cache for a file when that file is piped through
hookRequire().

I think the problem with require() and the cache has to do with the fact
that we are excluding the dice-expression-parser module from coverage.
This module is the only module that uses the
dice-expression-parser-utils module, which is the module that loses
coverage on the second and subsequent runs.  If I do not exclude
dice-expression-parser from coverage, the drop in coverage on the second
and subsequent runs does not occur (although the overall coverage for
all runs decreases because dice-expression-parser has lower coverage).

So, I think there's something about excluding a module from coverage
that affects how either the gulp-istanbul or require() caches are
managed, such that, in this specific case, the
dice-expression-parser-utils module does not get instrumented on the
second and subsequent runs.

I tried to reproduce this in an MCVE that mimiced the module exclusion
and dependency graph but was unsuccessful.

614 of 645 relevant lines covered (95.19%)

14.07 hits per line

Jobs
ID Job ID Ran Files Coverage
1 421.1 0
95.19
 Travis Job 421.1
Source Files on build 421
Detailed source file information is not available for this build.