4793
94%

Ran 10 Dec 2018 10:27AM UTC

Jobs 1

Files 88

Run time 1min

Badge

Embed ▾

pending completion

Build # 4793

Build Type

push

travis-ci

Committed by

pyokagan

Commit Message

ci: do not retry on test failures

Whenever test failures occur, our Travis and Appveyor configurations
will re-run the entire test script, up to 3 times.

Re-running tests is usually used as a workaround when certain tests are
flaky. However, in this case, it does more harm than good:

  * If the tests are indeed failing due to a problem with the code, the
    build time would still be extremely long due to the unnecessary
    test-retrying. This could cause unnecessary jams in our build
    queues.

  * If we accidentally introduce a flaky test, developers who only test
    via the CI, and do not view the logs of CI jobs that pass, will
    likely not find out since the tests are re-run until they pass
    (unless they happen to fail all 3 times).  For development purposes,
    it is more beneficial if we are alerted to flaky tests as soon as
    possible, preferably before the code is even merged.

  * If we _do_ have flaky tests, the tests should be individually marked
    "to be retried" using JUnit's mechanisms, rather than leaving it to
    the CI configuration. This would both help save build time (since
    only the flaky test would be re-run) and would be more explicit
    (since the flaky test is clearly marked out in code).

As such, let's change our CI configuration to not retry on test
failures.

Run Details

1404 of 1501 relevant lines covered (93.54%)

0.94 hits per line

Jobs

ID	Job ID	Ran	Files	Coverage
1	4793.1	10 Dec 2018 10:27AM UTC	0	93.54	Travis Job 4793.1

Source Files on build 4793

Detailed source file information is not available for this build.

se-edu / addressbook-level4 / 4793
94%

README BADGES
x

Markdown

Textile

RDoc

HTML

Rst

Jobs

Source Files on build 4793

se-edu / addressbook-level4 / 4793 94%

README BADGES x

Markdown

Textile

RDoc

HTML

Rst

Jobs

Source Files on build 4793

se-edu / addressbook-level4 / 4793
94%

README BADGES
x