cross-browser-tests-runner / cross-browser-tests-runner / 163

93%

push

travis-ci

Code Coverage, Bug fixes, Test Nomenclature

- Testem Runner and its dependencies code coverage improved with
  tests that simulate errors to cover promise rejection handlers that
  are not otherwise straightforward to get invoked through the runner
  workflow
- Native Runner and its dependencies code coverage improved with more
  tests and eliminating dead/unnecessary branches. Needed to add a new
  API to native runner to get currently running runs and tests.
- Nomenclature improvement of integration/ and functional/ tests (unit/
  tests were covered previously)

Fixes
- In builds, there were error about "too many tunnels", which is a core
  issue, never seen locally. The only explanation to this is that even if
  we kill tunnels by SIGINT'ing the local process, probably it's not enough,
  and one needs to take of server side instances as well. For this, the
  tunnel DELETE API is now used. The flow in the Tunnel process abstraction
  is to first do the DELETE for the running tunnel, wait till the status
  of the server side instance is "terminated", and then kill the local
  process (it would be already in a self-destruction mode) and then ensure
  it is killed.
- Native runner manager now can handle bad browser/platform specification
  in input cbtr.json. It intercepts errors thrown while running tests
  (runMultiple/runScriptMultiple), and closes the platform before quitting
- Scripts failure should not lead to no stopping of the script session
  which can make native runner hang indefinitely. The logic in scheduled
  script control flow changed to handle errors before the quit step so
  that the quit step cannot be skipped due to such errors.
- Broke all the integration and functional tests that use the function
  that wraps command under istanbul cover. To fix Windows build error,
  'node' was added to the array this function outputs which is completely
  wrong. Fixed. Need to see what the Windows error was.
- cbtr-init.js related tests are always breaking because the ordering of
  browser versions, devices, os version etc. changes. This happens thanks
  mainly to SauceLabs API not sticking to any order. Now, the generation of
  the browsers config files takes care of sorting all os versions, browser
  versions, and devices fields etc. and all tests that needed to change
  were changed to now always get a sorted list in error case output. The
  browsers conf files have also been modified once these changes were tested.

Changes
- core.process.Process.stop now uses a signal parameter with SIGTERM being
  the default. This came out of testing when we are using istanbul --handle-sigint
  and need to stop the process using SIGINT and not SIGTERM so that istanbul
  can complete generating coverage data.

Cleanups
- All "function(*)" usage replaced with "* =>" notation
- In tests utils.log.error(err) was preventing printing the stack trace.
  Now it has the form utils.log.error('error: ', err)
- Integration/Functional tests won't print coverage related lines
- Since both integration and functional tests utils functions were copies,
  refactored into a common file
- Removal of comments from tests
- SauceLabs tunnel-deps to not include options now
- Removal of unused commented out Bluebird debugging code from tests

445 of 606 branches covered (73.43%)

Branch coverage included in aggregate %.

1535 of 1881 relevant lines covered (81.61%)

1272498.05 hits per line