alphagov / accessible-autocomplete / 1148

90%

master: 90%

push

travis-ci-com

Only run aria-live tests in Chrome, bump timeout

## Problem

We were seeing the new integration tests fail as they didn't complete
within the set time limit of 2000ms.

This would happen when they were run through Saucelabs as done on CI
or when you have the Saucelabs set up to run locally. When Saucelabs
is not available, the tests are run through Karma/headless Chrome,
which seems to reliably pass.

## Solution
Solution 3 to keep some test coverage, while also trying to ensure that
our PRs don't start failing due to Saucelabs timer issues. With this in
mind, we should keep an eye on the tests on CI going forward.

## Explored solutions
###  1. Remove the timeout limit
Setting timeout on
[waitUntil](https://webdriver.io/docs/api/browser/waitUntil.html)
is optional.

However removing it throws
```
Promise was rejected for the following reason: timeout
```

### 2. Increase the timeout limit
-Chrome on Saucelabs would intermittently fail within 2000ms; Firefox
and IE would always fail
- On 3000ms Firefox and IE11 would still fail.
- On 4000ms IE11 was flaky.
- IE10 needed at least 6000ms to pass, whilst IE9 wouldn't pass even
at 10000ms.
- Additionally, Firefox would fail intermittently regardless of the timeout
limit set.

### 3. Limit scope of tests
As Chrome seemed most reliable to run the tests with, we only run these tests
through Saucelabs only in Chrome on Windows 10.

However even then we would occassionally see the tests in Chrome fail within
the 2000ms limit.

We increased the limit to 10000ms, at which point no issues were observed.

### 4. Move the tests to /tests/functional
Saucelabs runs tests in tests/integration. karma/headless chrome run tests in
tests/functional.

Tests on the original PR would have been run through karma/headless Chrome as
Saucelabs wasn't available. They seem too flaky to run through Saucelabs because
of timer issues so we investigated moving them to /tests/functional.

However we found that trying to create the int... (continued)

191 of 243 branches covered (78.6%)

245 of 271 relevant lines covered (90.41%)

61.41 hits per line