gwicke / restbase

94%

master: 92%

LAST BUILD ON BRANCH test_failures

branch: test_failures

CHANGE BRANCH

x

Reset

• test_failures
• Pchelolo-access_check
• Pchelolo-redirect_docs
• Pchelolo-security_filter
• allow_pipelining
• api-user-agent
• api_cleanup
• aqs_caching
• background_race
• bump_preq
• cache_control_override
• cassandra_bump
• cassandra_driver_v2
• clarify_html_to_wikitext_selser_params
• clean-up-tid-requests
• compression
• config_cleanup
• container_travis
• content_type_enforcement
• cscott-quoted-content-type
• d00rman-rb-mod-cass-table
• dataparsoid-listing
• dependency_update
• description_update
• description_update_2
• disable-swagger-validator
• disable_nocache_instance_tests
• doc_updates
• document_metric_rate_limits
• dont_coerce_json_post
• enforce_parsoid_content_types
• etag_check
• etags
• fix_example_config
• fix_nc
• fixit
• forward_user_agent_to_parsoid
• hacky_listings
• html_purging_caching
• ipv6_embedded_support
• key_rev_value
• key_value_timeseries
• kv_cache
• limit-syntax-highlight-size
• list_revisions_once
• listing_clarification
• local_config_example
• log_bodyOnly
• master
• math_cache_control
• mathoid_caching
• metric_prefix_fix
• metric_trees
• mobileapps_varnish_caching
• mod_cleanup
• module_option_templating
• module_sharing_fix
• modules
• more_client_info_in_log
• more_efficient_revision_query
• more_reliable_parsoid_change_detection
• no_caching_for_non_canonical_requests
• no_parsoid_retry
• nocache-headers
• node_4.1
• node_5_testing
• normalize_title
• optimized_templating
• package-cleanup
• pageviews_proxy
• paging
• parsoid_bucket_type_option
• pass_original_previous_content_parsoid
• pdf_encoding
• pdf_filename_encoding_T171747
• preserve_parameters_in_path_by_default
• printable_pdf
• private_to_internal
• promise_chain_opt
• reqid_failure
• request_counts
• require_tid_if_revision
• rerender
• rerender_blacklist
• rest.wikimedia.org_sunset
• rest_wikimedia_org_logging
• revert_mobile_config_change
• review/mforns/366852
• robots_txt
• sampled_normalization_logging
• sampled_version_warnings
• scrubWikitext
• section-api
• security_header_filter
• security_headers
• selser
• service-runner
• simple_config_example_yaml
• siteinfo_fetch_failure_code
• spec_basepath
• spec_cleanup
• spec_merging
• spec_modules
• spec_swagger_ui_cleanup
• spec_ui
• special-wikis
• statsd-interface
• stream_response_support
• streaming_chunk_responses
• swagger-paths
• sys_filter
• table_spec_move
• temporarily_limit_commons_redirects_to_app
• test_iojs
• textextracts
• tighten_csp
• title_data-parsoid_cleanup
• transform_rate_limits
• update_cassandra
• update_cassandra_backend
• update_parsoid_version
• update_security_headers
• use_412_on_precondition_failure
• use_child_for_logging_and_metrics
• v0.7.11
• v0.9.1
• x-textarea
• xff

Build # 635

Build Type

push

travis-ci

Committed by gwicke

Commit Message

Make tests more reliable

We have recently seen a lot of spurious travis failures, which consumed a lot
of time for re-running tests & checking whether there were actual failures or
not.

Most of these spurious failures were related to local / remote request
classification, which is based on execution logs. The same tests are
consistently passing locally, so it seems likely that we are seeing some kind
of race condition, which is more likely to trigger when tests are executed
slowly in travis.

This patch changes these tests to check for equality of tids instead. While this
does not directly check for presence of local / remote requests, it does
indirectly test that the content is saved & then served from storage in
subsequent requests.

Another class of failures came from the event production test, which only left
100ms for async request processing, before testing that the event had indeed
been processed. This was not always enough in the travis environment, so this
patch bumps up the timeout to 2000ms.

Run Details

1181 of 1263 relevant lines covered (93.51%)

135.1 hits per line