gwicke / restbase

87%

master: 92%

LAST BUILD ON BRANCH rerender_blacklist

branch: rerender_blacklist

CHANGE BRANCH

x

Reset

• rerender_blacklist
• Pchelolo-access_check
• Pchelolo-redirect_docs
• Pchelolo-security_filter
• allow_pipelining
• api-user-agent
• api_cleanup
• aqs_caching
• background_race
• bump_preq
• cache_control_override
• cassandra_bump
• cassandra_driver_v2
• clarify_html_to_wikitext_selser_params
• clean-up-tid-requests
• compression
• config_cleanup
• container_travis
• content_type_enforcement
• cscott-quoted-content-type
• d00rman-rb-mod-cass-table
• dataparsoid-listing
• dependency_update
• description_update
• description_update_2
• disable-swagger-validator
• disable_nocache_instance_tests
• doc_updates
• document_metric_rate_limits
• dont_coerce_json_post
• enforce_parsoid_content_types
• etag_check
• etags
• fix_example_config
• fix_nc
• fixit
• forward_user_agent_to_parsoid
• hacky_listings
• html_purging_caching
• ipv6_embedded_support
• key_rev_value
• key_value_timeseries
• kv_cache
• limit-syntax-highlight-size
• list_revisions_once
• listing_clarification
• local_config_example
• log_bodyOnly
• master
• math_cache_control
• mathoid_caching
• metric_prefix_fix
• metric_trees
• mobileapps_varnish_caching
• mod_cleanup
• module_option_templating
• module_sharing_fix
• modules
• more_client_info_in_log
• more_efficient_revision_query
• more_reliable_parsoid_change_detection
• no_caching_for_non_canonical_requests
• no_parsoid_retry
• nocache-headers
• node_4.1
• node_5_testing
• normalize_title
• optimized_templating
• package-cleanup
• pageviews_proxy
• paging
• parsoid_bucket_type_option
• pass_original_previous_content_parsoid
• pdf_encoding
• pdf_filename_encoding_T171747
• preserve_parameters_in_path_by_default
• printable_pdf
• private_to_internal
• promise_chain_opt
• reqid_failure
• request_counts
• require_tid_if_revision
• rerender
• rest.wikimedia.org_sunset
• rest_wikimedia_org_logging
• revert_mobile_config_change
• review/mforns/366852
• robots_txt
• sampled_normalization_logging
• sampled_version_warnings
• scrubWikitext
• section-api
• security_header_filter
• security_headers
• selser
• service-runner
• simple_config_example_yaml
• siteinfo_fetch_failure_code
• spec_basepath
• spec_cleanup
• spec_merging
• spec_modules
• spec_swagger_ui_cleanup
• spec_ui
• special-wikis
• statsd-interface
• stream_response_support
• streaming_chunk_responses
• swagger-paths
• sys_filter
• table_spec_move
• temporarily_limit_commons_redirects_to_app
• test_failures
• test_iojs
• textextracts
• tighten_csp
• title_data-parsoid_cleanup
• transform_rate_limits
• update_cassandra
• update_cassandra_backend
• update_parsoid_version
• update_security_headers
• use_412_on_precondition_failure
• use_child_for_logging_and_metrics
• v0.7.11
• v0.9.1
• x-textarea
• xff

Build # 498

Build Type

push

travis-ci

Committed by gwicke

Commit Message

Add a rerenderBlacklist option for the parsoid module

A few huge & extremely frequently edited pages are using a disproportionate
amount of resources for re-renders in both storage and Parsoid.  While longer
term improvements like https://phabricator.wikimedia.org/T120171 and Parsoid
performance will make it easier to deal with such (ab)use, in the short term
it is useful to disable job queue triggered updates for specific problematic
pages. Regular access to those pages still works, but tends to be rare for
these pages, leading to substantial savings in overall resource usage.

To be clear: This is not meant as a replacement for performance improvements,
but reduces the impact those few pages have right now until efficient
incremental parsing & chunked storage are implemented.

Task: https://phabricator.wikimedia.org/T120971
Task: https://phabricator.wikimedia.org/T120171
Task: https://phabricator.wikimedia.org/T94121
Task: https://phabricator.wikimedia.org/T120972

Run Details

1790 of 2059 relevant lines covered (86.94%)

446.97 hits per line