Recidiviz / recidiviz-data

87%

master: 86%

LAST BUILD ON BRANCH jamwalla/scraper_refactor

branch: jamwalla/scraper_refactor

CHANGE BRANCH

x

Reset

• jamwalla/scraper_refactor
• 121-compare_and_set_snapshot
• 63-migration
• 63-migration-second-pass
• 63-migration-second-pass-final
• add-infraction
• add-local-yaml-docs
• add_booking_ids
• add_database_readme
• add_date_parsing
• add_emacs_backup_to_gitignore
• add_external_unknown_to_enums
• add_fl_osceolo
• add_libs
• add_logging
• add_logging_to_snapshots
• add_migration_aggregate_unique
• add_migration_for_court_type
• add_more_libs
• add_new_lib
• add_ny_site
• add_period_fields_to_master_entities
• add_raw
• add_session_engine
• add_ssl_to_alembic_url
• add_unique_record_assert
• aggregate-schema
• aggregate_tables_migration
• alachua
• all_branches
• all_site_aggr
• allow_single_tests
• api_infer_release
• archonix_fix
• ballard_ky
• base_ii_return
• base_scraper
• bell
• bond_partial_cash_migration
• bond_revoked_migration
• boyle
• breakup-manifest
• breckinridge
• brooks_jeffery
• brooks_jeffrey_fullname
• bucket_size
• builder-getattr
• bullitt2
• ca_site_aggr
• calc-readability
• calculate-cron-and-queue
• cca-log-request-context
• change_concurrent
• charge-converter
• charge_fourth_degree_migration
• ci-builds
• clark_caly_daviess
• clean_up_application_enums
• clean_up_enums
• clean_up_schema_test
• clear_generated_ids
• cli_script
• cloud-iap
• cloud_function
• co-mesa
• conditional-release-recidivism
• consolidate_snapshot_queries
• constants-enum
• convert-override-none
• convert_entity
• cookies
• create-no-vendor
• create_alembic_environment
• create_enum_migration_script
• create_scraper
• create_scraper_base_scraper_test
• create_scraper_class_name
• create_scraper_test
• cron
• cron_mesa
• cssselect_tables
• data_converter
• data_extractor_cell_order
• data_extractor_convert_key_to_cells
• data_extractor_copy_content
• data_extractor_error_checks
• database_readme_fixes
• date_to_time
• dateparser
• datetime
• delete-us-vt
• delete_actions
• deploy_more
• deploy_nassau
• deploy_release
• desoto
• disable_key_conversion
• docker
• double_full_name_assignment
• entity-matching
• entity_match_equality
• enum-errors
• enum-parses
• enum_overrides
• enum_values_test
• ethnicity_as_race
• export-psql
• external_ids_separately
• extract-docket
• extract-docket-module
• find_parent_index_info_index_error
• fipsscript
• fix-app-yaml
• fix-base-scraper
• fix-convert
• fix-create-scraper
• fix-dumb-queue
• fix-encoding
• fix-enum-script
• fix-ii-serialize
• fix-json
• fix-local
• fix-logging
• fix-lumpkin
• fix-monitoring
• fix-pip
• fix-python3
• fix-sessions-stop-iter
• fix-zuercher
• fix_alembic_config
• fix_archonix
• fix_bond
• fix_brooks
• fix_brooks_jeffrey
• fix_bug
• fix_cf_bug
• fix_cron
• fix_data_extractor
• fix_entity
• fix_enum_errors
• fix_find_parent_package_loop
• fix_future_warning
• fix_hawaii
• fix_jailtracker_errors
• fix_ky
• fix_ny
• fix_primary_key_handling
• fix_queue_bucket
• fix_texas
• fix_travis
• fix_type_check
• fix_vuln
• fixfixfix
• flask
• floyd-ga
• gallatin
• generalize-compare-and-set-snapshot-tests
• generalized_scraper
• generate_initial_migration
• generic-cleanup
• generic_extractor
• gunicorn-conf
• hart2
• historical_snapshots_use_scrape_time
• ii-typing
• iml
• increase_instance
• increase_timeout
• infer-log
• inferred_age_comparison
• ingest-info
• ingest-info-typed
• ingest-test-coverage
• ingest-test-coverage-final
• ingest_info_constructor
• ingest_info_converter
• ingest_info_deep_dif
• ingest_info_fields
• ingest_info_proto
• ingest_info_setattr
• ingest_info_test
• initial-params
• issue_49
• issue_848
• jailtracker
• jailtracker_merge
• jailtracker_rate_error_reporting
• jailtracker_scraper
• jamwalla-patch-1
• jamwalla/BOND_TYPE_MAP
• jamwalla/WADANoResultsMessage
• jamwalla/bond_type_and_race_maps
• jamwalla/concatenate_names
• jamwalla/coveralls
• jamwalla/create_run_tools
• jamwalla/create_scraper_ingest_init_file
• jamwalla/cron_feb_5
• jamwalla/cts
• jamwalla/data_extractor_bug
• jamwalla/data_extractor_empty_lookup_keys
• jamwalla/data_extractor_empty_values
• jamwalla/data_extractor_multiple_values
• jamwalla/data_extractor_single_booking
• jamwalla/data_extractor_tests
• jamwalla/data_extractor_types
• jamwalla/dont_convert_css_keys_as_string_keys
• jamwalla/eagle_advantage
• jamwalla/entity_match_errors
• jamwalla/entity_type
• jamwalla/enum_migration
• jamwalla/enum_overrides
• jamwalla/enums_ignore_punctuation
• jamwalla/extractor_multi_key_child_first
• jamwalla/february_4th_scrapers
• jamwalla/fix_brooks_jeffrey
• jamwalla/fix_pa
• jamwalla/fix_run
• jamwalla/gwinnett_fixes
• jamwalla/holds
• jamwalla/html5_scraper
• jamwalla/ingest_info_get_all
• jamwalla/ingest_info_prune
• jamwalla/ingest_info_prune_refactor
• jamwalla/ingest_info_repr
• jamwalla/ingest_info_type_hints
• jamwalla/json_data_extractor
• jamwalla/justice_solutions
• jamwalla/match_without_birthdate
• jamwalla/mt_gallatin
• jamwalla/nested_key
• jamwalla/new_world_get_initial_params
• jamwalla/newworld_date_no_data
• jamwalla/normalize
• jamwalla/pa_dauphin_fixes
• jamwalla/person_global_map
• jamwalla/pipenv_reinstall_all
• jamwalla/regions_refactor
• jamwalla/remove_booking_hold
• jamwalla/request_url_params
• jamwalla/run_scraper
• jamwalla/run_scraper_fail_fast
• jamwalla/sanitize_charge
• jamwalla/scrape_refactor
• jamwalla/scraper_template_code_indent
• jamwalla/scraper_template_readme
• jamwalla/scraper_tests
• jamwalla/scraper_utils.one
• jamwalla/travis_before_deploy
• jamwalla/update_pyyaml
• jamwalla/us_ny_full_name
• jamwalla/us_ny_sentence_lengths
• jamwalla/us_pa_greene_fixes
• jamwalla_tpw/entity_match_on_booking_id
• jbrambleDC/adds_offense_breakdowns
• jbrambleDC/pipeline-cleanup
• jbrambleDC/update_README.md
• jbrambleDC/version_bumps
• jre-instructions
• json-data
• jwl_tpw/hold_charge_bond_match
• jwl_tpw/match_on_subset_of_relationships
• kill-memcache
• ky_start_later
• last-updates
• last_seen_time
• local-app
• local-testing-readme
• log_all_the_things
• lumpkin-ga
• make_master_keys_foreign_keys
• make_minor_schema_changes
• make_shared_columns_mixins
• manage-queues
• manual_index
• mark_released
• master
• match-all-tzs
• max_concurrent_queues
• maybe-fix-travis
• mesa-enums
• mesa-fullname
• metadata-fields
• migration
• migration_update_aggregate_tables
• missed-post-data
• mixed_requests_pr
• monitoring
• more_more_schema_changes
• more_schema_cleanup
• more_site_aggr
• move-to-3
• move_base_scraper_to_persistance
• move_engine_setup
• move_in_pa_greene_vars
• multiple_people_extractor
• new-white-enum
• new_texas
• no-generate-id
• no-log-format
• no-memcache
• no_cache
• no_ssl_verify
• ny-ingest-coverage
• ny-ingest-coverage-final
• ny-models
• ny-retry
• ny-scraper-fix
• oh/export
• one-instance-min
• onlyinferreleaseonjails
• opencensus
• optional-names
• pa-greene-return-surname
• pa_greene_fixes
• pa_greene_surname
• package_refactor
• package_refactor_pr
• parse-dollar-NONE-strings
• persist-explicit
• persistence-flask
• persistence_test_use_ii
• pipenv
• pipenv-travis
• plurals
• post-migration
• post-migration-final
• post_data
• pretty_print
• prevent_orphan_entities
• production
• proto_lib
• proto_pip
• py3-upgrade
• pylint-test
• pylint_import_order
• pytest_docs
• python3
• python3queues
• quick
• readme
• readme-confirmation
• readme-deploy-projects
• readme-dev-update
• readme-update
• readme-update-oct-2018
• real-env-vars
• recidivism-bug-fix
• redo-docket
• refactor
• refactor-scraper-name
• regions-packaging
• release_schema
• remove-age-limit
• remove-appengine-ext
• remove-bad-docstrings
• remove-deferred
• remove-entity-kinds
• remove-init-stuff
• remove-libs
• remove-queue
• remove-region-entities
• remove-set-initial
• remove-wild-test
• remove_branch
• remove_date_imposed_from_fl_nassau
• remove_dateparser
• remove_generic_scraper
• remove_inferred_release
• remove_region_name
• remove_unneeded_readme
• rename-envvar
• rename-scraper-ingest
• requests-security-upgrade
• restore_unique_constraint
• retry_limit
• rm-export
• run-scraper-queue
• run_scraper_arg_types
• schem_entity_fixes
• schema_data_converter
• schema_script
• schema_updates
• scrape_all
• scrape_date
• scraper-fixes
• scraper_class_pr
• scraper_doc_run_to_test
• scrapy_example
• secrets-datastore
• secrets-typo
• serialize-ii
• service_unavailable
• session-extraction
• sessions-datastore
• set_up_snapshots_for_multi_person
• setup_for_offline_migration
• shared-jailtracker-queue
• single-queue-scraping
• skip-files-local-yaml
• small_changes_for_prod
• snapshot-scraper-cleanup
• speedup-pytest
• sql-interface
• staging
• start-all-cron
• stone_county_enums
• strict_entity_layer
• strict_entity_layer_pr
• superion
• superion_override_safety
• superion_test_refactor
• support-json-fetch
• supress_no_ssl_cert_warning
• tease-calculate
• test
• test-only
• test_for_master_and_historical_tables
• test_lib
• tests-manifest
• teststyle
• texas_aggr_fixes
• texas_parse_date
• toombs-ga
• tpw-dontjoinifyoudonthaveto
• tpw-jamwalla/true_equality
• tpw-jw/finish_entity_matching
• tpw-jw/finish_entity_matching_charges_and_bonds
• tpw-jwl/entity_matching_tests
• tpw-zl-converter_direction
• tpw-zl-enum-refactor
• tpw/all_children_updae
• tpw/bond_agent
• tpw/bond_pending
• tpw/brooks_ar
• tpw/brooks_in_ks_ms
• tpw/brooks_jeffrey_create_scraper
• tpw/brooks_jeffrey_divs
• tpw/brooks_jeffrey_preprocessing
• tpw/brooks_jeffrey_scraper_updates
• tpw/brooks_jeffrey_tests
• tpw/brooks_jeffrey_updates
• tpw/brooks_mo_ok_tx
• tpw/brooks_updates
• tpw/charge_inferred_drop
• tpw/data_extractor_ahref
• tpw/dedup_holds
• tpw/dont_prune_bookings
• tpw/drop_everything
• tpw/drop_statuses
• tpw/entity_match_id_and_nab
• tpw/extractor_changes_for_net_vendor
• tpw/fixny
• tpw/hold_external_id_and_statuses
• tpw/html_extractor_updates
• tpw/infer_statuses
• tpw/jailtracker_cases
• tpw/joined_load
• tpw/net_data_counties
• tpw/net_data_vendor_updates
• tpw/net_vendor
• tpw/pop_vs_popleft
• tpw/public_ingest_info
• tpw/queue_jailtracker
• tpw/raw_content_type
• tpw/region_specific
• tpw/remove_parents
• tpw/run_scraper_multiple_regions
• tpw/sets
• tpw/unknown_vs_inferred
• tpw_dont_lock_everything
• trace
• transient-test-failure
• travis-cache-pip
• travis-pc-slack-integration
• travis_fix
• two_way
• unify-flask
• update-region-manifest
• update_bond_enums
• update_create
• update_cron
• update_database_readme
• update_doc
• update_documentation
• update_historical_master_keys
• update_infer
• update_init
• update_migration_env_variables
• update_monitoring
• update_postgres_schema
• update_sa_schema
• update_yaml
• updated-region-manifest
• updateinfer
• updatemap
• upgrade-opencensus
• url-params
• urlfetch-hax
• urllibupdate
• us-wa-king
• us_co_mesa
• us_in_vigo
• us_in_washington
• us_mo_stone
• us_nc_alamance
• us_nc_buncombe
• us_nc_burke
• us_nc_cabarrus
• us_ny_2_base_scraper
• us_ny_charge_encoding
• us_ny_custody_status
• us_ny_life
• us_ny_list_index
• us_ny_no_tasks
• us_ny_sentence
• us_ny_to_base
• us_pa
• us_vt
• us_vt_pr
• use-enums
• utils-test-coverage
• v0.0.1
• v0.0.2
• v0.0.3
• v0.0.4
• v0.1.0
• v0.1.1
• v0.1.2
• v0.1.3
• v0.1.31
• v0.1.32
• v0.1.33
• various_schema_changes
• various_schema_migration
• vermont-region-fix
• vermont-snapshot-fix
• vscode-gitignore
• vt-snapshot-fixup
• vuln
• write_historical_snapshot
• zacharylawrence-create_scraper-readme
• zacharylawrence-initial_vars
• zacharylawrence-remove-person-status
• zacharylawrence_base_return
• zacharylawrence_ingest_info_name
• zdg2102-patch-1
• zl-add-copyright
• zl-add-test
• zl-aggregate-write
• zl-api-verify
• zl-arrest-builder
• zl-attr-rename
• zl-bond-builder
• zl-booking-builder-some-fields
• zl-buildable-hold
• zl-ca-aggregate
• zl-charge-builder
• zl-charge-count
• zl-convert-cleanup
• zl-converter-bond
• zl-converter-mock
• zl-converter-none
• zl-entities-optional-type
• zl-entities-type
• zl-entity-no-default
• zl-enum-map
• zl-error-rename
• zl-every-43
• zl-fips
• zl-fix-build
• zl-fix-pytest-warning
• zl-fl-aggregated
• zl-fl-date
• zl-fl-table2
• zl-full-name
• zl-ga-aggregate
• zl-ga-county-float-fix
• zl-ga-fix
• zl-hi-aggregate
• zl-hi-substring
• zl-ingest-info-stub
• zl-ky-aggregate
• zl-ky-aggregate-fix
• zl-last-day-of-month
• zl-metadata
• zl-more-builder
• zl-more-fips
• zl-mypy
• zl-mypy-readme
• zl-no-bond
• zl-no-none
• zl-ny-aggregate
• zl-ny-fix
• zl-pa-aggregate
• zl-persistence-entities
• zl-persistence-flask
• zl-persistence-proto-2
• zl-pylint-tree
• zl-remove-held-for-other
• zl-rename-and-select
• zl-rename-substring
• zl-sa-warnings
• zl-schema-pydoc
• zl-schema-rename
• zl-sentence-builder
• zl-tabula-warnings
• zl-test-lint-fix
• zl-tpw-court-type-enum
• zl-tpw_region_enum
• zl-tx-aggregate
• zl_lint
• zuercher

Build # 4134

Build Type

Pull #852

travis-ci

Committed by web-flow

Commit Message

Merge branch 'master' into jamwalla/scraper_refactor

Pull Request Pull Request #852: move base scraper files into recidiviz/ingest/scrape

Run Details

76 of 76 new or added lines in 31 files covered. (100.0%)

6075 of 7020 relevant lines covered (86.54%)

0.87 hits per line