OpenDataServices / cove

92%

master: 85%

LAST BUILD ON BRANCH tmp-etl-train

branch: tmp-etl-train

CHANGE BRANCH

x

Reset

• tmp-etl-train
• 1008-gdpr
• 101-humanize-datetimes
• 1012-records
• 1019-no-blank-sheets
• 1020-addfields-guidance
• 1022-human-dates
• 1024-release-schema
• 1027-closed_codelist
• 1029-codelists
• 1030-xml-comma-array
• 1050-ocds-show
• 1051-embedded-code-lists
• 1056-sum-decimals
• 1085-ocds
• 1092-fixtypo
• 11-tracking
• 1106-xml-line-number
• 1114-empty-fields-call-to-action
• 1117-group-validation
• 1118-one-from-many-msg-extra-checks
• 1120-multiple-additional-checks-sections
• 1122-additional-checks-flex-words
• 1124-tweak-copy
• 1131-spreadsheet-style-validation
• 1149-table-date-format
• 1158-non-emedded-codelists
• 1159-ocds-group-validation
• 118-fields-in-file-vs-schema
• 119-fixtures-360-data
• 1194-publication-policy
• 1201-ocds-inputs
• 1201-rearrange-inputs
• 1206-prolog
• 1208-iati-cove-orgxml-spreadsheet
• 121-pasteJSON
• 122-dataload-delete-status
• 1220-improve-non-unique-elements-error
• 1255-ods-link
• 127-check-files-actually-exist
• 1290-cove360-metatab-issued-on-display
• 1290-issued-on-bug
• 1293-360cove-additional-checks-page-formatting
• 1298_quality_check_email
• 130-etl-delete
• 1301-caching
• 1301-caching-2
• 1301-update-codelist
• 132-original-file-link-missing
• 1323-cove-iati-fix-google-doc-link
• 1366-update-flattentool
• 141-windows-1252-support
• 142-update-requirements-master
• 145-friendly-500
• 147-ocds-allow-records
• 149-contributing-doc
• 151-django-security
• 155-validation-speedups
• 157-spinner-time
• 158-unique-ids
• 162-http-errors
• 168-which-schema
• 169-offlinecdn
• 174-key-metrics
• 176-unreadable-xslx
• 17705_iati_fix_links
• 179-records-and-releases
• 182-change-bootstrap-theme
• 189-data-tables
• 192-speed-explore-reload
• 193-unordable-types
• 194-validate-dates
• 195-large-urls
• 198-fix-spinner-back
• 200-ocds-front-page
• 201-additional-check-title-length
• 2022-02-10-security
• 2022-03-01
• 2022-03-01-remove-old-dep
• 2022-03-01-tidyup
• 2023-03-08
• 2023-03-17
• 2024-10-03
• 2025-01-22
• 203-fix-dodgy-tags
• 203-fix-w3c-validation
• 207-refactor-to-library
• 211-update-requirements
• 213-ocds-explore
• 215-validation-visual-cleanup
• 218-recursion-error
• 220-allow-latin
• 221-show-all
• 226-change-prefix
• 228-test-alt-endpoints
• 229-warning-message-on-badly-encoded-file
• 230-button-to-flatten
• 234-validate-uri
• 235-add-links-to-ocds-page
• 236-add-transifex
• 238-spanish-machine-translate
• 25-footer
• 25-footer-copy
• 250-missing-revision
• 252-add-beta
• 252-new-ocds-theme
• 255-fix-package-metadata
• 260-underscore-allowed-extra-field
• 262-minor-ocds-theme-fixes
• 265-fix-tests
• 272-translation
• 277-roadmap
• 279-odd-link
• 285-requirements
• 285-update-requirements
• 288-deployment
• 292-dev-schema
• 292-use-dev-schema
• 293-dev-schema
• 294-readme
• 296-ignore-errors
• 298-ocds-analytics
• 3-text-for-readme
• 30-more-aggregates
• 305-sentry-release-tracking
• 307-sphinx-docs
• 310-automate-large-files
• 315-invalid-html
• 318-title-truncated
• 33-secret-key-public
• 330-content-disposition
• 330-use-content-type
• 339-small-fixes
• 341-rename-tool
• 344-encodings
• 346-user-agent
• 347-add-xdist
• 347-parallel-testing
• 349-recipient-org-id-feedback
• 353-trans-fixes
• 356-fix-old-file
• 356-improved-validation-messages
• 360-1200-additional-checks-count
• 360-additional-check-dates
• 360-copy-updates
• 360-cove-fixes
• 360-fix-format-error
• 360-link-test
• 360-links
• 360-new-design-logo
• 360-validation
• 360Giving
• 360giving-schema-change
• 363-button-padding
• 367-common-error-page
• 367-common-errors-page
• 367-common-errors-update
• 368-schema-link
• 380-additional-fields
• 380-look-betterer
• 380-make-betterer-logo
• 382-spinner
• 39-share-these-results
• 393-see-all-issues
• 4-translate
• 401-apply-layout-to-ocds
• 402-exclude-lang-field
• 403-failing-test
• 404-funding-orgs
• 408-disable-debug-toolbar
• 413-fts-fastererer
• 42-display-filesize
• 422-flattentool
• 424-key-info-360
• 424-selenium
• 426-files-directories
• 43-catch-conversion-errors
• 437-separate-translations
• 44-warnings
• 440-remove-360-language-switcher
• 443-roadmap
• 45-pass-url-as-get-param
• 46-validation-errors
• 469-error-required-field
• 47-remove-favicon
• 471-supplieddata-repr
• 472-conversion-warnings
• 476-bad-duplicate-ids
• 479-spelling-errors
• 482-uppercase-extensions
• 487-record-aggregates-ignore-errors
• 490-update-requirements
• 504-additional-checks
• 504-rearange-boxes
• 508-update-360-brand
• 509-fix-functional-tests
• 511-conversion-errors
• 523-date-warning
• 526-support-multiple-schema-versions
• 527-support-deprecation-warnings
• 528-codelist-529-extension-merge
• 528-codelist-varification
• 528-extension-codelist
• 529-add-OCDS-extension-support
• 537-fix-fts-tests
• 538-wrong-cursor-on-360-explore
• 544-ppp
• 545-https-js
• 548-add-header-link-main-360giving
• 551-titles-in-spreadsheet-converted-from-JSON
• 552-piwik-optout
• 553-more-information-box-is-blank
• 558-Data-Supplied-360-dateTime-feedback
• 559-data-supplied-to-is-bold
• 56-input
• 566-replace-here-link-text
• 567-zero-grant
• 568-recommended-fields-360-additional-checks
• 572-number-array
• 575-additional-checks-headline-highlight
• 583-grammar-fix
• 588-check-deprecation-key-on-JSON-reference-object
• 589-invalid-JSON-causes-deprecation-function-error
• 59-improve-360-explore-page
• 593-review-section-360-language
• 596-cove-360-UI-tweaks
• 597-scottish-check
• 601-app-separation-with-IATI
• 602-convert-upload
• 603-update-requirements
• 608-fixtures-moved
• 613-flatten_dict-bug
• 617-651-validation
• 619-initial-style-iati
• 621-no-print
• 622-extended-schema-remake-on-change
• 626-add-dot-to-email-check
• 630-parallelise-tests
• 635-sort-xml
• 638-360-templates
• 639-rename-data-supplied-section
• 641-use-flatten-tool-metatab
• 643-describe-additional-checks
• 646-extensions-headlines
• 655-additional-checks-360-clickable-counts
• 664-show-locations-for-360-multiple-funder-IDS
• 668-deploy-prep
• 672-additional-checks-360-blurb
• 674-more-info-always-open
• 675-check-ocds-prefix-format
• 676-768-bad-data
• 676-list-error
• 678-warning-extensions-version-1.0
• 679-additional-fields
• 68-360-validation
• 680-ocds-fix-typo
• 688--missing-ocds-deprecated-fields
• 690-records-template
• 694-version1.1-OCDS-selector-text
• 697-1-1-typo
• 704-link-extension-docs-url
• 705-708-errors
• 707-records-bug
• 707-test-record-version-switch
• 708-errors
• 709-bad-releases
• 71-update-SOURCES
• 715-bad-version-cove-ocds-data
• 722-upload-management
• 725-test-namespaces
• 733-update-lxml
• 734-wrong-validataion-field-name
• 736-new-extension-layout
• 737-broken-links
• 749
• 749-command-line
• 755-202-schema
• 762-expire_files-output
• 771-utf8-bom
• 774-iati-ruleset
• 775-spreadsheet-template
• 779-cove360-header-for-navigation
• 780-843-850-bug-fixes
• 784-sample-data-link
• 788-codelists-live
• 790-710-453-bug-fixes
• 790-bug-fixes
• 791-ocds-cli-use-schema-version-option-xlsx
• 796-use-orgids-for-360
• 800-802-fix-additional-checks
• 809-validating-extension-500-error
• 81-banner-message
• 81-functional-tests-dev
• 819-no-ruleset-directory
• 822-explicit-release-or-record
• 827-iati-cli
• 834-additional-iati-checks
• 835-validate-tag
• 836-improve-iati-inline-docs
• 838-float-int
• 84-add-link-360-sample-data
• 84-ocds-sample-data
• 84-small-tweaks
• 843-850-bug-fixes
• 846-panel-primary
• 850-bug-fixes
• 851-grammar
• 851-hack-scss-theme
• 855-xml-to-spreadsheet
• 857-doc-spreadsheet-data
• 859-update-requirements
• 862-bug-360-metatab
• 868-rulesets
• 87-outdated-comment
• 873-switchoff-two-additional-tests
• 882-record-extensions
• 887-utf8-encoding-error
• 89-refactor-explore
• 895-oneOf-messages
• 899-iati-ruleset-errors-table-structure
• 9-validate
• 90-error-contact-details
• 901-add-additional-checks
• 905-post-json
• 911-cache-schema
• 915-defusedexpat-github
• 915-mercurial-dep
• 92-load-new-data
• 921-tooltip-version-switch
• 925-ruleset-errors-list-no-values
• 929-keyerror-items
• 938-fixtures-360-personal-data
• 940-update-flattentool
• 940-update-flattentool-correctly
• 947-missing-ids-in-objects
• 948-cd
• 948-cd-docs
• 951-improve-messages
• 952-favicon-font
• 954-update-openpyxl
• 959-org-file
• 968-panel-warning
• 969-ocds-faling-test
• 977-openpyxl-trunk
• 979-203
• 991-fix-tests
• 998-iati-schema-link
• Bjwebb-patch-1
• add-description-testing-deprecation-local-schemas
• add-release-table-to-results
• add_how_to_iati_cove
• add_iati_xml_comment
• add_more_info_iati_cove
• additional-checks-url-add-class-name
• additional-checks-url-display-msg
• additional-test-classes-divide
• all-additional-fields
• appveyor
• better-ocds-buttons
• bjwebb-tmp
• branch-360-0.9-release-bugfix
• branch-360-additional-text-update
• branch-360-autumn-2017-bugfix
• cache-explore
• cd-360
• change-company-name
• cli-folder-validation
• collapse-icons
• cove-common-error-text-changes
• credit-external-contributors
• csv-export-additional-checks
• datagetter-9
• date-validation
• dave-selenium
• daves-working-branch
• decimals-bug
• dependabot/pip/bleach-3.1.1
• dependabot/pip/bleach-3.1.4
• dependabot/pip/django-2.2.24
• dependabot/pip/django-3.2.19
• dependabot/pip/django-3.2.20
• dependabot/pip/django-debug-toolbar-3.2.1
• dependabot/pip/pygments-2.15.0
• dependabot/pip/requests-2.31.0
• dependabot/pip/sqlparse-0.4.2
• dependabot/pip/sqlparse-0.4.4
• dependabot/pip/urllib3-1.26.5
• deploy-docs
• deployment-docs
• deployment-md
• deployment-md-1
• dev
• dev-tmp
• divide-sections
• django-2.2
• django-security
• django-security-updates
• django-update
• docs
• downgrade-openpyxl
• downgrade-openpyxl-2-6
• drop-blank-sheets-columns
• es-translations-reviewed
• fireproofboxstandard
• fix-360-template-indentation
• fix-additional-checks-url
• fix-copy
• fix-cove-template-indentation
• fix-cron-tests
• fix-docs
• fix-docs-2
• fix-iati-translations
• fix-oc-link-test
• fix-ocp-website
• fix-permissions
• fix-terms-url
• fix-tests
• fix-tests-for-cove-branch-name
• fix_how_to_text
• flatten-tool-ods-support
• flatten-tool-updates
• flattentool-0-4-0
• flattentool-146-sort-xml
• flattentool-177-xml-path-consistency
• flattentool-342-last-modified-does-not-convert
• flattentool-342-last-modified-does-not-convert-360
• flattentool-v0.8.0
• flow-charts-to-README
• fts-remote-server
• gdpr-privacy-notice
• get_iati_non_embedded_codelists
• github-actions
• hide-empty-sections
• hotjar
• html-in-error-messages
• iati-data-summary
• iati-dev
• iati-dportal-link
• iati-fix-links
• iati_cove_header_words
• iati_fix_links
• integration
• item-schemes-rename
• lib-cove-ocds-1
• lib-cove-ocds-2
• lib-cove-schema-rename
• libcove-decorator
• live
• live-ocds
• live-with-deprecated
• lxml-upgrade
• mark_safe_heading
• marning
• master
• master-360-data-quality-tool
• merge-indicator
• merge-live
• meta
• minor-live-improvements
• minor-readme
• more-coves
• mw/360_company_number_check
• mw/_update_req_ods_text
• mw/remove_360_cove
• mw/update_req
• mw/upgrade-lib-cove-web-0.12.0
• newer-openpyxl
• ocds-1.1.2
• ocds-1.1.3
• ocds-behind-proxy
• ocds-cove-both-urls
• ocds-data-review-tool
• ocds-dev-schema
• ocds-i18n-next-page
• ocds-python-lib
• ocds-sample-data-link
• ocds-schema-1-0-2
• ocds-v-1-1-4
• odscjames-patch-1
• openpyxl-commit
• org-ref-section
• org-type-schema-test
• padding-around-errors
• phantomjstest
• pin-openpyxl
• re-add-sentry-dealer
• readme
• readme-chromedriver
• readme-updates
• reapply-old-changes-from-360
• refactor-code
• refactor-templates
• release-201601-360
• release-201602
• release-201603
• release-201604
• release-201606
• release-201607
• release-201611
• release-201701-360
• release-201703
• release-201705
• release-201707
• release-201709
• release-201709-testdev
• release-201710
• release-201711-ocds
• release-201712
• release-201712-merge-master
• release-201801
• release-20705
• release-libcove-0-12-1
• release-libcove-0-12-2
• remove-360-secret-tests
• remove-google-doc
• remove-libcoveocds
• remove-ocds
• remove-ocds-2
• remove-ocds-secret-tests
• remove-roadmap
• replace-tests-names-URL-uppercase
• requires-io-master
• resourceprojects-etl
• resourceprojects-etl-merged
• resourceprojects-wireframe
• revert-273-flatten-tool-updates
• revert-281-revert-273-flatten-tool-updates
• rm-alpha
• rm-cove-packages
• rm-defusedexpat
• rm-requires.io-badge
• rm-requires.io-badge-1
• ruleset-no-500
• safe-html
• secret-data-test
• set_hash_comments_default_iati
• set_hashcomments_default_360
• specialist-language
• stats
• stevens-fixes
• tablesorter
• terms
• test-360-package-metadata
• test-branch-jb
• test-jb
• test-ocds-1-0-2
• text-fix-255
• tidy-up-ocds
• titles
• tmp
• tmp-bjwebb
• tmp-bjwebb-train
• tmp-conversion-warnings
• tmp-dev
• tmp-googledocs
• tmp-integration
• tmp-pr-shouldfail
• to-libs
• translations
• travis
• travis-pip-use-mirrors
• typo-fix
• typo-spelling-warning
• update-360giving-address
• update-bleach
• update-chromedriver
• update-dependencies
• update-django
• update-docs
• update-flattentool
• update-flattentool-openpyxl
• update-hypothesis
• update-insecure-requirements
• update-lib-cove
• update-lib-cove-ocds
• update-lib-cove-ocds-version-0.5.0
• update-lib-cove-web-v0.8.0
• update-libcove-libcoveweb
• update-libcove-versions
• update-license-info
• update-messages
• update-ocds-ts-cs-date
• update-odsc-reqs
• update-openpyxl
• update-readme-virtualenv
• update-req
• update-reqs
• update-requirements
• update-roadmap
• update-terms
• update-version-lib-cove-0.16.1
• update_flattentool_again
• update_flattentool_version_0.7.0
• update_readme_translations
• update_requirements_bleach
• upgrade-libs
• xml-detect-encoding

Build # 244

Build Type

push

travis-ci

Committed by Bjwebb

Commit Message

Refactor how dataload processes are defined

This also renames Process to ProcessRun, and uses process to refer to
e.g. the general process of converting a file, rather than a particular
conversion (which is now a run).

Processes for resourceprojects are now defined in a single OrderedDict
(PROCESSES) at the top of views.py.
The plan is to move this resourceprojects specific code out to a
separate repository, see
https://github.com/NRGI/resource-projects-etl/issues/40

Some computation is moved from the templates to the view, now that we
can iterate over PROCESSES. The hope is that this will make implementing
correct status information for deletion easier
(https://github.com/OpenDataServices/cove/issues/112 ).

Run Details

424 of 459 relevant lines covered (92.37%)

0.92 hits per line