IQSS / dataverse

24%

develop: 24%

LAST BUILD ON BRANCH 11405-refactor-exports

branch: 11405-refactor-exports

CHANGE BRANCH

x

Reset

• 11405-refactor-exports
• 10028-rewrite-querydsl-queries
• 10048-public-settings-static-lookup
• 10048-settings-api-redesign
• 10052-use-try-with-resources-to-close-fileinputstream-gzippedin
• 10052-use-try-with-resources-to-close-fileinputstream-gzippedin-=-new-fileinputstreamf-in-fileutiljava-in-method-determinefiletype
• 10099-reload
• 100_bump_nimbus_oauth2
• 10101-qa-guide
• 10104-get-dataset-citation-deaccessioned
• 10149-bump-release-version-6-1
• 10151-61-create-release-notes
• 10155-can-download-at-least-one-file-ds-user-perm
• 10161-Http2SolrClient
• 10164-bug-api-datasets-versions-fix
• 10168-fix-api-response-getArchivalStatus
• 10169-JSON-schema-validation
• 10179_PDF_thumbnail_NPE
• 10193_Shibboleth_SELinux
• 10202-extend-getVersionFiles-api-to-include-total-file-count
• 10207-bug-api-datasets-versions-latest-fix
• 10212-refactor-get-latest-published-dsv-cmd
• 10216-metadatablocks-payload-extension
• 10217-source-name-harvesting-client
• 10220-thumbnail-size-bug
• 10229-fix-featured-dv-selection
• 10232-guestbook
• 10240-file-citation
• 10242-set-featured-dv-via-api
• 10243-guestbook-sql-fix
• 10254-fix-harvested-redirects
• 10257-get-citation
• 10271-fix-external-tool-test
• 10280-get-file-api-extension
• 10286-add-owner-info
• 10297-dataset-metadata-endpoint-fix
• 10298-update-harvesting-source-facet
• 10304-add-move-dataverse-collection
• 10316-docs
• 10337-k8s-operator
• 10340-forbidden
• 10341-croissant
• 10365-jsonprinter-nullsafejsonbuilder
• 10381-timers
• 10389-metadata-field-display-on-create-api-ext
• 10398-geospatial-block
• 10422-62-release-notes
• 10423-6.2bump
• 10438-fix-intermittent-harvesting-tests
• 10442-index-draft-from-template
• 10472-review-modify-jsonprinter-for-datasetfieldtype
• 10476-display-on-create-field-option
• 10476-display-on-create-field-option-new
• 10477-dataset-fields-on-create-extension
• 10494-payara-6.2024.4
• 10506-previewers
• 10508-base-image-fixes
• 10517-dataset-types
• 10517-dataset-types-no-ff
• 10517-datasetTypes
• 10519-dataset-types
• 10519-scratch1
• 10520-datasetTypeLicenses
• 10542-signposting
• 10554-avoid-solr-join-guest
• 10570-search-improvements
• 10581-request-access-email-fix
• 10583-dataset-unlink-functionality-same-permission-as-link
• 10599-findDeep
• 10606-windows-dev
• 10611-harvested-datasets-display
• 10618-app-image-tags
• 10623-globus-improvements
• 10633-create-collection-api-ext
• 10637-metadatablocks-input-levels-extension
• 10646-63-release-notes
• 10647-63-bump-release-version-in-code
• 10661-guestbook-email-field-not-validated
• 10667-primefaces-upload-temp-files
• 10714-access-requests-missing-since-upgrade-v6-0
• 10718-no-op
• 10718-pf-themes
• 10726-facets-api-extension
• 10733-add-publication-status-to-search-api-result
• 10734-harvest-reindex-refactoring
• 10735-search-dataset-name-and-dataset-citation-different
• 10739-exportFormats
• 10741-list-metadatablocks-display-on-create-fix
• 10742-newest-oldest-sort-order-backwards
• 10749-dataverse-user-permissions-api
• 10754-edit-dataset-permission-description
• 10761-extend-dataset-version-paylod-with-latest-published
• 10764-updates
• 10800-add-dataverse-request-json-fix
• 10810-search-api-payload-extensions
• 10815-make-autoselect-thumbnail-optional
• 10819-publish-thumbnail-bug
• 10821-fix-for-upload-dataset-logo
• 10826-fix-controlledvocab-after-merging-10762
• 10831-standardize-image-url-of-search-api
• 10849-add-isreleased-field-to-ispartof
• 10850-citation-tsv-displayoncreate-depositor-dateofdeposit
• 10852-bump-to-6.4
• 10853-6.4-release-notes
• 10857-add-expiration-date-to-recreate-token-api
• 10875-update-conditions-to-display-image_url
• 10887-solr-types
• 10888-add-api-for-comparing-dataset-versions
• 10904-edit-dataverse-collection
• 10909-datacite-oai-harvesting
• 10909-oai-identifiers-as-pids
• 10910-users-token-api-credentials
• 10930-marketplace-external-tools-apis
• 10941-remove-custom-guides-navbar
• 10943-featured-items
• 10947-unpublished-files-appearing-in-search-results-for-anon-user
• 10952-release-notes
• 10954-bump-to-6.5
• 10959-bearer-token-auth-ext
• 10977-globus-filesize-lookup
• 10981-published-datasets-should-contain-files
• 10984-returnDatasetFieldTypes
• 10989-harvesting-regression
• 10993-stop-using-action-v3
• 11004-docker-block-solr
• 11012-get-dataverse-api-ext
• 11018-update-dataverse-fix
• 11020-unconsidered-harvesting-granularity
• 11027-extend-datasets-files-from-search-api
• 11032-async-datasetit-test-failure
• 11044-edit-facet-bug
• 11057-globus-downloads
• 11065-extend-search-api-to-include-type-counts
• 11075-ror
• 11076-access-previewUrl-with-perms
• 11083-mydata-npe-with-harvested-dataverses
• 11087-tweaks
• 111-address-issues-with-file-search
• 11120-add-3d-object-metadata-block
• 11127-search-api-counts
• 11129-send-feedback-to-contacts
• 11130-update-dataverse-api-remove-metadatablock
• 11157-builtin-users-oidc-auth
• 11159-anon-access-doc
• 11169-uppercase-shape
• 11182-index-after-ingest
• 11183-6.6-release-notes
• 11187-bump-to-6.6
• 11188-version-difference-summary
• 11191-assign-role-api-error-message
• 11198-list-file-versions-api
• 11208-fix-update-filemetadata
• 11226-file-perms
• 11243-editmetadata-api-empty-values
• 11244-get-dataset-download-count-api
• 11248-json-smart
• 11252-ncml
• 11262-copy-labels
• 11275-add-limit-to-number-of-dataset-files
• 11275-limit-number-of-dataset-files
• 11276-mdc-add-server-to-processing-state-db-table
• 11280-search-api-count-per-object
• 11281-search-services-docs
• 11287-show_my_data
• 11292-update-tabular-tags-with-replace
• 11299-file-restrict-api-extension
• 11303-flyway-script
• 11305-export-drafts
• 11305-export-drafts-scratch1
• 11305-export-non-latest-published
• 11312-display-on-create-followup
• 11323-includeDeaccesioned-compare-ds-versions
• 11324-flyway-script-concat-string-too-long
• 11337-shib-to-oidc
• 11364-rate-limit-file-page
• 11367-6.7-release-notes
• 11367-release-notes
• 11371-bump-version-to-6.7
• 11376-add-version-note-to-diff-summary
• 11377-empty-frequency
• 11382-featured-items-unicode-filename
• 11384-add-deleted-property-in-datafile-object
• 11387-fix-inputLevel-api
• 11391-displayOnCreate-with-template
• 11392-edit-file-metadata-empty-values
• 11401-update-categories-with-replace
• 11404-shib-login-mdq
• 11413-rate-limiting-statistics-api
• 11414-add-dvobject-type-to-featured-items
• 11415-app-tou
• 11434-roles-api
• 11438-fix-file-version-summary
• 11448-api-endpoint-for-analytics-html
• 11465-api-fetch-download-size-file-count
• 11465-api-fetch-download-size-file-count-2
• 11470-remove-cached-on-destroy
• 11472-fix-mailserviceit
• 11473-harvesting-client-ratelimit
• 11477-fix-pushing
• 11479-harvester-broken-generic-oai
• 11512-maven-central
• 11512-noop
• 11516-run-flyway
• 11519-my-data-api-bug-using-session-auth-for-spa
• 11525-retrieve-collections-a-user-can-create-datasets-in
• 11528-display-collection-size-on-add-files-page
• 11536-Inconsistency-between-subject-metrics-for-dataverses-and-datasets
• 11536-Inconsistency-between-subject-metrics-for-dataverses-and-datasets-bis
• 11540-ai-guide
• 11542-get-file-count-pg-2
• 11546-pid-separator
• 11550-update-dv-object-featured-items-response
• 11562-api-get-templates
• 11573-print-datasettype
• 11605-existing-external-users-api-auth
• 11606-hide-spa-oidc-providers-from-jsf-login-screen
• 11614-include-isAdvancedSearchField-property
• 11633-list-dataverse-links-api-change
• 11634-api-get-available-file-categories
• 11639-db-opts-idempotency
• 11645-old-oidc-users-api-auth
• 11648-jsf-vs-spa
• 11648-notification-patterns
• 11648-notifications-api-extension
• 11650-unread
• 11651-base.image.version
• 11652-quickstart-publish
• 11659-fix-ct-release
• 11670-notification-of-moved-datasets
• 11678-bump-to-6.8
• 11680-post6.8-base-image-version
• 11686-flaky-test-upload-file-with-limits
• 11689-builtin-users-api-auth-enhance
• 11692-search-api-bug
• 11695-change-api-get-storage-driver
• 11698-6.7.1-hotfix
• 11700-base-image-version
• 11703-return-isDefault-property-get-dataset-templates
• 11710-find-dataverses-for-linking
• 11714-prevent-exclude-email-from-export-for-permitted-user-get-dataset-version
• 11717-fix-api-errors
• 11724-extend-list-dataverse-collection-links
• 11726-remove-unused-code
• 11737-encoded-facet-and-title-values
• 11740-api-file-download-with-bearer-token
• 11744-cors-echo-origin-vary
• 11747-review-dataset-type
• 11747-review-extends-Dataset
• 11747-review-extends-DvObjectContainer
• 11752-croissant-restricted
• 11760-rm-api-admin-test
• 11766-new-io.gdcc.dataverse-spi
• 11771-fix-npe-in-update-licenses
• 11771-set-dataset-license-api
• 11772-api-terms-of-access
• 11773-incorrect-roles-listed-in-assignrole-notifications
• 11790-making-releases
• 11796-list-dataset-templates
• 117_bump_oauth2-oidc-sdk
• 11804-notifs-api-ext
• 11807-xoai-dependencies
• 11828-unacceptable-performance-deleting-datasets
• 11829-make-docs-html
• 11852-notifs-api-pagination-unread
• 11855-version-summaries-pagination
• 11879-6.9-release-notes
• 11885-bump-version-6-9
• 118_bump_apache_commons_text
• 11900-improved-cvoc-value-validation
• 11907-mdq-missing-attribute
• 11914-set-template-default
• 11918-template-apis
• 11921-dataset-version-summaries-changed-metadatacount-fix
• 11987-storage-quotas-on-datasets
• 11987-storage-quotas-on-datasets-experiment
• 11987-storage-quotas-on-datasets-extra
• 11996-fix-settings
• 11996-test-pure-develop
• 11998-features-in-guides
• 12030-unable-to-delete-dataverse-with-metrics
• 12033-add-template-command-fix
• 18_htmlunit
• 1f9e10cfc8acb85780db641d1a6644eb62545b05
• 220-audit-physical-files
• 255-operationalize-mdc-fix-log-formatting
• 2638-message-sent
• 315-fix-handle-to-citation-processing
• 3322-oai-list-sets
• 3437-oai-set-export-vs-reindex
• 359-enhance-publishing-message-acknowledgement
• 360-modify-notvalid-deaccession-reason
• 5051-ddi-tabular-file-description
• 5056-dataverse-deaccession-info
• 5645-geospatial-props-nslong-fix
• 5889-gen-schema-pojos
• 6.1-making-releases
• 6.7-updated-xoai-library
• 6.7.1-hotfix-to-develop
• 6.7.1-merge-to-develop
• 6467-optimize-permission-lookups-for-a-user
• 6575-move-dataset
• 6783-s3-tests
• 679-spi-multidatasource-setup
• 7031-phoenix-schemaspy
• 7239-mydata-results-by-username
• 75_google_cloud_protobuf
• 7618-file-level-permissions-restricted-draft
• 8013-history-of-access-request-available-via-api
• 8129-harvesting
• 8134-add-qpj-qmd-extensions
• 8136-annoying-draft-message-after-publishing
• 8184-preview-url-changes
• 8243-improve-language-controlled-vocab
• 82dad99d4eefbb194519fdadcfc630ff05b6e1b9
• 8328-versions-tab-experiment
• 8524-store-tabular-files-with-varheaders
• 8549-collection-quotas
• 8578-support-extended-iso-639-languages
• 8578-support-for-iso-639-3-languages
• 8581-add-crossref-pid-provider
• 8581-crossref-provider2
• 8739-publisher-during-harvesting
• 8760-bagit
• 8796-fix-no-license-display-index
• 8808-10575-update-global-role
• 8936-docs
• 8945-prevent-mislabelling-non-shapefiles-in-zip
• 8985-deprecate-rsync
• 89_bump_org_json
• 8c99a74f854a62050d850dccf30246c2fb06a5b9
• 906f874f9fd56241c41e6c2c2f6989c4406f5909
• 9077-fix-mime4j
• 9081-CC0-waiver-turned-into-custom-license
• 9275-harvest-invalid-query-params
• 9294-improvement-and-internationalization-of-harvest-status
• 92sec-upgrade-solr-9.4.1
• 9313-jsonld-try
• 9351-edit-optionds-deaccessioned-dataset-fix
• 9356-rate-limiting-command-engine
• 9361-followup-dvuploader-fix
• 9361-storage-quotas
• 93_duracloud_8_0_0
• 9401-fix-truncated-collection-name-guestbook-csv-file
• 9461-charset
• 9464-json-schema
• 9464-schema-creator-validator
• 9507-show-linked-collections
• 9547-validate-geospatial-metadata
• 9589-ds-configure-tool
• 9590-faster-redeploy
• 9590-xhtml-docker
• 95_bump_solrj
• 9620-better-file-bundle-name
• 9634-dev-guide-checklist-multiple-field
• 9635-solr-search-improvements
• 9686-move-harvesting-client-id
• 9692-files-api-extension-display-data
• 9748-fix-ExternalToolsIT
• 9763-lookup-optimizations
• 9771-driver
• 9771-fix-docker-build
• 9814-beta-commit
• 9846-sonar
• 9871-termsofuse_license
• 9880-info-zip-download-limit-embargo
• 9908_PG_ipv6_trust
• 9913-template-preview
• 9916-action-reports
• 9919-perf-doc
• 9920_postgres16
• 9992-harvest-metadata-missing-controlled-values
• AWSv2
• Anubis_fix
• ApacheHTTPUpdate
• CSLFix
• CVoc-improvement
• DANS-CSL
• DANS-bulk_file_delete
• DANS-multifile-edit
• DANS-remove_async_dates_from_main_tables
• DANS_1691
• DANS_Performance2
• DD-1988-500_error_validating_files
• DD-2075-IQSS-11992
• DD-2098
• DDIFix
• FTFix
• FeaturedItem_efficiency
• FilterUpdates
• Fix-dataset-storageDriver_api
• Fix_Flaky_DatasetsIT_test
• Fix_delete_file_test
• Flaky_XMLMetadataTemplateTest_fix
• GDCC/8914-COAR-compliant_messaging
• GDCC/8914-COAR-compliant_messaging2
• HDV/247-gbar_spacing
• IQSS-10975
• IQSS-11776-indexing_new_files
• IQSS/10108-StataMimeTypeRefinementForDIrectUpload
• IQSS/10516_fix_perma-legacy_support
• IQSS/10680-BagitFix
• IQSS/10883
• IQSS/11069-fix_index_status_query
• IQSS/11095-fix-extcvoc-indexing
• IQSS/11113-Fix_file_perm_doc_creation
• IQSS/11126-Payara6.2024.12
• IQSS/11133-search-fix0with-multiple-groups
• IQSS/11142-replace_in_file_differences
• IQSS/11174-Fix_authorAffiliation_DataCite_XML
• IQSS/11228_Summary_Fields_Fixes
• IQSS/11242-ExtVocabFix
• IQSS/11272-FixOrderOnAdvancedSearchPage
• IQSS/11394-versionNote
• IQSS/11418_poi_fix
• IQSS/11592-HandleFix
• IQSS/11777-improve_MDC_Citation_api_scaling
• IQSS/11778-speedup_custom_file_retrieval
• IQSS/11783-Curation_status_fixes
• IQSS/11800-GB-request_access_errors
• IQSS/11909-ucar_error
• IQSS/11990-controlledvocab_editing_error
• IQSS/12020-Payara7_Java21_update
• IQSS/7632_exclude_packages
• IQSS/8431_Version_Creation_Note
• IQSS/9247-CurationStatus_updates
• IQSS/DataCite_publishing_bug
• LRUStatementCache
• Metrics-fix_file/monthly
• MetricsFix
• ORCID_1
• OREBag1.0.2
• PIDFailTracking
• Payara-6.2025.9
• Payara-update
• Perf1
• QDR-Clearing_Exports
• QDR-DCiteScaling
• QDR-DataCiteRORErrorHandling
• QDR-improve-batch-permission-indexing
• QDR-perm-reindex-in-mult-transactions
• QDR-solr_and_libs_updates
• RemoteOverlayTestFix
• RoleAccessHistory
• SPA-_api_to_getToolLaunchUrl
• Search_Refactor
• SemAPITestFix
• TKLabels
• UTFTestFix
• a218417e875a41e03495581fb6f0e8207c07bd75
• add-isreleased-field-to-get-dataverse-api-response
• add-mit-license
• api-change-latest-version-status
• api_auth_required_openapi
• authn-arch-api-oidc
• awssdk_update
• bag-info-wrapping
• banner_null_json_message_fix
• blocked-deposit-page
• c6b6dac5c3dcc9b78bbc7765ba0eae0fc96a4b81
• cachelock
• cleanup_fix
• compound-field-ordering
• concurrent-role-assignment
• container-naming-fix
• d9cc5ebcaca1c4fdb651468d314ec7cb2dc1e067
• dd-2023-Missing-entity-with-workflow-POST-AuthorizedExternalStep
• deaccession_ds_success_message_doi_fix
• dependabot/github_actions/actions/cache-4
• dependabot/maven/conf/keycloak/builtin-users-spi/org.keycloak-keycloak-services-26.3.0
• dependabot/maven/modules/dataverse-parent/org.apache.commons-commons-lang3-3.18.0
• dependabot/maven/modules/dataverse-parent/org.postgresql-postgresql-42.7.7
• dependabot/maven/org.apache.james-apache-mime4j-core-0.8.10
• dependabot/maven/org.apache.poi-poi-ooxml-5.4.0
• dependabot/maven/org.apache.solr-solr-solrj-9.4.1
• dependabot/maven/org.xmlunit-xmlunit-core-2.10.0
• develop
• doc-contrib
• ds27-samesite-attr
• edit-metadata-api-500-error
• enforce-java-17
• expose-and-sort-publish-status-facet
• feat/10134
• feat/10171-versions-smaller-response
• feat/11147
• feat/dataset-count
• feat/is-in-review-state
• feat/link-permission
• feat/list-assignments-more-info
• feat/list-dataset-links
• feat/my-data-api-params
• feat/reconcilepid
• feat/search-api-collections
• feat/solr-field-types
• feat/user-assignable-roles
• feature_force_contentType
• fix-json-ld
• fix-search-api-nested-metadata-fields
• fix/10809-oai-ore-nested-compound
• fix/11568
• fix/9200-allowmultiples-for-deep-nesting
• fix/dataset-sort-order
• fix/files-sort-by-type-criteria-query
• fix/jsonParserTest
• fix/perma-exporters-and-citations
• fix/solr-multi-valued
• fix/updateAttributes
• fix_infinite_loop_mdc_api
• fix_infinite_loop_mdc_api_v2
• fix_ror-v2_api_cvoc
• graceful-failure-mode-for-TIFF-images-failure
• indexing_fixes
• intellij
• issue-10523
• jsonInfoInHomePage
• le-croissant-exportateur
• mailtest
• manage-guestbooks-page-performance
• master
• master_json_fix
• mdc-process-files-via-script
• metadataLanguage-API-call
• missing-term-fix
• mpconfig-personororg
• mpconfig-schema-dot-org-hide-download-urls
• multipid
• no-op-image-build
• npe-cvoc
• openaire_fix
• openapi-definition-endpoint
• openshift-poc
• operationalize-mdc-processing-state-api
• optimistic-lock
• optimize-role-assignment
• otherIdentifierBug
• poc/oidc_spa_integration
• post6.4-container-version
• pr-11825
• pr/9665
• prepare-next-iteration
• prod_docker
• prodpatch-6.8-globus-no-locking
• refactor-metadatablock
• restore-11301
• revert-10214-9686-move-harvesting-client-id
• revert-11224-10476-display-on-create-field-option
• revert-11301-blocked-deposit-page
• revert-11312-10476-display-on-create-field-option-new
• revert-11534-feat/link-permission
• revert-11551-11536-Inconsistency-between-subject-metrics-for-dataverses-and-datasets
• revert-11636-11465-api-fetch-download-size-file-count
• revert-9684-9683_get_dataset_api_in_single_query
• revert-9783-9714-files-api-extension-filters
• rocky_IX
• scolapasta-patch-1
• server-restart-problem-fix
• solr-date-sort-optimization
• solr-index-improvements
• sphinx-doc-update
• split-AnonymizedFieldTypeNames
• stable-unit-test
• toc-depth
• update-container-base-image
• xmlutil

Build # #27189

Build Type

Pull #11799

github

Committed by landreev

Commit Message

Made the batch-sizing math smarter (processing datavariable metadata in batches based on the number of variables, instead of the fixed number of datatables-worth at a time). #11405

Pull Request Pull Request #11799: Expands InternalExportDataProvider per dataverse-spi 2.1.0 and refactors the DDI exporter to utilize the new functionality

Run Details

42 of 260 new or added lines in 13 files covered. (16.15%)

21842 of 91903 relevant lines covered (23.77%)

0.24 hits per line