MerginMaps / server

93%

master: 93%

LAST BUILD ON BRANCH rework_concurrent_upload

branch: rework_concurrent_upload

CHANGE BRANCH

x

Reset

Sync Branches

• rework_concurrent_upload
• #2
• 202-wrong-prodenv-values-for-local_projects-and-temp_dir
• 20205.5.0
• 2023.6.1
• 2024.1.0
• 2024.2.0
• 2024.6.6
• 2024.6.7
• 2024.7.0
• 2024.7.1
• 2025.1.0
• 2025.1.1
• 2025.1.2
• 2025.2.0
• 2025.2.1
• 2025.2.2
• 2025.2.3
• 2025.3.0
• 2025.3.1
• 2025.3.2
• 2025.4.0
• 2025.4.1
• 2025.5.0
• 2025.5.0.beta.1
• 2025.6.0
• 2025.6.2
• 2025.7.0
• 2025.7.1
• 2025.7.2
• 2025.7.3
• 2025.7.3-ee
• 2025.8.0
• 2025.8.1
• 2025.8.2
• 2025.8.3
• 2025.8.4
• 2026.1.0
• 2026.1.1
• 2026.1.2
• 2026.1.3
• 2026.2.0
• 2026.2.1
• 2026.3.0
• 2026.3.1
• 408-extend-server-check-for-permissions
• 420-rework-deployment
• 457-sso-deployment-workflow-for-ee
• 522-webmaps-folder-permissions-test
• 540-docker-composeyml-point-to-wrong-image-tags-at-on-ce-server-releases
• Mergin_base_url_in_invitations
• PeterPetrik-patch-1
• account_expiration_rename_env
• add-app-page
• add-dark-green
• add-logo-customization
• add-object-deleted-error
• add-python-mimetype
• add-service-id-to-server-check
• add-severity-app-section-banner
• add_3rd_party_libs_list
• add_build_hash_ce
• add_hook_for_project_removal
• add_non_blocking_push
• api-update-batch-endpoint
• api_to_db_fields_map
• avatar_font_size
• backend_libs_update_v2
• basename-skipping-validation
• better_tracking
• black_commit_hook
• boxy_link_to_admin
• build-2024.1.0
• build-2024.7.0
• build-2025.1.0
• build-2025.3.0
• bumb-version
• bump-2024.2.0
• bump-2025.2.1
• bump-2025.4.0
• bump-2025.4.2
• bump-2025.5.0
• bump-2025.6.0
• bump-2025.7.3
• bump-2026.2.0
• bump-to-2024.2.1
• bump_2025.2.0
• bump_2025.3.0
• bump_2025.3.1
• bump_2025.4.1
• bump_2026.1.0
• bump_ce_version
• bump_public_images
• bump_python
• bump_server_version_2025.6.2
• cache-versions
• cache_last_login
• can_edit_profile
• celery-default-queue
• chunks_cleanup
• ci_build
• clone_button_enable
• clone_button_in_public_projects
• copilot-worktree-2026-01-20T12-29-12
• create_checkpoint_recursively
• create_project_signal
• currency-locale-accept
• cypress-tags
• cypress2
• cypress3
• debug_restore_404
• delta_add_v_prefix
• deploy-2026.1.1
• dev-2025.3.1
• dev-2025.6.2
• dev-backend-libs-upgrade
• dev-publish-maps
• dev-r19
• dev-r65-pull
• dev-r65-push
• dev-r84-concurrent-push
• dev-r85-v2-pull
• develop
• develop-redesign
• develop-redesign-admin
• develop-roles-refactor
• diagnostic-logs-endpoint
• do-not-check-diff-mimetype
• dont_add_creator_to_members
• dont_reveal_resource_to_anonymous
• drop_error_details_index_gh#2776
• editor_limit_blocking
• ee-ce-2025.7.2
• ee-overview-secondary-buttons
• ee_images_update
• enforce_valid_email_address_ce
• enh-accross-application
• enh-app-section-header
• enh-laptops-breakpoint
• enh-sharing-dialog-template
• enh-spacing-cleanup
• enh-vol1
• error_in_push_finish
• exlude_clone_filenames
• fe-deps-upgrades
• fe_get_project_collaborators
• fe_uses_v2_endpoints
• feature-allow-share
• feature-collaborators
• feature/podman-migration
• file_upload_restriction
• fix-admin-deactivate-account
• fix-admin-url-base
• fix-baseurl-admin
• fix-breadcrumps
• fix-download-version-rom-detail
• fix-duplicate-sql
• fix-error-integrity-handling
• fix-getting-large-files
• fix-gh211-access-requests-accept
• fix-global-admin-handling
• fix-install-link
• fix-message-server-check
• fix-missing-classes-purge
• fix-notifications-sharing
• fix-one-diff-concat
• fix-project-access-detail
• fix-projects-table-count
• fix-public-switch
• fix-rank-merges-on-delta-vibe
• fix-special-chars-downloaded-zip
• fix-tests-admin-is-now-ok-email
• fix_accel_uri
• fix_alembic_deleted_members
• fix_alembic_deleted_projects
• fix_alembic_error
• fix_auth_expiration_check
• fix_build
• fix_construct_checkpoint
• fix_crontab_editors
• fix_current_access_api
• fix_diff_checkpoints
• fix_download_cmd
• fix_download_project_archive
• fix_download_project_import
• fix_generate_username
• fix_gzip
• fix_inconsistent_permissions
• fix_issues_with_deletions
• fix_login_without_pwd
• fix_migration_revision_id_sequence
• fix_missing_latest_files
• fix_proxy_unpriviledged
• fix_push_only_delete
• fix_readme_links
• fix_remove_chunks
• fix_rollback_error
• fix_share_project_#2799
• fix_token_reuse
• fix_upload_version_collision
• fix_username_bigint_generation
• fix_username_generation
• fix_username_validation
• fix_v2_apis
• font-update-light-theme
• format_python_codestyle
• frozen_remove_after
• gevent_timeout
• gh#2229-last-seen-workspaces
• gh#2232-global-variables
• gh#2295-fix-coveralls
• gh#230-my-access-requests
• gh#230-subfolders
• gh#2536-flask-mail
• gh#2578-callhome-contributors
• gh#2597-fix-download-command-file
• gh#2613-add-push-finish-signal
• gh#2697-adjust-currency
• gh#2711-not-found-message-collaborators
• gh#2726-callhome-stats-export
• gh#2740-name-validations
• gh#2749-init-command
• gh#2751-enh-validation
• gh#2839-fix-access-request-emails
• gh#2907-breacrumps-fix
• gh#3042-chunks
• gh#67-migration-vue3
• gh#99-settings-access-requests-role
• gpgk_restore_from_checkpoints
• head-project-request
• hide_public_projects_sorting
• implement-#3233
• implement-#3263
• increase-max-logs-size
• increase_username_max_len
• init-command
• integrate-master
• integrate_last_sign_in
• introduce-configurable-session-lifetime
• large_project_download_error
• local_dev_compose
• log-diffs
• log_info_http_422
• make_tests_working
• master
• merge_user_profile
• merged-diffs
• new_diffs_table
• new_members_api
• nginx_logs
• page-titles
• page-titles-add-ws-name-projects-list
• path_validation_in_upload
• per_seat_pricing
• pipeline-ce-branch
• post_prepare_download
• prepare-2023-6-1-release
• prevent_version_duplicates_ce
• prod_setup_hints
• project-access
• project_admin_permission
• project_detail_v2
• provide-to-version
• publish_dialog
• push_to_dockerhub
• redesign-add-help
• redesign-admin-search-text
• redesign-auth
• redesign-comunity-banner
• redesign-dashboard-projects
• redesign-dialog-projects
• redesign-dialogs-vol2
• redesign-improvements-1
• redesign-notifications
• redesign-porting-ee-1
• redesign-profile
• redesign-project-detail-files
• redesign-projects
• redesign-sharing-dialog
• redesign-versions
• redesing-cleanup-1
• redesing-settings
• refactor-circle-component
• refactor-fonts-classes
• refactor-roles-values
• refactor_changes_schema
• refuse_trailing_whitespace_in_path
• registration_date_not_null
• release-2025.2.0
• release-tweaks
• remove-contributors-admin-overview
• remove-unhandled-expire-parameter
• remove-upload-id-from-logs
• remove_migration_branch
• responsive-dataview
• revert-320-create_project_signal
• revert-566-font-update-light-theme
• revert-572-font-update-light-theme
• revert-some-mimetypes
• rewrite_zip_download
• run-extractor-celery
• server_usage_in_admin
• show_inv_expiration_in_days_#2692
• show_maps_2789
• sso-ee-bump
• sso-ee-disable-magic-link
• sso_statistics
• support_for_otel
• switching-download-toast-properly
• sync-remove-chunks
• test
• test_cli_commands
• tip-message-severity
• trial_expired_message
• update-access
• update-limits-for-heade-and-cleanup
• update-projects-table-options
• update-statistics-counts
• update_migration
• upgrade-nodejs-22
• upgrade-padding-sidebar-footer
• upgrade_geodiff
• upload-allowed-files
• use-v2-update-project-access
• v2-project-push
• v2-pull-flag
• v2_get_ws_projects
• v2_project_info
• v2_project_project_version
• validate_email_domain
• wider_sidebar

Build # 24395881105

Build Type

Pull #610

github

Committed by varmar05

Commit Message

Rework concurrent upload using upsert strategy

Replace the old try/except IntegrityError + cleanup loop pattern with an atomic upsert in Upload.create_upload().
Decouple the upload directory name from the DB primary key via transaction_id. Upload directory is created at this stage, no need to take care for it later.

With the upsert logic, the ObjectDeletedError scenario which happened because a concurrent request could delete a stale upload row mid-operation is eliminated:
  - During push_finish, the heartbeat context manager continuously updates last_ping, keeping the upload fresh
  throughout the operation
  - A concurrent request can only take over an upload whose last_ping has expired
  - Since heartbeat prevents expiry, no other request can claim the row while push_finish is running
  - The upload object therefore stays valid for the full lifetime of the request — ObjectDeletedError becomes
  impossible

Pull Request Pull Request #610: rework concurrent upload

Coverage Stats

156 of 165 new or added lines in 8 files covered. (94.55%)

7 existing lines in 3 files now uncovered.

9114 of 9779 relevant lines covered (93.2%)

0.93 hits per line