supabase / etl

67%

main: 83%

LAST BUILD ON BRANCH riccardobusetti/feat/improve-refresh

branch: riccardobusetti/feat/improve-refresh

CHANGE BRANCH

x

Reset

Sync Branches

• riccardobusetti/feat/improve-refresh
• abhi/ensure-pg-14-support
• add-metrics
• ci-fix
• ci-improve
• cleanup-releases
• codex/implement-fail-rs-for-fault-injection
• concurrency
• fix-ci
• fix-ci-docker
• fix-docker
• fix-null-parse
• fix-sentry
• fix/allow-empty-publication
• kafaichoi/ensure-unique-per-source-and-destination-pair
• main
• or/iceberg
• raminder/etl-148-process_syncing_tables-in-apply-loop-should-be-called-on
• raminder/etl-150-new-table-sync-workers-keep-launching
• release/v0.1.1
• riccardo/feat/add-replication-client
• riccardo/feat/implement-apply-loop
• riccardo/feat/workers-structure
• riccardo/fix/deadlock
• riccardobusetti/etl-100-allow-multiple-api-keys-from-use-by-different-clients
• riccardobusetti/etl-104-handle-values-outside-the-range-accepted-by-bigquery
• riccardobusetti/etl-129-set-sane-options-when-connecting-to-postgres
• riccardobusetti/etl-141-investigate-the-possibility-to-reduce-the-number-of-error
• riccardobusetti/etl-146-properly-handle-arraycell-option-values
• riccardobusetti/etl-15-rename-pgdatabaseoptions-to-pgdatabaseconfig-for-sqlx
• riccardobusetti/etl-156-find-a-solution-for-inability-to-truncate-bigquery-table
• riccardobusetti/etl-167-implement-cascade-delete-of-postgres-replication-state
• riccardobusetti/etl-17-create-new-types-for-oid
• riccardobusetti/etl-174-add-tests-for-conversions
• riccardobusetti/etl-19-implement-mechanism-to-cleanup-unused-slots-and-origins
• riccardobusetti/etl-207-the-pipelineshutdown_and_wait-function-sometimes-returns
• riccardobusetti/etl-210-extend-the-state-stored-in-postgres
• riccardobusetti/etl-212-handle-failed-table-sync-workers
• riccardobusetti/etl-214-simplify-how-a-failure-of-the-pipeline-is-exposed-to-the
• riccardobusetti/etl-215-its-neighborly-to-re-export-types-used-in-your-public-api
• riccardobusetti/etl-218-cleanup-unused-dependencies
• riccardobusetti/etl-225-move-table-schemas-into-the-state-store
• riccardobusetti/etl-232-store-table-creation-state-in-bigquery-destination
• riccardobusetti/etl-234-write-from-scratch-the-initial-migrations-before-the-alpha
• riccardobusetti/etl-236-add-support-for-more-types-in-the-table-schema-of-the
• riccardobusetti/etl-240-cleanup-replication-slots-on-pipeline-deletion
• riccardobusetti/etl-252-handle-row-errors-in-fallback-during-streaming-in-bigquery
• riccardobusetti/etl-254-implement-mechanism-for-defaults
• riccardobusetti/etl-257-use-gcp-bigquery-client-instead-of-our-fork
• riccardobusetti/etl-259-force-restart-pipeline-after-starting-it
• riccardobusetti/etl-262-handle-truncate-event-when-table-was-not-loaded
• riccardobusetti/etl-265-handle-state-reset-properly
• riccardobusetti/etl-266-handle-table-schema-and-table-mappings-override
• riccardobusetti/etl-272-detect-tables-being-removed
• riccardobusetti/etl-273-allow-filtering-logs-by-pipeline-id
• riccardobusetti/etl-274-avoid-buildup-of-the-replication-states
• riccardobusetti/etl-277-implement-the-backend-endpoints-for-returning-image
• riccardobusetti/etl-283-figure-out-test-failure
• riccardobusetti/etl-285-implement-mechanism-to-version-etl
• riccardobusetti/etl-287-implement-batch-timeout-with-timer
• riccardobusetti/etl-288-implement-mechanism-to-defer-shutdown-when-in-a-transaction
• riccardobusetti/etl-289-add-taints-to-etl-resources
• riccardobusetti/etl-290-implement-image-update-endpoint
• riccardobusetti/etl-309-add-support-for-renaming-publication
• riccardobusetti/etl-319-change-backoff-of-k8s
• riccardobusetti/etl-56-ensure-theres-always-exactly-one-default-image
• riccardobusetti/etl-66-ensure-unique-table-names-in-destinations
• riccardobusetti/feat/add-agents-file
• riccardobusetti/feat/add-function
• riccardobusetti/feat/add-labels
• riccardobusetti/feat/add-memory-profiling-script
• riccardobusetti/feat/add-more-data-to-logs
• riccardobusetti/feat/add-more-metrics
• riccardobusetti/feat/add-rust-docs
• riccardobusetti/feat/add-sentry-replicator
• riccardobusetti/feat/add-trusted
• riccardobusetti/feat/add-update-endpoint
• riccardobusetti/feat/better-resource-provisioning
• riccardobusetti/feat/default-to-manual
• riccardobusetti/feat/do-not-return-password
• riccardobusetti/feat/emit-project-logs
• riccardobusetti/feat/etl-docs
• riccardobusetti/feat/expose-errors
• riccardobusetti/feat/expose-lag
• riccardobusetti/feat/fail-rs-setup
• riccardobusetti/feat/implement-state-for-tables
• riccardobusetti/feat/improve-application-name
• riccardobusetti/feat/improve-big-query
• riccardobusetti/feat/improve-bigquery
• riccardobusetti/feat/improve-deletio
• riccardobusetti/feat/improve-endpoints
• riccardobusetti/feat/improve-errors
• riccardobusetti/feat/improve-exports
• riccardobusetti/feat/improve-github
• riccardobusetti/feat/improve-k8s
• riccardobusetti/feat/improve-logs
• riccardobusetti/feat/improve-pipeline
• riccardobusetti/feat/improve-readme
• riccardobusetti/feat/improve-rx
• riccardobusetti/feat/improve-tests
• riccardobusetti/feat/improve-vector-flushing
• riccardobusetti/feat/integrate-sentry
• riccardobusetti/feat/limit-pipelines
• riccardobusetti/feat/log-error
• riccardobusetti/feat/new-endpoint
• riccardobusetti/feat/new-endpooint
• riccardobusetti/feat/new-logo
• riccardobusetti/feat/pre-stop-hook
• riccardobusetti/feat/publish-rust-docs
• riccardobusetti/feat/remove-bg-emulator
• riccardobusetti/feat/remove-context-path
• riccardobusetti/feat/return-key
• riccardobusetti/feat/return-table-id
• riccardobusetti/feat/rework-logs
• riccardobusetti/feat/status-endpoint
• riccardobusetti/feat/swap-image
• riccardobusetti/feat/update-dependencies
• riccardobusetti/feat/update-docs
• riccardobusetti/feat/update-migration
• riccardobusetti/feat/use-mutex
• riccardobusetti/feat/use-secret
• riccardobusetti/feat/use-sentry-secret
• riccardobusetti/feat/write-docs-mkdocs
• riccardobusetti/fix/bigquery-conversion
• riccardobusetti/fix/ci
• riccardobusetti/fix/config
• riccardobusetti/fix/docker-build
• riccardobusetti/fix/formating
• riccardobusetti/fix/improve-ci
• riccardobusetti/fix/improve-output
• riccardobusetti/fix/optional-skip
• riccardobusetti/fix/panics
• riccardobusetti/fix/run-action
• riccardobusetti/fix/sentry-setup
• riccardobusetti/fix/shutdown-signal
• riccardobusetti/fix/tests-stuck
• riccardobusetti/fix/unwrap
• riccardobusetti/fix/workflows
• riccardobusetti/inject-version-env-variable
• riccardobusetti/psql-551-fill-schema-cache-from-destination
• riccardobusetti/psql-553-implement-bigquery-destination-v2
• riccardobusetti/psql-557-hook-the-v2-pipeline-in-the-replicator
• riccardobusetti/ref/change-application-name
• riccardobusetti/ref/change-defaults
• riccardobusetti/ref/change-endpoint
• riccardobusetti/ref/improve
• riccardobusetti/ref/improve-actions
• riccardobusetti/ref/improve-exports
• riccardobusetti/ref/increase-timeout
• riccardobusetti/ref/move-notify-state
• riccardobusetti/ref/remove-readme-section
• riccardobusetti/ref/rename-crates
• riccardobusetti/ref/simplify-schema
• rs/abort-on-panic
• rs/add-array-support
• rs/add-metrics-port
• rs/add-missing-project-ref-in-logs
• rs/add-pipeline-id-to-metrics
• rs/add-scalar-support
• rs/add-update-config-endpoint
• rs/api-metrics
• rs/benchmarking
• rs/bump-editions
• rs/bump-max-replication-slots
• rs/clippy-no-deps
• rs/db-sinks-to-destinations
• rs/debug-stuck-tests
• rs/docs
• rs/egress-metric
• rs/expect-clippy
• rs/feat/improve-logs
• rs/fix-abort-on-panic
• rs/fix-api-binary-name
• rs/fix-app-env
• rs/fix-assets
• rs/fix-batch-sent-metric
• rs/fix-clippy-errors
• rs/fix-docker-build
• rs/fix-failing-test
• rs/fix-links-in-readme
• rs/fix-panic
• rs/fix-test-tracing
• rs/fix-wrong-null-values-encoding
• rs/fix/sync-done-bug
• rs/fix/update-config
• rs/iceberg-client
• rs/iceberg-destination
• rs/iceberg_scripts
• rs/ignore-stuck-tests
• rs/improve-config
• rs/improve-connections
• rs/improve-etl-error-macro
• rs/improve-logging
• rs/improve-metrics
• rs/improve-replication-client
• rs/limit-max-connections
• rs/listen-on-all-interfaces
• rs/make-tests-run-in-parallel-with-clippy-and-fmt
• rs/mock-k8s-client
• rs/no-primary-key
• rs/remove-array-null
• rs/remove-duplicate-config
• rs/remove-expect
• rs/remove-old-code
• rs/rename-namespaces
• rs/rename-pg_replicate-to-supabase_etl
• rs/rename-supabase_etl-to-etl
• rs/renames-and-other-cleanup
• rs/replicator-metrics
• rs/sinks-to-destinations
• rs/state-store
• rs/state-store-new
• rs/tls-config
• rs/toast
• rs/tracing-in-tests
• rs/update-app-labels
• rs/update-benchmark-scripts
• rs/update-codeowners
• rs/use-consts

Build # 16367082343

Build Type

Pull #200

github

Committed by web-flow

Commit Message

Merge bac71ed0e into eb5d3c655

Pull Request Pull Request #200: feat(pipeline): Improve performance of the pipeline

Run Details

42 of 55 new or added lines in 7 files covered. (76.36%)

5478 of 8117 relevant lines covered (67.49%)

30.09 hits per line