6062556821
78%
main: 78%

Ran 03 Sep 2023 05:20AM UTC

Jobs 1

Files 407

Run time 11s

Badge

Embed ▾

Committed 01 Sep 2023 04:12PM UTC coverage: 77.717%. First build

Build # 6062556821

Build Type

push

github

Committed by

web-flow

Commit Message

fix: resume connectors on network errors (#1956)

* fix: retry on network errors in MySQL connector

Resume database queries on network errors.
Select queries with multiple rows resume from the last row received.
CDC continues from its last position.

* fix: retry on network errors in Postgres connector

Similarly to the MySQL connector, select queries resume from the last row received.
The CDC resumes from the position where it was stopped.

* fix: retry on network errors in Kafka connector

Detect network failures, reconnect, and resume.

* fix: retry on network errors in Object Store connector

This is not a complete solution but we should use retry infrastructure provided by the object_store crate.

* chore: add sleep between retries

---------

Co-authored-by: chubei <914745487@qq.com>

Run Details

638 of 638 new or added lines in 26 files covered. (100.0%)

49643 of 63877 relevant lines covered (77.72%)

51433.87 hits per line

New Missed Lines in Diff

Lines	Coverage	File
1	85.37	dozer-ingestion/src/connectors/postgres/iterator.rs
1	94.29	dozer-ingestion/src/connectors/postgres/tests/client.rs
2	95.74	dozer-ingestion/src/connectors/postgres/tests/continue_replication_tests.rs
3	54.55	dozer-ingestion/src/connectors/postgres/test_utils.rs
4	0.0	dozer-ingestion/src/connectors/kafka/connector.rs
5	89.78	dozer-ingestion/src/connectors/mysql/binlog.rs
6	50.51	dozer-ingestion/src/connectors/object_store/adapters.rs
6	69.07	dozer-ingestion/src/connectors/postgres/connection/helper.rs
8	0.0	dozer-ingestion/src/connectors/kafka/debezium/schema_registry.rs
11	0.0	dozer-ingestion/src/connectors/kafka/stream_consumer_basic.rs
14	0.94	dozer-ingestion/src/connectors/kafka/debezium/stream_consumer.rs
22	83.08	dozer-ingestion/src/connectors/mysql/connection.rs
39	39.05	dozer-ingestion/src/connectors/postgres/replicator.rs
66	59.51	dozer-ingestion/src/connectors/postgres/connection/client.rs
67	0.0	dozer-ingestion/src/connectors/kafka/stream_consumer_helper.rs

Jobs

ID	Job ID	Ran	Files	Coverage
1	6062556821.1	03 Sep 2023 05:20AM UTC	407	77.72	GitHub Action Run

getdozer / dozer / 6062556821
78%
main: 78%

README BADGES
x

Markdown

Textile

RDoc

HTML

Rst

New Missed Lines in Diff

Jobs

Source Files on build 6062556821

getdozer / dozer / 6062556821 78% main: 78%

README BADGES x

Markdown

Textile

RDoc

HTML

Rst

New Missed Lines in Diff

Jobs

Source Files on build 6062556821

getdozer / dozer / 6062556821
78%
main: 78%

README BADGES
x