supabase / etl / 15886387446
82%

DEFAULT BRANCH: main
Ran
Jobs 1
Files 84
Run time 1min
Badge
Embed ▾
README BADGES
x

If you need to use a raster PNG badge, change the '.svg' to '.png' in the link

Markdown

Textile

RDoc

HTML

Rst
coverage: 60.116% (-1.3%) from 61.443%
15886387446

push

github

web-flow 
feat: add postgres state store to allow state to be persisted to the source db (#152)

* add ability to migrate the source database for storing state

* bump sqlx to 0.8.6 to remove unnecessary warnings in logs

* remove replication origin state

* remove replication table state

* remove dependency on pipeline identity

* remove unknown phase and add comments

* fix: a bug in which table schema was not added to cache

* add a log line

* make tests run faster

* add postgres state store

* make StateStore trait dyn compatible

* move BoxedStateStore to a new file

* make code compile

* use const for hardcoded values

* fix another compilre error

* use const for hardcoded values

* simplify config

* revert schema cache fix

* rename a variant

* update a log line

* rename a variable

* use variants explicitly instead of _

* use static dispatch in state store and always use PostgresStateStore in replicator

* remove Send + Sync from StateStoe

* save sync done lsn in state store as well

* use text type to save sync_done_lsn

* improve the state store trait

* do not load state store in apply worker as it it already done in pipeline

* add some comments

* add more comments

* rename an error

Co-authored-by: Riccardo Busetti <riccardob36@gmail.com>

* update a comment

Co-authored-by: Riccardo Busetti <riccardob36@gmail.com>

* fix a typo

Co-authored-by: Riccardo Busetti <riccardob36@gmail.com>

* fix cargo fmt

---------

Co-authored-by: Riccardo Busetti <riccardob36@gmail.com>

81 of 313 new or added lines in 16 files covered. (25.88%)

25 existing lines in 8 files now uncovered.

5580 of 9282 relevant lines covered (60.12%)

27.42 hits per line

New Missed Lines in Diff

Lines Coverage ∆ File
1
0.0
 0.0% api/src/main.rs
2
92.37
 2.66% etl/src/v2/workers/apply.rs
2
88.84
 -1.73% etl/src/v2/workers/table_sync.rs
3
85.71
 -3.06% etl/src/v2/replication/table_sync.rs
5
93.68
 -1.12% api/src/db/sources.rs
7
0.0
 0.0% replicator/src/core.rs
15
0.0
 0.0% etl/src/v2/state/store/memory.rs
18
0.0
 0.0% config/src/shared/source.rs
19
0.0
 replicator/src/migrations.rs
26
38.64
 -48.86% postgres/src/sqlx/config.rs
134
0.0
 etl/src/v2/state/store/postgres.rs

Uncovered Existing Lines

Lines Coverage ∆ File
1
0.0
 0.0% etl/src/v2/state/store/memory.rs
1
63.83
 1.14% etl/src/v2/state/table.rs
1
92.37
 2.66% etl/src/v2/workers/apply.rs
1
0.0
 0.0% replicator/src/core.rs
3
85.71
 -3.06% etl/src/v2/replication/table_sync.rs
4
88.84
 -1.73% etl/src/v2/workers/table_sync.rs
7
80.28
 -5.63% etl/src/pipeline/batching/stream.rs
7
79.59
 -4.55% etl/src/v2/pipeline.rs
Jobs
ID Job ID Ran Files Coverage
1 15886387446.1 84
60.12
 GitHub Action Run
Source Files on build 15886387446