FlexMeasures / flexmeasures
83%
main: 84%

LAST BUILD BRANCH: feat/chp
DEFAULT BRANCH: main
Repo Added
Files 175
Badge
Embed ▾
README BADGES
x

If you need to use a raster PNG badge, change the '.svg' to '.png' in the link

Markdown

Textile

RDoc

HTML

Rst

LAST BUILD ON BRANCH feat/sensor-data-source-association
branch: feat/sensor-data-source-association
CHANGE BRANCH
x

coverage: 83.464% (-0.002%) from 83.466%
30828312771

Pull #2382

github

Flix6x 
Install the trigger before backfilling, so concurrent inserts are not lost

The migration backfilled sensor_data_source and only then created the trigger,
both inside Alembic's transaction.
That loses data.
The backfill reads a snapshot taken when its statement begins,
so beliefs inserted while it runs are not in it,
and a trigger created afterwards in the same transaction is not visible to the
sessions doing that inserting either,
so nothing ever records those pairs.
On a large table the backfill takes minutes, so the window is not theoretical.

Create the trigger first and commit it, then backfill.
The two then overlap rather than leaving a gap between them,
and ON CONFLICT DO NOTHING absorbs the overlap.
Because the steps now commit as they go, a run that fails during the backfill leaves
the table and trigger behind, so creating the table is guarded and a retry gets past it.

Demonstrated both ways by stepping two connections explicitly,
with the second inserting a new pair between the backfill and the trigger creation.
The first attempt used a sleep and failed to reproduce the loss,
which would have made the problem look theoretical;
sequencing the connections instead shows the old order dropping the pair and the new
order keeping it.

Also merges the base branch, whose only conflict was the changelog:
this branch still carried the pre-correction "three indexes" wording alongside its own
entry, so the resolution keeps both entries with the base's wording.

flexmeasures/data: 256 passed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Signed-off-by: F.N. Claessen <claessen@seita.nl>
Pull Request #2382: Summarise which sources recorded for which sensors

19 of 22 new or added lines in 2 files covered. (86.36%)

16611 of 19902 relevant lines covered (83.46%)

0.83 hits per line

Relevant lines Covered
19902 RELEVANT LINES 16611 COVERED LINES
0.83 HITS PER LINE
Source Files on feat/sensor-data-source-association

Recent builds

Builds Branch Commit Type Ran Committer Via Coverage
30828312771 feat/sensor-data-source-association Install the trigger before backfilling, so concurrent inserts are not lost The migration backfilled sensor_data_source and only then created the trigger, both inside Alembic's transaction. That loses data. The backfill reads a snapshot taken when... Pull #2382 Flix6x github
83.46
30828319771 feat/sensor-data-source-association Merge b35e9e308 into 98052cd7c Pull #2382 web-flow github
83.46
30827085192 feat/sensor-data-source-association Address review: install the trigger from the metadata, and fix stale docstrings The trigger DDL was attached to timed_belief's own after_create, but the function it installs refers to sensor_data_source, and nothing orders that table's creation r... Pull #2382 Flix6x github
83.46
30826086545 feat/sensor-data-source-association Address review: break docstrings and comments only after punctuation Reflow the docstrings and inline comments added here so that every physical line ends at a comma, semicolon, colon or period, per .github/instructions/docstrings.instructions.md... Pull #2382 Flix6x github
83.47
30824436689 feat/sensor-data-source-association Point the changelog entry at the right PR number Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Signed-off-by: F.N. Claessen <felix@seita.nl> Pull #2382 Flix6x github
83.47
30824372247 feat/sensor-data-source-association Summarise which sources recorded for which sensors Two mirrored accessors ask which data sources have recorded for which sensors: Sensor.data_sources reads timed_belief in sensor_id order, DataSource.sensors reads it in source_id order. Both answ... push Flix6x github
83.47
See All Builds (7893)