FlexMeasures / flexmeasures / 23790681427

78%

push

github

Feat: `save_to_db` no longer attempts to save duplicate beliefs (#2059)

* fix: prevent duplicate belief constraint violations with automatic deduplication
Add a new remove_existing_beliefs() utility function to the time_series module
that proactively removes beliefs already in the database before saving.
This function compares (event_start, belief_horizon, event_value) tuples to
identify duplicates and filters them out, preventing unique constraint violations
when forecasters are run multiple times with identical data.
The deduplication is now transparently integrated into save_to_db(), meaning all
forecasters automatically benefit from this protection without needing to call it
explicitly. Forecasters can simply call save_to_db(bdf) and know that duplicate
beliefs will be handled.
Key improvements:
- Prevents IntegrityError when re-running forecasts with unchanged data
- Works with most_recent_beliefs_only=False for processing all historical beliefs
- Handles edge cases (empty DataFrames, multiple belief times)
- Centralized deduplication logic in FlexMeasures core
Signed-off-by: F.N. Claessen <claessen@seita.nl>

* revert: redundant part of the fix

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* refactor: reuse bdf_db (no need to query again)

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* refactor: prevent call to remove_existing_beliefs in lower level groupby statement

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* chore: minimize diff

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* style: RST docstrings

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* feat: add tests for saving both deterministic and probabilistic beliefs

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* docs: changelog entry

Signed-off-by: F.N. Claessen <claessen@seita.nl>

* fix: use separate test fixture for new tests

Signed-off-by: F.N. Claessen <claessen@seita.nl>

---------

Signed-off-by: F.N. Claessen <claessen@seita.nl>
Signed-off-by: Nicol... (continued)

33 of 33 new or added lines in 2 files covered. (100.0%)

11496 of 14795 relevant lines covered (77.7%)

0.78 hits per line