FlexMeasures / flexmeasures / 36438469167

85%

push

github

Fix: concurrent identical scheduling requests no longer create duplicate data sources (#2612)

* Reuse the oldest data source when duplicates exist

Concurrent calls to get_or_create_source can each insert the same source,
because the unique constraint on data_source treats NULL user and account IDs as distinct.
Every later lookup then raised MultipleResultsFound, failing all schedules under that configuration (#2611).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: F.N. Claessen <claessen@seita.nl>

* Share one job between concurrent identical calls to a job-cached function

The job cache checked for a cached job, and only stored a new one after creating it,
so two identical calls arriving together both missed the cache and each created their own job.
For scheduling, the two jobs then each created their own data source for the same configuration (#2611).
Hold a Redis lock on the hash of the call's arguments while checking the cache and creating the job.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: F.N. Claessen <claessen@seita.nl>

* Add changelog entry for PR #2612

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: F.N. Claessen <claessen@seita.nl>

* Read the cached job ID once, as the key can expire between two reads

Checking that the job cache key exists and then getting it could return None if the key expired in between,
failing on .decode().

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: F.N. Claessen <claessen@seita.nl>

* Name the duplicate source IDs in the warning, and empty the queue after the concurrency test

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: F.N. Claessen <claessen@seita.nl>

* Join the concurrency test's threads with a timeout, and stop shadowing the source argument

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: F.N. Claessen <claessen@seita.nl>

---------

Signed-off-by: F.N. C... (continued)

25 of 28 new or added lines in 2 files covered. (89.29%)

3984 of 4694 relevant lines covered (84.87%)

4.39 hits per line