FlexMeasures / flexmeasures / 41819946573

88%

Build Type

push

github

Committed by web-flow

Commit Message

Keep appsi solver output off the app's log handlers, which could deadlock a worker (#2807)

* Keep appsi solver output off the app's log handlers

Pyomo's appsi solvers (e.g. appsi_highs) solve inside capture_output(capture_fd=True),
which points file descriptors 1 and 2 at a pipe that a reader thread drains into config.solver_output_logger.
That logger propagates to the root by default,
so a console handler writes each line back into the pipe it came from:
lines pile up with one more prefix per pass,
and once a line no longer fits in the pipe, the reader thread blocks on its own write while holding the handler's lock,
which hangs the worker.

device_scheduler now hands appsi solvers a logger of its own, registered nowhere,
whose one handler writes to a copy of stdout taken before the capture starts.
Solver output and any warnings logged during the solve then reach the console exactly once.

The new subprocess test reproduces the loop through device_scheduler at DEBUG,
and fails without the fix (lines carry up to five prefixes) as well as with a fix that merely drops the output.

Closes #2742

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* Changelog entry and docs for keeping appsi solver output off the log handlers

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* Link the changelog entry to PR #2807

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* Say in the docs that appsi solver output no longer reaches the log file, and how to keep a HiGHS log

Signed-off-by: Ahmad-Wahid <ahmedwahid16101@gmail.com>

* Move the appsi solver-output helper to the planning utils

It is a utility around the solve, not part of the scheduling model, so it distracts from what linear_optimization.py is about.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UBBvKZkzbfEPKctwTN8SDf
Signed-off-by: F.N. Claessen <claessen@seita.nl>

* Merge the changelog entries of the two HiGHS output fixe... (continued)

Coverage Stats

23 of 26 new or added lines in 2 files covered. (88.46%)

96 existing lines in 12 files now uncovered.

23102 of 26344 relevant lines covered (87.69%)

0.88 hits per line