openego / eDisGo / 41996504366

87%

Build Type

push

github

Committed by web-flow

Commit Message

Limit the threads of the OPF and the spatial reduction (#834)

* Limit Gurobi to 8 threads in the OPF by default

Without a Threads setting, Gurobi used all cores of the machine, which
slowed down other processes and parallel runs of several grids. The new
parameter threads (default 8, 0 for all cores) of EDisGo.pm_optimize and
of the runner step optimize is handed to Main.jl as sixth argument and
set as Gurobi's Threads parameter.

* Limit the thread pools of the spatial reduction to 1 thread by default

The k-means clustering of scikit-learn (OpenMP) and numpy/scipy (BLAS)
used all cores of the machine during the spatial complexity reduction,
which slowed down parallel runs on shared servers. More threads hardly
speed up the reduction, as most of its run time is spent in
single-threaded code (networkx, pandas). The new parameter threads
(default 1, 0 for all cores) of EDisGo.spatial_complexity_reduction and
of the runner block spatial_reduction limits the pools with
threadpoolctl. threadpoolctl is already a dependency of scikit-learn; it
is now listed explicitly because eDisGo imports it directly.

Closes #844

Coverage Stats

6 of 6 new or added lines in 1 file covered. (100.0%)

2 existing lines in 1 file now uncovered.

10256 of 11819 relevant lines covered (86.78%)

0.87 hits per line