openego / eDisGo / 42053336231

87%

push

github

Speed up the spatial complexity reduction and fix three bugs in it (#854)

* Speed up the spatial complexity reduction and fix three bugs in it

The reduction spent most of its run time in pandas row loops: per-bus
weights that searched the generator and load tables for every bus,
per-bus .loc writes of the busmap, row-wise apply calls to map buses and
transform coordinates, groupby.apply over all lines and per-node writes
of the pseudo coordinates. These are now done on whole columns, and the
shortest paths to the cluster medoids are searched within the feeder
and its transformer bus instead of the whole grid. Apart from the
susceptance fix below, busmap, linemap, topology and time series are
bit-identical to before on the test grids and on four real grids for all
cluster areas and modes; the reduction runs 2.5 to 9 times faster.

Bugs fixed:
- make_busmap transformed the coordinates of the EDisGo object to
  EPSG:3035 in place for the cluster areas 'feeder' and 'main_feeder'.
- apply_busmap set the susceptance b of every aggregated line to 0. It
  now keeps the susceptance per km of the aggregated lines.
- The prefix "virtual_" was removed with str.lstrip, which strips all
  leading characters contained in it.

* Address review: clear errors, reference busmaps, correct changelog

- make_busmap (kmeansdijkstra) raises a ValueError naming the bus if a
  bus cannot be reached from any cluster medoid, instead of assigning
  it to cluster 0.
- apply_busmap raises a ValueError naming the line if a line between two
  new buses has no type_info. Lines with other missing values are kept;
  the previous code dropped every line with a NaN in any column.
- New test comparing the busmap of test grid 1 with stored references
  for four cluster areas and modes, including the focus mode. The
  references were generated with the previous code and match it
  byte for byte.
- The susceptance test uses a parallel line with a different b per km,
  so that the sum over a... (continued)

212 of 220 new or added lines in 2 files covered. (96.36%)

10186 of 11737 relevant lines covered (86.79%)

0.87 hits per line