sylvan-energy / gridpath / 29040303337

89%

push

github

Memory efficiency enhancements (#1390)

* Refactor updating results dataframes to use a new update_results_df() helper function

This adds a results dataframe's columns to the consolidated frame at the source column's dtype and fills values in instead of first assigning None. This replaces the df[c] = None + .update() that created object-dtype columns with much higher memory requirements.

* Use params for availability when possible

If every required availability type produces a constant derate (no variables in the derate, e.g. all-exogenous), make the derate components Params rather than Expressions, as params are much more memory-efficient and we usually don't have endogenous availability.

* Don't store zero-valued reserve to energy adjustments

* Add option to skip duals

* Use SetOf for AVL_EXOG_OPR_TMPS and VAR_OM_COST_ALL_PRJS_OPR_TMPS

When we only have exogenous availability time, don't create AVL_EXOG_OPR_TMPS set, but use SetOf(m.PRJ_OPRT_TMPS).

When we only have simple var costs, don't create VAR_OM_COST_ALL_PRJS_OPR_TMPS, but use SetOf(m.VAR_OM_COST_SIMPLE_PRJ_OPR_TMPS)

* Remove defaults for var cost params

* Tx_Availability_Derate as param, not expression, unless endogenous

* Test common functions

213 of 232 new or added lines in 72 files covered. (91.81%)

28630 of 32371 relevant lines covered (88.44%)

0.88 hits per line