15296154421

Committed 28 May 2025 09:07AM UTC coverage: 67.817% (+0.02%) from 67.793%

Build # 15296154421

Build Type

push

github

Committed by

web-flow

Commit Message

Merge pull request #334 from Zeitsperre/typo-adjustments

Typo adjustments, black formatting, drop Python 3.9, support Python 3.13

Run Details

24 of 99 new or added lines in 15 files covered. (24.24%)

13 existing lines in 7 files now uncovered.

3753 of 5534 relevant lines covered (67.82%)

2.03 hits per line

Source File
Press 'n' to go to next uncovered line, 'b' for previous

16.67

/src/spotpy/algorithms/efast.py

# -*- coding: utf-8 -*-
"""
EFAST algorithm transfered from CRAN R package 'fast' by Dominik Reusser (2015)
Copy of most functions and descriptions and implementation in SPOTPY framework by Anna Herzog (2024)

This Version of FAST differs from the previously impemented Version, as it does not require a precalculation of minimum model runs by the user. It rather uses a hard coded definition of minimum runs based on Cukier or McRae.
It therefore requieres considreably less model runs (for 5 Parameters 2245 (fast) vs. 71 runs (efast)).

For further information on the functions, the origninal R package and the fast algorithm please have a look at the R package description from CRAN, or see the following references:

Reusser, Dominik E., Wouter Buytaert, and Erwin Zehe.
"Temporal dynamics of model parameter sensitivity for computationally expensive models with FAST (Fourier Amplitude Sensitivity Test)." Water Resources Research 47 (2011): W07551.

Further References:
CUKIER, R. I.; LEVINE, H. B. & SHULER, K. E. Non-Linear
"Sensitivity Analysis Of Multi-Parameter Model Systems" Journal Of Computational Physics, 1978 , 26 , 1-42
CUKIER, R. I.; FORTUIN, C. M.; SHULER, K. E.; PETSCHEK, A. G. & SCHAIBLY, J. H.
"Study Of Sensitivity Of Coupled Reaction Systems To Uncertainties In Rate Coefficients .1. Theory" Journal Of Chemical Physics, 1973 , 59 , 3873-3878
SCHAIBLY, J. H. & SHULER, K. E.
"Study Of Sensitivity Of Coupled Reaction Systems To Uncertainties In Rate Coefficients .2. Applications" Journal Of Chemical Physics, 1973 , 59 , 3879-3888
CUKIER, R. I.; SCHAIBLY, J. H. & SHULER, K. E.
"Study Of Sensitivity Of Coupled Reaction Systems To Uncertainties In Rate Coefficients .3. Analysis Of Approximations" Journal Of Chemical Physics, 1975 , 63 , 1140-1149


Copyright (c) 2018 by Tobias Houska
This file is part of Statistical Parameter Optimization Tool for Python(SPOTPY).
:author: Anna Herzog, Tobias Houska, Dominik Reusser
"""

import warnings

import numpy as np

from ..analyser import efast_sensitivity, get_modelruns
from . import _algorithm


class efast(_algorithm):
    """
    EFAST Algorithm for (distributed) parameter Sensitivity after FAST algorithm according to Cukier 1975 or McRae 1982
    The extended Fourier Amplitude Sensitivity Test (eFAST) is a method to determine global sensitvities of a model on parameter changes
    within relatively few model runs over time.
    """

    _unaccepted_parameter_types = ()

    def __init__(self, *args, **kwargs):
        """
        Input
        ----------
        spot_setup: class
            model: function
                Should be callable with a parameter combination of the parameter-function
                and return an list of simulation results (as long as evaluation list)
            parameter: function
                When called, it should return a random parameter combination. Which can
                be e.g. uniform or Gaussian
            objectivefunction: function
                Should return the objectivefunction for a given list of a model simulation and
                observation.
            evaluation: function
                Should return the true values as return by the model.

        dbname: str
            * Name of the database where parameter, objectivefunction value and simulation results will be saved.

        dbformat: str
            * ram: fast suited for short sampling time. no file will be created and results are saved in an array.
            * csv: A csv file will be created, which you can import afterwards.

        parallel: str
            * seq: Sequentiel sampling (default): Normal iterations on one core of your cpu.
            * mpi: Message Passing Interface: Parallel computing on cluster pcs (recommended for unix os).

        save_sim: boolean
            * True:  Simulation results will be saved
            * False: Simulation results will not be saved
        """

        kwargs["algorithm_name"] = "EFast Sampler"
        super(efast, self).__init__(*args, **kwargs)

    # hard coded repetition and frequency values from R package Fast
    # fmt: off
    d_m_cukier75 = [4,8,6,10,20,22,32,40,38,26,56,62,46,76,96,60,86,126,134,112,92,128,154,196,34,416,106,208,328,198,382,88,348,186,140,170,284,568,302,438,410,248,448,388,596,217,100,488,166]
    min_runs_cukier75 = [np.nan, np.nan, 19, 39, 71, 91, 167, 243, 315, 403, 487, 579, 687, 907, 1019, 1223, 1367, 1655, 1919, 2087, 2351, 2771, 3087, 3427, 3555, 4091, 4467, 4795, 5679, 5763, 6507, 7103, 7523, 8351, 9187, 9667, 10211, 10775, 11339, 7467, 12891, 13739, 14743, 15743, 16975, 18275, 18927, 19907, 20759, 21803]
    omega_m_cukier75 = [np.nan, np.nan, 1, 5, 11, 1, 17, 23, 19, 25, 41, 31, 23, 87, 67, 73, 58, 143, 149, 99, 119, 237, 267, 283, 151, 385, 157, 215, 449, 163, 337, 253, 375, 441, 673, 773, 875, 873, 587, 849, 623, 637, 891, 943, 1171, 1225, 1335, 1725, 1663, 2019]
    d_m_mcrae82 = [4, 8, 6, 10, 20, 22, 32, 40, 38, 26, 56, 62, 46, 76, 96]
    min_runs_mcrae82 = [0, 15, 27, 47, 79, 99, 175, 251, 323, 411, 495, 587, 695, 915, 1027]
    omega_m_mcrae82 = [0, 3, 1, 5, 11, 1, 17, 23, 19, 25, 41, 31, 23, 87, 67]
    # fmt: on

    def freq_cukier(self, m, i=1, omega_before=-1):
        """
        This function creates an array of independent frequencies according to
        Cukier 1975 for usage in the eFAST method.

        Parameters
        ----------
        m: int
            number of parameters (frequencies) needed
        i: (intern) int
            internally used recursion counter
        omega_before: (intern) int
            internally used previous frequency

        Raises
        ----------
        Exception:
            "Parameter number m to high, not implemented"
            Execution is stopped, if the number of Parameters is too high (max. Number of Parameters for Cukier = 50)

        Returns
        ----------
        np.array of float:
            A 1d-Array of independent frequencies to the order of 4

        Author
        ----------
        Dominic Reusser
        spotpy translation: Anna Herzog

        """

        if i <= 1:
            if m >= len(self.omega_m_cukier75):
                raise Exception("Parameter number m is too high, not implemented")
            else:
                o = self.omega_m_cukier75[m - 1]
                return np.append(o, self.freq_cukier(m, i + 1, o))
        else:
            o = omega_before + self.d_m_cukier75[m - i]
            if i == m:
                return o
            else:
                return np.append(o, self.freq_cukier(m, i + 1, o))

    def freq_mcrae82(self, m, i=1, omega_before=-1):
        """
        This function creates an array of independent frequencies according to
        McRae 1982 for usage in the eFAST method.

        Parameters
        ----------
        m: int
            number of parameters (frequencies) needed
        i: (intern) int
            internally used recursion counter
        omega_before: (intern) int
            internally used previous frequency

        Raises
        ----------
        Exception:
            "Parameter number m to high, not implemented"
            Execution is stopped, if the number of Parameters is to high (max. Number of Parameters for McRae = 15)

        Returns
        ----------
        np.array of float:
            A 1d-Array of independent frequencies to the order of 4

        Author
        ----------
        Dominic Reusser
        spotpy translation: Anna Herzog

        """

        if i <= 1:
            if m >= len(self.omega_m_mcrae82):
                raise Exception("Parameter number m is too high, not implemented")
            else:
                o = self.omega_m_mcrae82[m - 1]
                return np.append(o, self.freq_mcrae82(m, i + 1, o))
        else:
            o = omega_before + self.d_m_mcrae82[m - i]
            if i == m:
                return o
            else:
                return np.append(o, self.freq_mcrae82(m, i + 1, o))

    def s(self, m, freq="cukier"):
        """
        Function that generates a number of equally spaced values between -p1/2
        and pi/2. The number is determined by the number of runs required of the
        eFAST method for a number of parameters (min_runs_cukier or min_runs_mcrae)

        Parameters
        ----------
        m: int
            number of parameters/ frequencies

        freq: (optional) str
            indicates weather to use the frequencies after 'cukier' or McRae 'mcrae'
            Default is Cukier

        Raises
        ----------
        Exeption:
            "Missing option for Frequency selection for Parameter definition, choose cukier or mcrae!"
            Execution is stopped if an invalid method for the frequency selection is provided

        Returns
        ----------
        2D array of float:
            an array of equally spaced values between -pi/2 and pi/2

        Author
        ----------
        Dominic Reusser
        spotpy translation: Anna Herzog

        """

        if freq == "cukier":
            min_runs = self.min_runs_cukier75
        elif freq == "mcrae":
            min_runs = self.min_runs_mcrae82
        else:
            raise Exception(
                "Missing option for Frequency selection for Parameter definition, choose cukier or mcrae!"
            )

        r = min_runs[m - 1]
        r_range = np.array(range(1, r + 1))
        s = np.pi / r * (2 * r_range - r - 1) / 2

        return s

    def S(self, m, freq="cukier"):
        """
        Function to generate an array of values, which provides the base for the parameterdistribution
        in the FAST method. It is usually not used directly but called from the
        fast_parameters function.

        Parameters
        ----------
        m: int
            number of parameters/frequencies

        freq: (optional) str
            indicates weather to use the frequencies after 'cukier' or McRae 'mcrae'
            Default is Cukier

        Raises
        ----------
        Exception:
            "Missing option for Frequency selection for Parameter definition, choose cukier or mcrae!"
            Execution is stopped if an invalid method for the frequency selection is provided

        Returns
        ----------
        2D array of float:
            an array with the shape (number of runs, parameters)

        Author
        ----------
        Dominic Reusser
        spotpy translation: Anna Herzog
        """

        if freq == "cukier":
            omega = self.freq_cukier(m)
        elif freq == "mcrae":
            omega = self.freq_mcrae82(m)
        else:
            raise Exception(
                "Missing option for Frequency selection for Parameter definition, choose cukier or mcrae!"
            )

        tab = np.outer(self.s(m, freq), omega)

        toreturn = np.arcsin(np.sin(tab)) / np.pi

        # naming array dimensions is not possible with numpy but convention would be toreturn.shape () = (runs, parameternames)

        return toreturn

    def rerange(self, data, min_goal=0, max_goal=1, center=np.nan):
        """
        This function performes a linear transformation of the data, such that
        afterwards range(data) = (theMin, theMax)

        Parameters
        ----------
        data: array of flaot
            an array with the data to transform
            in this case the parameter distribution generated by function S
        min_goal: float
            the new minimum value (lower parameter bound)
        max_goal: float
            the new maximum value (upper parameter bound)
        center: (optional) float
            indicates which old value should become the new center
            default: (max_goal+min_goal)/2

        Returns
        ----------
        2D array of float:
            array with the transformed data

        Author
        ----------
        Dominic Reusser
        spotpy translation: Anna Herzog

        """

        min_data = min(data)
        max_data = max(data)

        if np.isnan(center):
            max_data = max_data
            dat = data - min_data
            dat = dat / (max_data - min_data) * (max_goal - min_goal)
            dat = dat + min_goal
            return dat
        else:
            # split linear transformation, the data is split into two segments, one blow center, one above,
            # each segment undergoes its own linear transformation
            below = data <= center
            above = data > center
            new_data = np.copy(data)
            np.place(
                new_data,
                below,
                self.rerange(
                    np.insert(data[below], 0, center),
                    min_goal=min_goal,
                    max_goal=max_goal / 2,
                )[1:],
            )
            np.place(
                new_data,
                above,
                self.rerange(
                    np.insert(data[above], 0, center),
                    min_goal=max_goal / 2,
                    max_goal=max_goal,
                )[1:],
            )
            return new_data

    def fast_parameters(
        self, minimum, maximum, freq="cukier", logscale=np.nan, names=np.nan
    ):
        """
        Function for the creation of a FAST Parameter set based on their range

        Parameters
        ----------
        minimum: array of float
            array containing the lower parameter boundaries
        maximum: array of float
            array containing the upper parameter boundaries
        names: (optional) str
            array containing the parameter names
        freq: (optional) str
            indicates weather to use the frequencies after 'cukier' or McRae 'mcrae'
            Default is Cukier
        logscale: (optional) bool
            array containing bool values indicating weather a parameter is varied
            on a logarithmic scale. In that case minimum and maximum are exponents

        Raises
        ----------
        Exception:
            "Expecting minimum and maximum of same size"
            Execution is stopped if the number of minimum, maximum or logscale values differ

        Returns
        ----------
        value: array of float
            an array with the shape (number of runs, parameters) containing the
            required parameter sets

        Author
        ----------
        Dominic Reusser
        spotpy translation: Anna Herzog

        """

        if np.isnan(logscale):
            logscale = np.full(minimum.shape, False)
        if np.isnan(names):
            names = ["P" + str(i) for i in range(minimum.shape[0])]
            names = np.array(names)

        n = len(minimum)

        if n != len(maximum):
            raise Exception("Expecting minimum and maximum of same size")
        elif n != len(names):
            raise Exception("Expecting minimum and names of same size")
        elif n != len(logscale):
            raise Exception("Expecting minimum and logscale of same size")

        toreturn = self.S(m=n, freq=freq)  # par_names = names,

        for i in range(0, n):
            toreturn[:, i] = self.rerange(toreturn[:, i], minimum[i], maximum[i])
            if logscale[i]:
                toreturn[:, i] = 10 ** toreturn[:, i]

        return toreturn

    def sample(self, repetitions, freq="cukier", logscale=np.nan):
        """
        Samples from the eFAST algorithm.

        Input
        ----------
        repetitions: int
            Maximum number of runs.
        freq: (optional) str
            indicates weather to use the frequencies after 'cukier' or McRae 'mcrae'
            Default is Cukier
        logscale: (optional) bool
            array containing bool values indicating weather a parameter is varied
            on a logarithmic scale. In that case minimum and maximum are exponents

        Raises
        ----------
        Warning:
            "specified number of repetitions is smaller than minimum required for eFAST analysis!
            Simulations will be performed but eFAST analysis might not be possible"

            If the specified number of repetitions is smaller than required by the algorithm, the
            algorithm will still execute, but the usage of spotpy.analyser.efast.calc_sensitivity()
            might not be possible. Rather define a large number of reps as only required minimum runs
            will be performed anyway.

        Warning:
            "Number of specified repetitions equals or exceeds number of minimum required repetitions for eFAST analysis
             program will stop after required runs."

        Returns
        ----------
        database:
            spotpy database in chosen format with objective function values, parameter values and simulation results

        Author
        ----------
        Tobias Houska, Anna Herzog
        """

        print("generating eFAST Parameters")
        # Get the names of the parameters to analyse
        names = self.parameter()["name"]
        # Get the minimum and maximum value for each parameter from the
        # distribution
        parmin, parmax = self.parameter()["minbound"], self.parameter()["maxbound"]
        self.numberf = len(parmin)
        min_runs = self.min_runs_cukier75[self.numberf - 1]

        if min_runs > repetitions:
            warnings.warn(
                "specified number of repetitions is smaller than minimum required for eFAST analysis!\n"
                "Simulations will be performed but eFAST analysis might not be possible"
            )
        else:
            repetitions = min_runs
            print(
                "Number of specified repetitions equals or exceeds number of minimum required repetitions for eFAST analysis\n"
                "program will stop after required runs."
            )

        self.set_repetiton(repetitions)

        # Generate Matrix with eFAST parameter sets
        N = self.fast_parameters(parmin, parmax, freq, logscale)

        print("Starting the eFast algorithm with {} repetitions...".format(repetitions))

        # A generator that produces parametersets if called
        param_generator = ((rep, N[rep, :]) for rep in range(int(repetitions)))
        for rep, randompar, simulations in self.repeat(param_generator):
            # A function that calculates the fitness of the run and the manages the database
            self.postprocessing(rep, randompar, simulations)

            if self.breakpoint == "write" or self.breakpoint == "readandwrite":
                if rep >= lastbackup + self.backup_every_rep:
                    work = (rep, N[rep, :])
                    self.write_breakdata(self.dbname, work)
                    lastbackup = rep
        self.final_call()

    def calc_sensitivity(self, results, dbname, freq="cukier"):
        """
        Function to calculate the sensitivity for a data series (e.g. a time series)
        based on the eFAST algorithm.

        Parameters
        ----------
        results: np.array of void
            spopty restults array as loaded with spotpy.analyser.load_csv_results()
        dbname: (optional) str
            name of file to store sensitivity values
        freq: (optional) str
            indicates weather to use the frequencies after Cukier 'cukier' or McRae 'mcrae'
            Default is Cukier

        Raises
        ----------
        Exeption:
            "Missing option for Frequency selection for Parameter definition, choose cukier or mcrae!"
            Execution is stopped if an invalid method for the frequency selection is provided

        Returns
        ----------
        database:
            database containing the temporal parameter sensitivities with seperate column
            for each parameter and row for e.g. time step

        Author
        ----------
        Dominic Reusser
        spotpy translation: Anna Herzog

        """
        numberf = len(self.parameter()["minbound"])

        # sens_data = np.full((len(results[0]), numberf), np.nan)

        # idendtify frequency
        if freq == "cukier":
            t_runs = self.min_runs_cukier75[numberf - 1]
            t_freq = self.freq_cukier(numberf)
        elif freq == "mcrae":
            t_runs = self.min_runs_mcrae82[numberf - 1]
            t_freq = self.freq_mcrae82(numberf)
        else:
            raise Exception(
                "Missing option for Frequency selection for Parameter definition, choose cukier or mcrae!"
            )

        # Get the names of the parameters to analyse
        names = ["par" + name for name in self.parameter()["name"]]

        # open the file for sensitivity results
        f = open(f"{dbname}.csv", "w+")
        np.savetxt(f, [names], "%s", delimiter=",")

        data = np.array(results)

        try:
            # convert array of void to array of float
            mod_results = np.full((data.shape[0], len(data[0])), np.nan)

            for index in range(len(data)):
                mod_results[index, :] = list(data[index])

            print("data series")
            for index in range(mod_results.shape[1]):
                x_data = mod_results[:, index]
                sens_data = efast_sensitivity(x_data, numberf, t_runs, t_freq)
                np.savetxt(f, [sens_data], delimiter=",", fmt="%1.5f")

        except:
            print("single data")
            x_data = data
            sens_data = efast_sensitivity(x_data, numberf, t_runs, t_freq)
            np.savetxt(f, [sens_data], delimiter=",", fmt="%1.5f")

        f.close()

1	# -- coding: utf-8 --
2	"""
3	EFAST algorithm transfered from CRAN R package 'fast' by Dominik Reusser (2015)
4	Copy of most functions and descriptions and implementation in SPOTPY framework by Anna Herzog (2024)
5
6	This Version of FAST differs from the previously impemented Version, as it does not require a precalculation of minimum model runs by the user. It rather uses a hard coded definition of minimum runs based on Cukier or McRae.
7	It therefore requieres considreably less model runs (for 5 Parameters 2245 (fast) vs. 71 runs (efast)).
8
9	For further information on the functions, the origninal R package and the fast algorithm please have a look at the R package description from CRAN, or see the following references:
10
11	Reusser, Dominik E., Wouter Buytaert, and Erwin Zehe.
12	"Temporal dynamics of model parameter sensitivity for computationally expensive models with FAST (Fourier Amplitude Sensitivity Test)." Water Resources Research 47 (2011): W07551.
13
14	Further References:
15	CUKIER, R. I.; LEVINE, H. B. & SHULER, K. E. Non-Linear
16	"Sensitivity Analysis Of Multi-Parameter Model Systems" Journal Of Computational Physics, 1978 , 26 , 1-42
17	CUKIER, R. I.; FORTUIN, C. M.; SHULER, K. E.; PETSCHEK, A. G. & SCHAIBLY, J. H.
18	"Study Of Sensitivity Of Coupled Reaction Systems To Uncertainties In Rate Coefficients .1. Theory" Journal Of Chemical Physics, 1973 , 59 , 3873-3878
19	SCHAIBLY, J. H. & SHULER, K. E.
20	"Study Of Sensitivity Of Coupled Reaction Systems To Uncertainties In Rate Coefficients .2. Applications" Journal Of Chemical Physics, 1973 , 59 , 3879-3888
21	CUKIER, R. I.; SCHAIBLY, J. H. & SHULER, K. E.
22	"Study Of Sensitivity Of Coupled Reaction Systems To Uncertainties In Rate Coefficients .3. Analysis Of Approximations" Journal Of Chemical Physics, 1975 , 63 , 1140-1149
23
24
25	Copyright (c) 2018 by Tobias Houska
26	This file is part of Statistical Parameter Optimization Tool for Python(SPOTPY).
27	:author: Anna Herzog, Tobias Houska, Dominik Reusser
28	"""
29
30	import warnings	3✔
31
32	import numpy as np	3✔
33
34	from ..analyser import efast_sensitivity, get_modelruns	3✔
35	from . import _algorithm	3✔
36
37
38	class efast(_algorithm):	3✔
39	"""
40	EFAST Algorithm for (distributed) parameter Sensitivity after FAST algorithm according to Cukier 1975 or McRae 1982
41	The extended Fourier Amplitude Sensitivity Test (eFAST) is a method to determine global sensitvities of a model on parameter changes
42	within relatively few model runs over time.
43	"""
44
45	_unaccepted_parameter_types = ()	3✔
46
47	def __init__(self, args, *kwargs):	3✔
48	"""
49	Input
50	----------
51	spot_setup: class
52	model: function
53	Should be callable with a parameter combination of the parameter-function
54	and return an list of simulation results (as long as evaluation list)
55	parameter: function
56	When called, it should return a random parameter combination. Which can
57	be e.g. uniform or Gaussian
58	objectivefunction: function
59	Should return the objectivefunction for a given list of a model simulation and
60	observation.
61	evaluation: function
62	Should return the true values as return by the model.
63
64	dbname: str
65	* Name of the database where parameter, objectivefunction value and simulation results will be saved.
66
67	dbformat: str
68	* ram: fast suited for short sampling time. no file will be created and results are saved in an array.
69	* csv: A csv file will be created, which you can import afterwards.
70
71	parallel: str
72	* seq: Sequentiel sampling (default): Normal iterations on one core of your cpu.
73	* mpi: Message Passing Interface: Parallel computing on cluster pcs (recommended for unix os).
74
75	save_sim: boolean
76	* True: Simulation results will be saved
77	* False: Simulation results will not be saved
78	"""
79
80	kwargs["algorithm_name"] = "EFast Sampler"	3✔
81	super(efast, self).__init__(args, *kwargs)	3✔
82
83	# hard coded repetition and frequency values from R package Fast
84	# fmt: off
85	d_m_cukier75 = [4,8,6,10,20,22,32,40,38,26,56,62,46,76,96,60,86,126,134,112,92,128,154,196,34,416,106,208,328,198,382,88,348,186,140,170,284,568,302,438,410,248,448,388,596,217,100,488,166]	3✔
86	min_runs_cukier75 = [np.nan, np.nan, 19, 39, 71, 91, 167, 243, 315, 403, 487, 579, 687, 907, 1019, 1223, 1367, 1655, 1919, 2087, 2351, 2771, 3087, 3427, 3555, 4091, 4467, 4795, 5679, 5763, 6507, 7103, 7523, 8351, 9187, 9667, 10211, 10775, 11339, 7467, 12891, 13739, 14743, 15743, 16975, 18275, 18927, 19907, 20759, 21803]	3✔
87	omega_m_cukier75 = [np.nan, np.nan, 1, 5, 11, 1, 17, 23, 19, 25, 41, 31, 23, 87, 67, 73, 58, 143, 149, 99, 119, 237, 267, 283, 151, 385, 157, 215, 449, 163, 337, 253, 375, 441, 673, 773, 875, 873, 587, 849, 623, 637, 891, 943, 1171, 1225, 1335, 1725, 1663, 2019]	3✔
88	d_m_mcrae82 = [4, 8, 6, 10, 20, 22, 32, 40, 38, 26, 56, 62, 46, 76, 96]	3✔
89	min_runs_mcrae82 = [0, 15, 27, 47, 79, 99, 175, 251, 323, 411, 495, 587, 695, 915, 1027]	3✔
90	omega_m_mcrae82 = [0, 3, 1, 5, 11, 1, 17, 23, 19, 25, 41, 31, 23, 87, 67]	3✔
91	# fmt: on
92
93	def freq_cukier(self, m, i=1, omega_before=-1):	3✔
94	"""
95	This function creates an array of independent frequencies according to
96	Cukier 1975 for usage in the eFAST method.
97
98	Parameters
99	----------
100	m: int
101	number of parameters (frequencies) needed
102	i: (intern) int
103	internally used recursion counter
104	omega_before: (intern) int
105	internally used previous frequency
106
107	Raises
108	----------
109	Exception:
110	"Parameter number m to high, not implemented"
111	Execution is stopped, if the number of Parameters is too high (max. Number of Parameters for Cukier = 50)
112
113	Returns
114	----------
115	np.array of float:
116	A 1d-Array of independent frequencies to the order of 4
117
118	Author
119	----------
120	Dominic Reusser
121	spotpy translation: Anna Herzog
122
123	"""
124
125	if i <= 1:	×
126	if m >= len(self.omega_m_cukier75):	×
NEW 127	raise Exception("Parameter number m is too high, not implemented")	×
128	else:
NEW 129	o = self.omega_m_cukier75[m - 1]	×
NEW 130	return np.append(o, self.freq_cukier(m, i + 1, o))	×
131	else:
NEW 132	o = omega_before + self.d_m_cukier75[m - i]	×
133	if i == m:	×
134	return o	×
135	else:
NEW 136	return np.append(o, self.freq_cukier(m, i + 1, o))	×
137
138	def freq_mcrae82(self, m, i=1, omega_before=-1):	3✔
139	"""
140	This function creates an array of independent frequencies according to
141	McRae 1982 for usage in the eFAST method.
142
143	Parameters
144	----------
145	m: int
146	number of parameters (frequencies) needed
147	i: (intern) int
148	internally used recursion counter
149	omega_before: (intern) int
150	internally used previous frequency
151
152	Raises
153	----------
154	Exception:
155	"Parameter number m to high, not implemented"
156	Execution is stopped, if the number of Parameters is to high (max. Number of Parameters for McRae = 15)
157
158	Returns
159	----------
160	np.array of float:
161	A 1d-Array of independent frequencies to the order of 4
162
163	Author
164	----------
165	Dominic Reusser
166	spotpy translation: Anna Herzog
167
168	"""
169
170	if i <= 1:	×
171	if m >= len(self.omega_m_mcrae82):	×
NEW 172	raise Exception("Parameter number m is too high, not implemented")	×
173	else:
NEW 174	o = self.omega_m_mcrae82[m - 1]	×
NEW 175	return np.append(o, self.freq_mcrae82(m, i + 1, o))	×
176	else:
NEW 177	o = omega_before + self.d_m_mcrae82[m - i]	×
178	if i == m:	×
179	return o	×
180	else:
NEW 181	return np.append(o, self.freq_mcrae82(m, i + 1, o))	×
182
183	def s(self, m, freq="cukier"):	3✔
184	"""
185	Function that generates a number of equally spaced values between -p1/2
186	and pi/2. The number is determined by the number of runs required of the
187	eFAST method for a number of parameters (min_runs_cukier or min_runs_mcrae)
188
189	Parameters
190	----------
191	m: int
192	number of parameters/ frequencies
193
194	freq: (optional) str
195	indicates weather to use the frequencies after 'cukier' or McRae 'mcrae'
196	Default is Cukier
197
198	Raises
199	----------
200	Exeption:
201	"Missing option for Frequency selection for Parameter definition, choose cukier or mcrae!"
202	Execution is stopped if an invalid method for the frequency selection is provided
203
204	Returns
205	----------
206	2D array of float:
207	an array of equally spaced values between -pi/2 and pi/2
208
209	Author
210	----------
211	Dominic Reusser
212	spotpy translation: Anna Herzog
213
214	"""
215
NEW 216	if freq == "cukier":	×
217	min_runs = self.min_runs_cukier75	×
NEW 218	elif freq == "mcrae":	×
219	min_runs = self.min_runs_mcrae82	×
220	else:
NEW 221	raise Exception(	×
222	"Missing option for Frequency selection for Parameter definition, choose cukier or mcrae!"
223	)
224
NEW 225	r = min_runs[m - 1]	×
NEW 226	r_range = np.array(range(1, r + 1))	×
NEW 227	s = np.pi / r * (2 * r_range - r - 1) / 2	×
228
NEW 229	return s	×
230
231	def S(self, m, freq="cukier"):	3✔
232	"""
233	Function to generate an array of values, which provides the base for the parameterdistribution
234	in the FAST method. It is usually not used directly but called from the
235	fast_parameters function.
236
237	Parameters
238	----------
239	m: int
240	number of parameters/frequencies
241
242	freq: (optional) str
243	indicates weather to use the frequencies after 'cukier' or McRae 'mcrae'
244	Default is Cukier
245
246	Raises
247	----------
248	Exception:
249	"Missing option for Frequency selection for Parameter definition, choose cukier or mcrae!"
250	Execution is stopped if an invalid method for the frequency selection is provided
251
252	Returns
253	----------
254	2D array of float:
255	an array with the shape (number of runs, parameters)
256
257	Author
258	----------
259	Dominic Reusser
260	spotpy translation: Anna Herzog
261	"""
262
NEW 263	if freq == "cukier":	×
264	omega = self.freq_cukier(m)	×
NEW 265	elif freq == "mcrae":	×
266	omega = self.freq_mcrae82(m)	×
267	else:
NEW 268	raise Exception(	×
269	"Missing option for Frequency selection for Parameter definition, choose cukier or mcrae!"
270	)
271
UNCOV 272	tab = np.outer(self.s(m, freq), omega)	×
273
NEW 274	toreturn = np.arcsin(np.sin(tab)) / np.pi	×
275
276	# naming array dimensions is not possible with numpy but convention would be toreturn.shape () = (runs, parameternames)
277
NEW 278	return toreturn	×
279
280	def rerange(self, data, min_goal=0, max_goal=1, center=np.nan):	3✔
281	"""
282	This function performes a linear transformation of the data, such that
283	afterwards range(data) = (theMin, theMax)
284
285	Parameters
286	----------
287	data: array of flaot
288	an array with the data to transform
289	in this case the parameter distribution generated by function S
290	min_goal: float
291	the new minimum value (lower parameter bound)
292	max_goal: float
293	the new maximum value (upper parameter bound)
294	center: (optional) float
295	indicates which old value should become the new center
296	default: (max_goal+min_goal)/2
297
298	Returns
299	----------
300	2D array of float:
301	array with the transformed data
302
303	Author
304	----------
305	Dominic Reusser
306	spotpy translation: Anna Herzog
307
308	"""
309
310	min_data = min(data)	×
311	max_data = max(data)	×
312
313	if np.isnan(center):	×
314	max_data = max_data	×
315	dat = data - min_data	×
NEW 316	dat = dat / (max_data - min_data) * (max_goal - min_goal)	×
317	dat = dat + min_goal	×
NEW 318	return dat	×
319	else:
320	# split linear transformation, the data is split into two segments, one blow center, one above,
321	# each segment undergoes its own linear transformation
322	below = data <= center	×
323	above = data > center	×
324	new_data = np.copy(data)	×
NEW 325	np.place(	×
326	new_data,
327	below,
328	self.rerange(
329	np.insert(data[below], 0, center),
330	min_goal=min_goal,
331	max_goal=max_goal / 2,
332	)[1:],
333	)
NEW 334	np.place(	×
335	new_data,
336	above,
337	self.rerange(
338	np.insert(data[above], 0, center),
339	min_goal=max_goal / 2,
340	max_goal=max_goal,
341	)[1:],
342	)
NEW 343	return new_data	×
344
345	def fast_parameters(	3✔
346	self, minimum, maximum, freq="cukier", logscale=np.nan, names=np.nan
347	):
348	"""
349	Function for the creation of a FAST Parameter set based on their range
350
351	Parameters
352	----------
353	minimum: array of float
354	array containing the lower parameter boundaries
355	maximum: array of float
356	array containing the upper parameter boundaries
357	names: (optional) str
358	array containing the parameter names
359	freq: (optional) str
360	indicates weather to use the frequencies after 'cukier' or McRae 'mcrae'
361	Default is Cukier
362	logscale: (optional) bool
363	array containing bool values indicating weather a parameter is varied
364	on a logarithmic scale. In that case minimum and maximum are exponents
365
366	Raises
367	----------
368	Exception:
369	"Expecting minimum and maximum of same size"
370	Execution is stopped if the number of minimum, maximum or logscale values differ
371
372	Returns
373	----------
374	value: array of float
375	an array with the shape (number of runs, parameters) containing the
376	required parameter sets
377
378	Author
379	----------
380	Dominic Reusser
381	spotpy translation: Anna Herzog
382
383	"""
384
385	if np.isnan(logscale):	×
386	logscale = np.full(minimum.shape, False)	×
387	if np.isnan(names):	×
NEW 388	names = ["P" + str(i) for i in range(minimum.shape[0])]	×
NEW 389	names = np.array(names)	×
390
UNCOV 391	n = len(minimum)	×
392
NEW 393	if n != len(maximum):	×
NEW 394	raise Exception("Expecting minimum and maximum of same size")	×
NEW 395	elif n != len(names):	×
NEW 396	raise Exception("Expecting minimum and names of same size")	×
NEW 397	elif n != len(logscale):	×
NEW 398	raise Exception("Expecting minimum and logscale of same size")	×
399
NEW 400	toreturn = self.S(m=n, freq=freq) # par_names = names,	×
401
NEW 402	for i in range(0, n):	×
NEW 403	toreturn[:, i] = self.rerange(toreturn[:, i], minimum[i], maximum[i])	×
404	if logscale[i]:	×
NEW 405	toreturn[:, i] = 10 ** toreturn[:, i]	×
406
NEW 407	return toreturn	×
408
409	def sample(self, repetitions, freq="cukier", logscale=np.nan):	3✔
410	"""
411	Samples from the eFAST algorithm.
412
413	Input
414	----------
415	repetitions: int
416	Maximum number of runs.
417	freq: (optional) str
418	indicates weather to use the frequencies after 'cukier' or McRae 'mcrae'
419	Default is Cukier
420	logscale: (optional) bool
421	array containing bool values indicating weather a parameter is varied
422	on a logarithmic scale. In that case minimum and maximum are exponents
423
424	Raises
425	----------
426	Warning:
427	"specified number of repetitions is smaller than minimum required for eFAST analysis!
428	Simulations will be performed but eFAST analysis might not be possible"
429
430	If the specified number of repetitions is smaller than required by the algorithm, the
431	algorithm will still execute, but the usage of spotpy.analyser.efast.calc_sensitivity()
432	might not be possible. Rather define a large number of reps as only required minimum runs
433	will be performed anyway.
434
435	Warning:
436	"Number of specified repetitions equals or exceeds number of minimum required repetitions for eFAST analysis
437	program will stop after required runs."
438
439	Returns
440	----------
441	database:
442	spotpy database in chosen format with objective function values, parameter values and simulation results
443
444	Author
445	----------
446	Tobias Houska, Anna Herzog
447	"""
448
449	print("generating eFAST Parameters")	×
450	# Get the names of the parameters to analyse
451	names = self.parameter()["name"]	×
452	# Get the minimum and maximum value for each parameter from the
453	# distribution
454	parmin, parmax = self.parameter()["minbound"], self.parameter()["maxbound"]	×
455	self.numberf = len(parmin)	×
NEW 456	min_runs = self.min_runs_cukier75[self.numberf - 1]	×
457
458	if min_runs > repetitions:	×
NEW 459	warnings.warn(	×
460	"specified number of repetitions is smaller than minimum required for eFAST analysis!\n"
461	"Simulations will be performed but eFAST analysis might not be possible"
462	)
463	else:
464	repetitions = min_runs	×
NEW 465	print(	×
466	"Number of specified repetitions equals or exceeds number of minimum required repetitions for eFAST analysis\n"
467	"program will stop after required runs."
468	)
469
470	self.set_repetiton(repetitions)	×
471
472	# Generate Matrix with eFAST parameter sets
473	N = self.fast_parameters(parmin, parmax, freq, logscale)	×
474
475	print("Starting the eFast algorithm with {} repetitions...".format(repetitions))	×
476
477	# A generator that produces parametersets if called
NEW 478	param_generator = ((rep, N[rep, :]) for rep in range(int(repetitions)))	×
UNCOV 479	for rep, randompar, simulations in self.repeat(param_generator):	×
480	# A function that calculates the fitness of the run and the manages the database
481	self.postprocessing(rep, randompar, simulations)	×
482
483	if self.breakpoint == "write" or self.breakpoint == "readandwrite":	×
484	if rep >= lastbackup + self.backup_every_rep:	×
NEW 485	work = (rep, N[rep, :])	×
486	self.write_breakdata(self.dbname, work)	×
487	lastbackup = rep	×
488	self.final_call()	×
489
490	def calc_sensitivity(self, results, dbname, freq="cukier"):	3✔
491	"""
492	Function to calculate the sensitivity for a data series (e.g. a time series)
493	based on the eFAST algorithm.
494
495	Parameters
496	----------
497	results: np.array of void
498	spopty restults array as loaded with spotpy.analyser.load_csv_results()
499	dbname: (optional) str
500	name of file to store sensitivity values
501	freq: (optional) str
502	indicates weather to use the frequencies after Cukier 'cukier' or McRae 'mcrae'
503	Default is Cukier
504
505	Raises
506	----------
507	Exeption:
508	"Missing option for Frequency selection for Parameter definition, choose cukier or mcrae!"
509	Execution is stopped if an invalid method for the frequency selection is provided
510
511	Returns
512	----------
513	database:
514	database containing the temporal parameter sensitivities with seperate column
515	for each parameter and row for e.g. time step
516
517	Author
518	----------
519	Dominic Reusser
520	spotpy translation: Anna Herzog
521
522	"""
UNCOV 523	numberf = len(self.parameter()["minbound"])	×
524
525	# sens_data = np.full((len(results[0]), numberf), np.nan)
526
527	# idendtify frequency
NEW 528	if freq == "cukier":	×
NEW 529	t_runs = self.min_runs_cukier75[numberf - 1]	×
530	t_freq = self.freq_cukier(numberf)	×
NEW 531	elif freq == "mcrae":	×
NEW 532	t_runs = self.min_runs_mcrae82[numberf - 1]	×
UNCOV 533	t_freq = self.freq_mcrae82(numberf)	×
534	else:
NEW 535	raise Exception(	×
536	"Missing option for Frequency selection for Parameter definition, choose cukier or mcrae!"
537	)
538
539	# Get the names of the parameters to analyse
540	names = ["par" + name for name in self.parameter()["name"]]	×
541
542	# open the file for sensitivity results
NEW 543	f = open(f"{dbname}.csv", "w+")	×
544	np.savetxt(f, [names], "%s", delimiter=",")	×
545
546	data = np.array(results)	×
547
NEW 548	try:	×
549	# convert array of void to array of float
550	mod_results = np.full((data.shape[0], len(data[0])), np.nan)	×
551
NEW 552	for index in range(len(data)):	×
553	mod_results[index, :] = list(data[index])	×
554
555	print("data series")	×
NEW 556	for index in range(mod_results.shape[1]):	×
557	x_data = mod_results[:, index]	×
558	sens_data = efast_sensitivity(x_data, numberf, t_runs, t_freq)	×
NEW 559	np.savetxt(f, [sens_data], delimiter=",", fmt="%1.5f")	×
560
561	except:	×
562	print("single data")	×
563	x_data = data	×
564	sens_data = efast_sensitivity(x_data, numberf, t_runs, t_freq)	×
NEW 565	np.savetxt(f, [sens_data], delimiter=",", fmt="%1.5f")	×
566
UNCOV 567	f.close()	×

thouska / spotpy / 15296154421

Source File Press 'n' to go to next uncovered line, 'b' for previous

Source File
Press 'n' to go to next uncovered line, 'b' for previous