13287027981

Committed 12 Feb 2025 01:57PM UTC coverage: 13.126% (-64.2%) from 77.332%

Build # 13287027981

Build Type

Pull #330

github

Committed by

web-flow

Commit Message

Merge e8d40e509 into 7e5e3f4f2

Pull Request Pull Request #330: Pr 316

Run Details

29 of 201 new or added lines in 7 files covered. (14.43%)

245 existing lines in 4 files now uncovered.

726 of 5531 relevant lines covered (13.13%)

0.53 hits per line

Source File
Press 'n' to go to next uncovered line, 'b' for previous

15.22

/src/spotpy/algorithms/efast.py

# -*- coding: utf-8 -*-
"""
EFAST algorithm transfered from CRAN R package 'fast' by Dominik Reusser (2015)
Copy of most functions and descriptions and implementation in SPOTPY framework by Anna Herzog (2024)

This Version of FAST differs from the previously impemented Version, as it does not require a precalculation of minimum model runs by the user. It rather uses a hard coded definition of minimum runs based on Cukier or McRae.
It therefore requieres considreably less model runs (for 5 Parameters 2245 (fast) vs. 71 runs (efast)).

For further information on the functions, the origninal R package and the fast algorithm please have a look at the R package description from CRAN, or see the following references:

Reusser, Dominik E., Wouter Buytaert, and Erwin Zehe. 
"Temporal dynamics of model parameter sensitivity for computationally expensive models with FAST (Fourier Amplitude Sensitivity Test)." Water Resources Research 47 (2011): W07551.

Further References:
CUKIER, R. I.; LEVINE, H. B. & SHULER, K. E. Non-Linear 
"Sensitivity Analysis Of Multi-Parameter Model Systems" Journal Of Computational Physics, 1978 , 26 , 1-42
CUKIER, R. I.; FORTUIN, C. M.; SHULER, K. E.; PETSCHEK, A. G. & SCHAIBLY, J. H. 
"Study Of Sensitivity Of Coupled Reaction Systems To Uncertainties In Rate Coefficients .1. Theory" Journal Of Chemical Physics, 1973 , 59 , 3873-3878
SCHAIBLY, J. H. & SHULER, K. E. 
"Study Of Sensitivity Of Coupled Reaction Systems To Uncertainties In Rate Coefficients .2. Applications" Journal Of Chemical Physics, 1973 , 59 , 3879-3888
CUKIER, R. I.; SCHAIBLY, J. H. & SHULER, K. E. 
"Study Of Sensitivity Of Coupled Reaction Systems To Uncertainties In Rate Coefficients .3. Analysis Of Approximations" Journal Of Chemical Physics, 1975 , 63 , 1140-1149


Copyright (c) 2018 by Tobias Houska
This file is part of Statistical Parameter Optimization Tool for Python(SPOTPY).
:author: Anna Herzog, Tobias Houska, Dominik Reusser
"""

import warnings
import numpy as np

from ..analyser import   efast_sensitivity, get_modelruns
from . import _algorithm

class efast(_algorithm):
    """
    EFAST Algorithm for (distributed) parameter Sensitivity after FAST algorithm according to Cukier 1975 or McRae 1982
    The Fourier Amplitude Sensitivity Test (FAST) is a method to determine global sensitvities of a model on parameter changes
    within relatively few model runs. 
    """

    _unaccepted_parameter_types = ()

    def __init__(self, *args, **kwargs):
        """
        Input
        ----------
        spot_setup: class
            model: function
                Should be callable with a parameter combination of the parameter-function
                and return an list of simulation results (as long as evaluation list)
            parameter: function
                When called, it should return a random parameter combination. Which can
                be e.g. uniform or Gaussian
            objectivefunction: function
                Should return the objectivefunction for a given list of a model simulation and
                observation.
            evaluation: function
                Should return the true values as return by the model.

        dbname: str
            * Name of the database where parameter, objectivefunction value and simulation results will be saved.

        dbformat: str
            * ram: fast suited for short sampling time. no file will be created and results are saved in an array.
            * csv: A csv file will be created, which you can import afterwards.

        parallel: str
            * seq: Sequentiel sampling (default): Normal iterations on one core of your cpu.
            * mpi: Message Passing Interface: Parallel computing on cluster pcs (recommended for unix os).

        save_sim: boolean
            * True:  Simulation results will be saved
            * False: Simulation results will not be saved
        """

        

        kwargs["algorithm_name"] = "EFast Sampler"
        super(efast, self).__init__(*args, **kwargs)

    # hard coded repetition and frequency values from R package Fast
    d_m_cukier75 = [4,8,6,10,20,22,32,40,38,26,56,62,46,76,96,60,86,126,134,112,92,128,154,196,34,416,106,208,328,198,382,88,348,186,140,170,284,568,302,438,410,248,448,388,596,217,100,488,166]
    min_runs_cukier75 = [np.nan, np.nan, 19, 39, 71, 91, 167, 243, 315, 403, 487, 579, 687, 907, 1019, 1223, 1367, 1655, 1919, 2087, 2351, 2771, 3087, 3427, 3555, 4091, 4467, 4795, 5679, 5763, 6507, 7103, 7523, 8351, 9187, 9667, 10211, 10775, 11339, 7467, 12891, 13739, 14743, 15743, 16975, 18275, 18927, 19907, 20759, 21803]
    omega_m_cukier75 = [np.nan, np.nan, 1, 5, 11, 1, 17, 23, 19, 25, 41, 31, 23, 87, 67, 73, 58, 143, 149, 99, 119, 237, 267, 283, 151, 385, 157, 215, 449, 163, 337, 253, 375, 441, 673, 773, 875, 873, 587, 849, 623, 637, 891, 943, 1171, 1225, 1335, 1725, 1663, 2019]
    d_m_mcrae82 = [4, 8, 6, 10, 20, 22, 32, 40, 38, 26, 56, 62, 46, 76, 96]
    min_runs_mcrae82 = [0, 15, 27, 47, 79, 99, 175, 251, 323, 411, 495, 587, 695, 915, 1027]
    omega_m_mcrae82 = [0, 3, 1, 5, 11, 1, 17, 23, 19, 25, 41, 31, 23, 87, 67]

    def freq_cukier(self, m, i = 1, omega_before = -1):
    
        """
        This function creates an array of independant frequencies accroding to 
        Cukier 1975 for usage in the eFAST method.

        Parameters
        ----------
        m: int
            number of parameters (frequencies) needed
        i: (intern) int
            internally used recursion counter
        omega_before: (intern) int
            internally used previous frequency

        Raises
        ----------
        Exeption:
            "Parameter number m to high, not implemented"
            Execution is stopped, if the number of Parameters is to high (max. Number of Parameters for Cukier = 50)    

        Returns
        ----------
        np.array of float:
            A 1d-Array of independant frequencies to the order of 4

        Author
        ----------
        Dominic Reusser
        spotpy translation: Anna Herzog

        """

        if i <= 1:
            if m >= len(self.omega_m_cukier75):
                raise Exception("Parameter number m to high, not implemented") 
            else:
                o = self.omega_m_cukier75[m-1]
                return np.append(o, self.freq_cukier(m, i+1, o))
        else:
            o = omega_before + self.d_m_cukier75[m-i]
            if i == m:
                return o
            else:
                return np.append(o, self.freq_cukier(m, i+1, o))
        

    def freq_mcrae82(self, m, i = 1, omega_before = -1):
    
        """
        This function creates an array of independant frequencies accroding to 
        McRae 1982 for usage in the eFAST method.

        Parameters
        ----------
        m: int
            number of parameters (frequencies) needed
        i: (intern) int
            internally used recursion counter
        omega_before: (intern) int
            internally used previous frequency

        Raises
        ----------
        Exeption:
            "Parameter number m to high, not implemented"
            Execution is stopped, if the number of Parameters is to high (max. Number of Parameters for McRae = 15)    

        Returns
        ----------
        np.array of float:
            A 1d-Array of independant frequencies to the order of 4

        Author
        ----------
        Dominic Reusser
        spotpy translation: Anna Herzog

        """

        if i <= 1:
            if m >= len(self.omega_m_mcrae82):
                raise Exception("Parameter number m to high, not implemented") 
            else:
                o = self.omega_m_mcrae82[m-1]
                return np.append(o, self.freq_mcrae82(m, i+1, o))
        else:
            o = omega_before + self.d_m_mcrae82[m-i]
            if i == m:
                return o
            else:
                return np.append(o, self.freq_mcrae82(m, i+1, o))
        

    def s(self, m, freq = 'cukier'):

        """
        Function that generates a number of equally spaced values between -p1/2 
        and pi/2. The number is determined by the number of runs required of the
        eFAST method for a number of parameters (min_runs_cukier or min_runs_mcrae)

        Parameters
        ----------
        m: int
            number of parameters/ frequencies

        freq: (optional) str
            indicates weather to use the frequencies after 'cukier' or McRae 'mcrae'
            Default is Cukier

        Raises
        ----------
        Exeption:
            "Missing option for Frequency selection for Parameter definition, choose cukier or mcrae!"
            Execution is stopped if an invalid method for the frequency selection is provided

        Returns
        ----------
        2D array of float:
            an array of equally spaced values between -pi/2 and pi/2

        Author
        ----------
        Dominic Reusser
        spotpy translation: Anna Herzog

        """

        if freq == 'cukier':
            min_runs = self.min_runs_cukier75
        elif freq == 'mcrae':
            min_runs = self.min_runs_mcrae82
        else:
            raise Exception("Missing option for Frequency selection for Parameter definition, choose cukier or mcrae!")
        
        r = min_runs[m-1]
        r_range = np.array(range(1,r+1))
        s = np.pi/r * (2*r_range-r-1)/2
    
        return s


    def S(self, m, freq = 'cukier'):
    
        """
        Function to generate an array of values, which provides the base for the parameterdistribution 
        in the FAST method. It is usally not used directly but called from the 
        fast_parameters function.

        Parameters
        ----------
        m: int
            number of parameters/frequencies

        freq: (optional) str
            indicates weather to use the frequencies after 'cukier' or McRae 'mcrae'
            Default is Cukier

        Raises
        ----------
        Exeption:
            "Missing option for Frequency selection for Parameter definition, choose cukier or mcrae!"
            Execution is stopped if an invalid method for the frequency selection is provided

        Returns
        ----------
        2D array of float:
            an array with the shape (number of runs, parameters)

        Author
        ----------
        Dominic Reusser
        spotpy translation: Anna Herzog
        """

        if freq == 'cukier':
            omega = self.freq_cukier(m)
        elif freq == 'mcrae':
            omega = self.freq_mcrae82(m)
        else:
            raise Exception("Missing option for Frequency selection for Parameter definition, choose cukier or mcrae!")
    
        tab = np.outer(self.s(m, freq), omega)
    
        toreturn = np.arcsin(np.sin(tab))/np.pi
    
        # naming array dimensions is not possible with numpy but convention would be toreturn.shape () = (runs, parameternames)
    
        return toreturn


    def rerange(self, data, min_goal = 0, max_goal = 1, center = np.nan):

        """
        This function performes a linear transformation of the data, such that
        afterwards range(data) = (theMin, theMax)

        Parameters
        ----------
        data: array of flaot
            an array with the data to transform
            in this case the parameter distribution generated by function S
        min_goal: float
            the new minimum value (lower parameter bound)
        max_goal: float
            the new maximum value (upper parameter bound)
        center: (optional) float
            indicates which old value should become the new center
            default: (max_goal+min_goal)/2

        Returns
        ----------
        2D array of float:
            array with the transformed data

        Author
        ----------
        Dominic Reusser
        spotpy translation: Anna Herzog
    
        """
    
        min_data = min(data)
        max_data = max(data)
    
        if np.isnan(center):
            max_data = max_data
            dat = data - min_data
            dat = dat/(max_data-min_data) * (max_goal-min_goal)
            dat = dat + min_goal
            return(dat)
        else:
            # split linear transformation, the data is split into two segments, one blow center, one above, 
            # each segment undergoes it's own linear transformation
            below = data <= center
            above = data > center
            new_data = np.copy(data)
            np.place(new_data, below, self.rerange(np.insert(data[below], 0, center), min_goal = min_goal, max_goal= max_goal/2)[1:])
            np.place(new_data, above, self.rerange(np.insert(data[above], 0, center), min_goal = max_goal/2, max_goal= max_goal)[1:])
            return(new_data)
    

    def fast_parameters(self, minimum, maximum, freq = 'cukier', logscale = np.nan, names = np.nan):
    
        """
        Function for the creation of a FAST Parameter set based on thier range

        Parameters
        ----------
        minimum: array of float 
            array containing the lower parameter boundries
        maximum: array of float 
            array containing the upper parameter boundries
        names: (optional) str
            array containing the parameter names
        freq: (optional) str
            indicates weather to use the frequencies after 'cukier' or McRae 'mcrae'
            Default is Cukier
        logscale: (optional) bool
            array containing bool values indicating weather a parameter is varied
            on a logarithmic scale. In that case minimum and maximum are exponents

        Raises
        ----------
        Exeption:
            "Expecting minimum and maximum of same size"
            Execution is stopped if the number of minimum, maximum or logscale values differ

        Returns
        ----------
        value: array of float
            an array with the shape (number of runs, parameters) containing the 
            required parameter sets

        Author
        ----------
        Dominic Reusser
        spotpy translation: Anna Herzog

        """
        
        if np.isnan(logscale):
            logscale = np.full(minimum.shape, False)
        if np.isnan(names):
            names = ["P"+str(i) for i in range(minimum.shape[0])]
            names = np.array(names)  
    
        n = len(minimum)
    
        if (n != len(maximum)):
            raise Exception("Expecting minimum and maximum of same size") 
        elif(n != len(names)):
            raise Exception("Expecting minimum and names of same size") 
        elif(n != len(logscale)):
            raise Exception("Expecting minimum and logscale of same size") 
    
        toreturn = self.S(m=n, freq = freq) #par_names = names, 
    
        for i in range(0,n):
            toreturn[:,i] = self.rerange(toreturn[:,i], minimum[i], maximum[i])
            if logscale[i]:
                toreturn[:,i] = 10**toreturn[:,i]
            
        return toreturn


    def sample(self, repetitions, freq = 'cukier', logscale = np.nan):
        """
        Samples from the eFAST algorithm.

        Input
        ----------
        repetitions: int
            Maximum number of runs.
        freq: (optional) str
            indicates weather to use the frequencies after 'cukier' or McRae 'mcrae'
            Default is Cukier
        logscale: (optional) bool
            array containing bool values indicating weather a parameter is varied
            on a logarithmic scale. In that case minimum and maximum are exponents

        Raises
        ----------
        Warning:
            "specified number of repetitions is smaller than minimum required for eFAST analysis!
            Simultions will be perfomed but eFAST analysis might not be possible"

            If the specified number of repetitions is smaller than required by the algorithm, the
            algortihm will still execute, but the usage of spotpy.analyser.efast.calc_sensitivity() 
            might not be possible. Rather define a large number of reps as only required minimum runs
            will be performed anyway. 
        
        Warning:
            "Number of specified repetitions equals or exeeds number of minimum required repetitions for eFAST analysis
             programm will stop after required runs."

        Returns
        ----------
        database:
            spotpy data base in chosen format with objective function values, parameter values and simulation results

        Author
        ----------
        Tobias Houska, Anna Herzog
        """

        print("generating eFAST Parameters")
        # Get the names of the parameters to analyse
        names = self.parameter()["name"]
        # Get the minimum and maximum value for each parameter from the
        # distribution
        parmin, parmax = self.parameter()["minbound"], self.parameter()["maxbound"]
        self.numberf = len(parmin)
        min_runs = self.min_runs_cukier75[self.numberf-1]

        if min_runs > repetitions:
            warnings.warn("specified number of repetitions is smaller than minimum required for eFAST analysis!\n"+
                          "Simultions will be perfomed but eFAST analysis might not be possible")
        else:
            repetitions = min_runs
            print("Number of specified repetitions equals or exeeds number of minimum required repetitions for eFAST analysis\n"+
                  "programm will stop after required runs.")

        self.set_repetiton(repetitions)

        # Generate Matrix with eFAST parameter sets
        N = self.fast_parameters(parmin, parmax, freq, logscale)

        print("Starting the eFast algorithm with {} repetitions...".format(repetitions))

        # A generator that produces parametersets if called
        param_generator = (
            (rep, N[rep,:]) for rep in range(int(repetitions))
        )
        for rep, randompar, simulations in self.repeat(param_generator):
            # A function that calculates the fitness of the run and the manages the database
            self.postprocessing(rep, randompar, simulations)

            if self.breakpoint == "write" or self.breakpoint == "readandwrite":
                if rep >= lastbackup + self.backup_every_rep:
                    work = (rep, N[rep,:])
                    self.write_breakdata(self.dbname, work)
                    lastbackup = rep
        self.final_call()

    def calc_sensitivity(self, results, dbname, freq = 'cukier'):
        """
        Function to calcultae the sensitivity for a data series (eg. a time series) 
        based on the eFAST algorithm.

        Parameters
        ----------
        results: np.array of void
            spopty restults array as loaded with spotpy.analyser.load_csv_results()
        dbname: (optional) str
            name of file to store sensitivity values        
        freq: (optional) str
            indicates weather to use the frequencies after Cukier 'cukier' or McRae 'mcrae'
            Default is Cukier

        Raises
        ----------
        Exeption:
            "Missing option for Frequency selection for Parameter definition, choose cukier or mcrae!"
            Execution is stopped if an invalid method for the frequency selection is provided

        Returns
        ----------
        database:
            database containing the temporal parameter sensitivities with seperate column
            for each parameter and row for eg. time step

        Author
        ----------
        Dominic Reusser
        spotpy translation: Anna Herzog
    
        """ 
        numberf = len(self.parameter()["minbound"])

        # sens_data = np.full((len(results[0]), numberf), np.nan)
        
        # idendtify frequency
        if freq == 'cukier':
            t_runs = self.min_runs_cukier75[numberf-1]
            t_freq = self.freq_cukier(numberf)
        elif freq == 'mcrae': 
            t_runs = self.min_runs_mcrae82[numberf-1]
            t_freq = self.freq_mcrae82(numberf)
        else:
            raise Exception("Missing option for Frequency selection for Parameter definition, choose cukier or mcrae!")
    
        # Get the names of the parameters to analyse
        names = ["par" + name for name in self.parameter()["name"]]

        # open the file for sensitivity results
        f = open(dbname+".csv", 'w+')
        np.savetxt(f, [names], "%s", delimiter=",")
        
        data = np.array(results)
        
        try:            
            # convert array of void to array of float
            mod_results = np.full((data.shape[0], len(data[0])), np.nan)

            for index in range(len(data)):        
                mod_results[index, :] = list(data[index])
      
            print("data series")
            for index in range(mod_results.shape[1]):    
                x_data = mod_results[:, index]
                sens_data = efast_sensitivity(x_data, numberf, t_runs, t_freq)
                np.savetxt(f, [sens_data], delimiter=",", fmt='%1.5f')
                
        except:
            print("single data")
            x_data = data
            sens_data = efast_sensitivity(x_data, numberf, t_runs, t_freq)
            np.savetxt(f, [sens_data], delimiter=",", fmt='%1.5f')
        
        f.close()

1	# -- coding: utf-8 --
2	"""
3	EFAST algorithm transfered from CRAN R package 'fast' by Dominik Reusser (2015)
4	Copy of most functions and descriptions and implementation in SPOTPY framework by Anna Herzog (2024)
5
6	This Version of FAST differs from the previously impemented Version, as it does not require a precalculation of minimum model runs by the user. It rather uses a hard coded definition of minimum runs based on Cukier or McRae.
7	It therefore requieres considreably less model runs (for 5 Parameters 2245 (fast) vs. 71 runs (efast)).
8
9	For further information on the functions, the origninal R package and the fast algorithm please have a look at the R package description from CRAN, or see the following references:
10
11	Reusser, Dominik E., Wouter Buytaert, and Erwin Zehe.
12	"Temporal dynamics of model parameter sensitivity for computationally expensive models with FAST (Fourier Amplitude Sensitivity Test)." Water Resources Research 47 (2011): W07551.
13
14	Further References:
15	CUKIER, R. I.; LEVINE, H. B. & SHULER, K. E. Non-Linear
16	"Sensitivity Analysis Of Multi-Parameter Model Systems" Journal Of Computational Physics, 1978 , 26 , 1-42
17	CUKIER, R. I.; FORTUIN, C. M.; SHULER, K. E.; PETSCHEK, A. G. & SCHAIBLY, J. H.
18	"Study Of Sensitivity Of Coupled Reaction Systems To Uncertainties In Rate Coefficients .1. Theory" Journal Of Chemical Physics, 1973 , 59 , 3873-3878
19	SCHAIBLY, J. H. & SHULER, K. E.
20	"Study Of Sensitivity Of Coupled Reaction Systems To Uncertainties In Rate Coefficients .2. Applications" Journal Of Chemical Physics, 1973 , 59 , 3879-3888
21	CUKIER, R. I.; SCHAIBLY, J. H. & SHULER, K. E.
22	"Study Of Sensitivity Of Coupled Reaction Systems To Uncertainties In Rate Coefficients .3. Analysis Of Approximations" Journal Of Chemical Physics, 1975 , 63 , 1140-1149
23
24
25	Copyright (c) 2018 by Tobias Houska
26	This file is part of Statistical Parameter Optimization Tool for Python(SPOTPY).
27	:author: Anna Herzog, Tobias Houska, Dominik Reusser
28	"""
29
30	import warnings	4✔
31	import numpy as np	4✔
32
33	from ..analyser import efast_sensitivity, get_modelruns	4✔
34	from . import _algorithm	4✔
35
36	class efast(_algorithm):	4✔
37	"""
38	EFAST Algorithm for (distributed) parameter Sensitivity after FAST algorithm according to Cukier 1975 or McRae 1982
39	The Fourier Amplitude Sensitivity Test (FAST) is a method to determine global sensitvities of a model on parameter changes
40	within relatively few model runs.
41	"""
42
43	_unaccepted_parameter_types = ()	4✔
44
45	def __init__(self, args, *kwargs):	4✔
46	"""
47	Input
48	----------
49	spot_setup: class
50	model: function
51	Should be callable with a parameter combination of the parameter-function
52	and return an list of simulation results (as long as evaluation list)
53	parameter: function
54	When called, it should return a random parameter combination. Which can
55	be e.g. uniform or Gaussian
56	objectivefunction: function
57	Should return the objectivefunction for a given list of a model simulation and
58	observation.
59	evaluation: function
60	Should return the true values as return by the model.
61
62	dbname: str
63	* Name of the database where parameter, objectivefunction value and simulation results will be saved.
64
65	dbformat: str
66	* ram: fast suited for short sampling time. no file will be created and results are saved in an array.
67	* csv: A csv file will be created, which you can import afterwards.
68
69	parallel: str
70	* seq: Sequentiel sampling (default): Normal iterations on one core of your cpu.
71	* mpi: Message Passing Interface: Parallel computing on cluster pcs (recommended for unix os).
72
73	save_sim: boolean
74	* True: Simulation results will be saved
75	* False: Simulation results will not be saved
76	"""
77
78
79
NEW 80	kwargs["algorithm_name"] = "EFast Sampler"	×
NEW 81	super(efast, self).__init__(args, *kwargs)	×
82
83	# hard coded repetition and frequency values from R package Fast
84	d_m_cukier75 = [4,8,6,10,20,22,32,40,38,26,56,62,46,76,96,60,86,126,134,112,92,128,154,196,34,416,106,208,328,198,382,88,348,186,140,170,284,568,302,438,410,248,448,388,596,217,100,488,166]	4✔
85	min_runs_cukier75 = [np.nan, np.nan, 19, 39, 71, 91, 167, 243, 315, 403, 487, 579, 687, 907, 1019, 1223, 1367, 1655, 1919, 2087, 2351, 2771, 3087, 3427, 3555, 4091, 4467, 4795, 5679, 5763, 6507, 7103, 7523, 8351, 9187, 9667, 10211, 10775, 11339, 7467, 12891, 13739, 14743, 15743, 16975, 18275, 18927, 19907, 20759, 21803]	4✔
86	omega_m_cukier75 = [np.nan, np.nan, 1, 5, 11, 1, 17, 23, 19, 25, 41, 31, 23, 87, 67, 73, 58, 143, 149, 99, 119, 237, 267, 283, 151, 385, 157, 215, 449, 163, 337, 253, 375, 441, 673, 773, 875, 873, 587, 849, 623, 637, 891, 943, 1171, 1225, 1335, 1725, 1663, 2019]	4✔
87	d_m_mcrae82 = [4, 8, 6, 10, 20, 22, 32, 40, 38, 26, 56, 62, 46, 76, 96]	4✔
88	min_runs_mcrae82 = [0, 15, 27, 47, 79, 99, 175, 251, 323, 411, 495, 587, 695, 915, 1027]	4✔
89	omega_m_mcrae82 = [0, 3, 1, 5, 11, 1, 17, 23, 19, 25, 41, 31, 23, 87, 67]	4✔
90
91	def freq_cukier(self, m, i = 1, omega_before = -1):	4✔
92
93	"""
94	This function creates an array of independant frequencies accroding to
95	Cukier 1975 for usage in the eFAST method.
96
97	Parameters
98	----------
99	m: int
100	number of parameters (frequencies) needed
101	i: (intern) int
102	internally used recursion counter
103	omega_before: (intern) int
104	internally used previous frequency
105
106	Raises
107	----------
108	Exeption:
109	"Parameter number m to high, not implemented"
110	Execution is stopped, if the number of Parameters is to high (max. Number of Parameters for Cukier = 50)
111
112	Returns
113	----------
114	np.array of float:
115	A 1d-Array of independant frequencies to the order of 4
116
117	Author
118	----------
119	Dominic Reusser
120	spotpy translation: Anna Herzog
121
122	"""
123
NEW 124	if i <= 1:	×
NEW 125	if m >= len(self.omega_m_cukier75):	×
NEW 126	raise Exception("Parameter number m to high, not implemented")	×
127	else:
NEW 128	o = self.omega_m_cukier75[m-1]	×
NEW 129	return np.append(o, self.freq_cukier(m, i+1, o))	×
130	else:
NEW 131	o = omega_before + self.d_m_cukier75[m-i]	×
NEW 132	if i == m:	×
NEW 133	return o	×
134	else:
NEW 135	return np.append(o, self.freq_cukier(m, i+1, o))	×
136
137
138	def freq_mcrae82(self, m, i = 1, omega_before = -1):	4✔
139
140	"""
141	This function creates an array of independant frequencies accroding to
142	McRae 1982 for usage in the eFAST method.
143
144	Parameters
145	----------
146	m: int
147	number of parameters (frequencies) needed
148	i: (intern) int
149	internally used recursion counter
150	omega_before: (intern) int
151	internally used previous frequency
152
153	Raises
154	----------
155	Exeption:
156	"Parameter number m to high, not implemented"
157	Execution is stopped, if the number of Parameters is to high (max. Number of Parameters for McRae = 15)
158
159	Returns
160	----------
161	np.array of float:
162	A 1d-Array of independant frequencies to the order of 4
163
164	Author
165	----------
166	Dominic Reusser
167	spotpy translation: Anna Herzog
168
169	"""
170
NEW 171	if i <= 1:	×
NEW 172	if m >= len(self.omega_m_mcrae82):	×
NEW 173	raise Exception("Parameter number m to high, not implemented")	×
174	else:
NEW 175	o = self.omega_m_mcrae82[m-1]	×
NEW 176	return np.append(o, self.freq_mcrae82(m, i+1, o))	×
177	else:
NEW 178	o = omega_before + self.d_m_mcrae82[m-i]	×
NEW 179	if i == m:	×
NEW 180	return o	×
181	else:
NEW 182	return np.append(o, self.freq_mcrae82(m, i+1, o))	×
183
184
185	def s(self, m, freq = 'cukier'):	4✔
186
187	"""
188	Function that generates a number of equally spaced values between -p1/2
189	and pi/2. The number is determined by the number of runs required of the
190	eFAST method for a number of parameters (min_runs_cukier or min_runs_mcrae)
191
192	Parameters
193	----------
194	m: int
195	number of parameters/ frequencies
196
197	freq: (optional) str
198	indicates weather to use the frequencies after 'cukier' or McRae 'mcrae'
199	Default is Cukier
200
201	Raises
202	----------
203	Exeption:
204	"Missing option for Frequency selection for Parameter definition, choose cukier or mcrae!"
205	Execution is stopped if an invalid method for the frequency selection is provided
206
207	Returns
208	----------
209	2D array of float:
210	an array of equally spaced values between -pi/2 and pi/2
211
212	Author
213	----------
214	Dominic Reusser
215	spotpy translation: Anna Herzog
216
217	"""
218
NEW 219	if freq == 'cukier':	×
NEW 220	min_runs = self.min_runs_cukier75	×
NEW 221	elif freq == 'mcrae':	×
NEW 222	min_runs = self.min_runs_mcrae82	×
223	else:
NEW 224	raise Exception("Missing option for Frequency selection for Parameter definition, choose cukier or mcrae!")	×
225
NEW 226	r = min_runs[m-1]	×
NEW 227	r_range = np.array(range(1,r+1))	×
NEW 228	s = np.pi/r * (2*r_range-r-1)/2	×
229
NEW 230	return s	×
231
232
233	def S(self, m, freq = 'cukier'):	4✔
234
235	"""
236	Function to generate an array of values, which provides the base for the parameterdistribution
237	in the FAST method. It is usally not used directly but called from the
238	fast_parameters function.
239
240	Parameters
241	----------
242	m: int
243	number of parameters/frequencies
244
245	freq: (optional) str
246	indicates weather to use the frequencies after 'cukier' or McRae 'mcrae'
247	Default is Cukier
248
249	Raises
250	----------
251	Exeption:
252	"Missing option for Frequency selection for Parameter definition, choose cukier or mcrae!"
253	Execution is stopped if an invalid method for the frequency selection is provided
254
255	Returns
256	----------
257	2D array of float:
258	an array with the shape (number of runs, parameters)
259
260	Author
261	----------
262	Dominic Reusser
263	spotpy translation: Anna Herzog
264	"""
265
NEW 266	if freq == 'cukier':	×
NEW 267	omega = self.freq_cukier(m)	×
NEW 268	elif freq == 'mcrae':	×
NEW 269	omega = self.freq_mcrae82(m)	×
270	else:
NEW 271	raise Exception("Missing option for Frequency selection for Parameter definition, choose cukier or mcrae!")	×
272
NEW 273	tab = np.outer(self.s(m, freq), omega)	×
274
NEW 275	toreturn = np.arcsin(np.sin(tab))/np.pi	×
276
277	# naming array dimensions is not possible with numpy but convention would be toreturn.shape () = (runs, parameternames)
278
NEW 279	return toreturn	×
280
281
282	def rerange(self, data, min_goal = 0, max_goal = 1, center = np.nan):	4✔
283
284	"""
285	This function performes a linear transformation of the data, such that
286	afterwards range(data) = (theMin, theMax)
287
288	Parameters
289	----------
290	data: array of flaot
291	an array with the data to transform
292	in this case the parameter distribution generated by function S
293	min_goal: float
294	the new minimum value (lower parameter bound)
295	max_goal: float
296	the new maximum value (upper parameter bound)
297	center: (optional) float
298	indicates which old value should become the new center
299	default: (max_goal+min_goal)/2
300
301	Returns
302	----------
303	2D array of float:
304	array with the transformed data
305
306	Author
307	----------
308	Dominic Reusser
309	spotpy translation: Anna Herzog
310
311	"""
312
NEW 313	min_data = min(data)	×
NEW 314	max_data = max(data)	×
315
NEW 316	if np.isnan(center):	×
NEW 317	max_data = max_data	×
NEW 318	dat = data - min_data	×
NEW 319	dat = dat/(max_data-min_data) * (max_goal-min_goal)	×
NEW 320	dat = dat + min_goal	×
NEW 321	return(dat)	×
322	else:
323	# split linear transformation, the data is split into two segments, one blow center, one above,
324	# each segment undergoes it's own linear transformation
NEW 325	below = data <= center	×
NEW 326	above = data > center	×
NEW 327	new_data = np.copy(data)	×
NEW 328	np.place(new_data, below, self.rerange(np.insert(data[below], 0, center), min_goal = min_goal, max_goal= max_goal/2)[1:])	×
NEW 329	np.place(new_data, above, self.rerange(np.insert(data[above], 0, center), min_goal = max_goal/2, max_goal= max_goal)[1:])	×
NEW 330	return(new_data)	×
331
332
333	def fast_parameters(self, minimum, maximum, freq = 'cukier', logscale = np.nan, names = np.nan):	4✔
334
335	"""
336	Function for the creation of a FAST Parameter set based on thier range
337
338	Parameters
339	----------
340	minimum: array of float
341	array containing the lower parameter boundries
342	maximum: array of float
343	array containing the upper parameter boundries
344	names: (optional) str
345	array containing the parameter names
346	freq: (optional) str
347	indicates weather to use the frequencies after 'cukier' or McRae 'mcrae'
348	Default is Cukier
349	logscale: (optional) bool
350	array containing bool values indicating weather a parameter is varied
351	on a logarithmic scale. In that case minimum and maximum are exponents
352
353	Raises
354	----------
355	Exeption:
356	"Expecting minimum and maximum of same size"
357	Execution is stopped if the number of minimum, maximum or logscale values differ
358
359	Returns
360	----------
361	value: array of float
362	an array with the shape (number of runs, parameters) containing the
363	required parameter sets
364
365	Author
366	----------
367	Dominic Reusser
368	spotpy translation: Anna Herzog
369
370	"""
371
NEW 372	if np.isnan(logscale):	×
NEW 373	logscale = np.full(minimum.shape, False)	×
NEW 374	if np.isnan(names):	×
NEW 375	names = ["P"+str(i) for i in range(minimum.shape[0])]	×
NEW 376	names = np.array(names)	×
377
NEW 378	n = len(minimum)	×
379
NEW 380	if (n != len(maximum)):	×
NEW 381	raise Exception("Expecting minimum and maximum of same size")	×
NEW 382	elif(n != len(names)):	×
NEW 383	raise Exception("Expecting minimum and names of same size")	×
NEW 384	elif(n != len(logscale)):	×
NEW 385	raise Exception("Expecting minimum and logscale of same size")	×
386
NEW 387	toreturn = self.S(m=n, freq = freq) #par_names = names,	×
388
NEW 389	for i in range(0,n):	×
NEW 390	toreturn[:,i] = self.rerange(toreturn[:,i], minimum[i], maximum[i])	×
NEW 391	if logscale[i]:	×
NEW 392	toreturn[:,i] = 10**toreturn[:,i]	×
393
NEW 394	return toreturn	×
395
396
397	def sample(self, repetitions, freq = 'cukier', logscale = np.nan):	4✔
398	"""
399	Samples from the eFAST algorithm.
400
401	Input
402	----------
403	repetitions: int
404	Maximum number of runs.
405	freq: (optional) str
406	indicates weather to use the frequencies after 'cukier' or McRae 'mcrae'
407	Default is Cukier
408	logscale: (optional) bool
409	array containing bool values indicating weather a parameter is varied
410	on a logarithmic scale. In that case minimum and maximum are exponents
411
412	Raises
413	----------
414	Warning:
415	"specified number of repetitions is smaller than minimum required for eFAST analysis!
416	Simultions will be perfomed but eFAST analysis might not be possible"
417
418	If the specified number of repetitions is smaller than required by the algorithm, the
419	algortihm will still execute, but the usage of spotpy.analyser.efast.calc_sensitivity()
420	might not be possible. Rather define a large number of reps as only required minimum runs
421	will be performed anyway.
422
423	Warning:
424	"Number of specified repetitions equals or exeeds number of minimum required repetitions for eFAST analysis
425	programm will stop after required runs."
426
427	Returns
428	----------
429	database:
430	spotpy data base in chosen format with objective function values, parameter values and simulation results
431
432	Author
433	----------
434	Tobias Houska, Anna Herzog
435	"""
436
NEW 437	print("generating eFAST Parameters")	×
438	# Get the names of the parameters to analyse
NEW 439	names = self.parameter()["name"]	×
440	# Get the minimum and maximum value for each parameter from the
441	# distribution
NEW 442	parmin, parmax = self.parameter()["minbound"], self.parameter()["maxbound"]	×
NEW 443	self.numberf = len(parmin)	×
NEW 444	min_runs = self.min_runs_cukier75[self.numberf-1]	×
445
NEW 446	if min_runs > repetitions:	×
NEW 447	warnings.warn("specified number of repetitions is smaller than minimum required for eFAST analysis!\n"+	×
448	"Simultions will be perfomed but eFAST analysis might not be possible")
449	else:
NEW 450	repetitions = min_runs	×
NEW 451	print("Number of specified repetitions equals or exeeds number of minimum required repetitions for eFAST analysis\n"+	×
452	"programm will stop after required runs.")
453
NEW 454	self.set_repetiton(repetitions)	×
455
456	# Generate Matrix with eFAST parameter sets
NEW 457	N = self.fast_parameters(parmin, parmax, freq, logscale)	×
458
NEW 459	print("Starting the eFast algorithm with {} repetitions...".format(repetitions))	×
460
461	# A generator that produces parametersets if called
NEW 462	param_generator = (	×
463	(rep, N[rep,:]) for rep in range(int(repetitions))
464	)
NEW 465	for rep, randompar, simulations in self.repeat(param_generator):	×
466	# A function that calculates the fitness of the run and the manages the database
NEW 467	self.postprocessing(rep, randompar, simulations)	×
468
NEW 469	if self.breakpoint == "write" or self.breakpoint == "readandwrite":	×
NEW 470	if rep >= lastbackup + self.backup_every_rep:	×
NEW 471	work = (rep, N[rep,:])	×
NEW 472	self.write_breakdata(self.dbname, work)	×
NEW 473	lastbackup = rep	×
NEW 474	self.final_call()	×
475
476	def calc_sensitivity(self, results, dbname, freq = 'cukier'):	4✔
477	"""
478	Function to calcultae the sensitivity for a data series (eg. a time series)
479	based on the eFAST algorithm.
480
481	Parameters
482	----------
483	results: np.array of void
484	spopty restults array as loaded with spotpy.analyser.load_csv_results()
485	dbname: (optional) str
486	name of file to store sensitivity values
487	freq: (optional) str
488	indicates weather to use the frequencies after Cukier 'cukier' or McRae 'mcrae'
489	Default is Cukier
490
491	Raises
492	----------
493	Exeption:
494	"Missing option for Frequency selection for Parameter definition, choose cukier or mcrae!"
495	Execution is stopped if an invalid method for the frequency selection is provided
496
497	Returns
498	----------
499	database:
500	database containing the temporal parameter sensitivities with seperate column
501	for each parameter and row for eg. time step
502
503	Author
504	----------
505	Dominic Reusser
506	spotpy translation: Anna Herzog
507
508	"""
NEW 509	numberf = len(self.parameter()["minbound"])	×
510
511	# sens_data = np.full((len(results[0]), numberf), np.nan)
512
513	# idendtify frequency
NEW 514	if freq == 'cukier':	×
NEW 515	t_runs = self.min_runs_cukier75[numberf-1]	×
NEW 516	t_freq = self.freq_cukier(numberf)	×
NEW 517	elif freq == 'mcrae':	×
NEW 518	t_runs = self.min_runs_mcrae82[numberf-1]	×
NEW 519	t_freq = self.freq_mcrae82(numberf)	×
520	else:
NEW 521	raise Exception("Missing option for Frequency selection for Parameter definition, choose cukier or mcrae!")	×
522
523	# Get the names of the parameters to analyse
NEW 524	names = ["par" + name for name in self.parameter()["name"]]	×
525
526	# open the file for sensitivity results
NEW 527	f = open(dbname+".csv", 'w+')	×
NEW 528	np.savetxt(f, [names], "%s", delimiter=",")	×
529
NEW 530	data = np.array(results)	×
531
NEW 532	try:	×
533	# convert array of void to array of float
NEW 534	mod_results = np.full((data.shape[0], len(data[0])), np.nan)	×
535
NEW 536	for index in range(len(data)):	×
NEW 537	mod_results[index, :] = list(data[index])	×
538
NEW 539	print("data series")	×
NEW 540	for index in range(mod_results.shape[1]):	×
NEW 541	x_data = mod_results[:, index]	×
NEW 542	sens_data = efast_sensitivity(x_data, numberf, t_runs, t_freq)	×
NEW 543	np.savetxt(f, [sens_data], delimiter=",", fmt='%1.5f')	×
544
NEW 545	except:	×
NEW 546	print("single data")	×
NEW 547	x_data = data	×
NEW 548	sens_data = efast_sensitivity(x_data, numberf, t_runs, t_freq)	×
NEW 549	np.savetxt(f, [sens_data], delimiter=",", fmt='%1.5f')	×
550
NEW 551	f.close()	×

thouska / spotpy / 13287027981

Source File Press 'n' to go to next uncovered line, 'b' for previous

Source File
Press 'n' to go to next uncovered line, 'b' for previous