PrincetonUniversity / PsyNeuLink / 11508734597

# Princeton University licenses this file to You under the Apache License, Version 2.0 (the "License");

# you may not use this file except in compliance with the License.  You may obtain a copy of the License at:

#     http://www.apache.org/licenses/LICENSE-2.0

# Unless required by applicable law or agreed to in writing, software distributed under the License is distributed

# on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.

# See the License for the specific language governing permissions and limitations under the License.

# ********************************************* EMComposition *************************************************

# TODO:

# - QUESTION:

#   - SHOULD differential of SoftmaxGainControl Node be included in learning?

#   - SHOULD MEMORY DECAY OCCUR IF STORAGE DOES NOT? CURRENTLY IT DOES NOT (SEE EMStorage Function)

# - FIX: NAMING

# - FIX: Concatenation:

# -      LLVM for function and derivative

# -      Add Concatenate to pytorchcreator_function

# -      Deal with matrix assignment in LearningProjection LINE 643

# -      Reinstate test for execution of Concatenate with learning in test_emcomposition (currently commented out)

# - FIX: Softmax Gain Control:

#        Test if it current works (they are added to Composition but not in BackProp processing pathway)

#        Does backprop have to run through this if not learnable?

#        If so, need to add PNL Function, with derivative and LLVM and Pytorch implementations

# - FIX: WRITE MORE TESTS FOR EXECUTION, WARNINGS, AND ERROR MESSAGES

#         - learning (with and without learning field weights

#         - 3d tuple with first entry != memory_capacity if specified

#         - list with number of entries > memory_capacity if specified

#         - input is added to the correct row of the matrix for each key and value for

#                for non-contiguous keys (e.g, field_weights = [1,0,1]))

#         - explicitly that storage occurs after retrieval

# - FIX: WARNING NOT OCCURRING FOR Normalize ON ZEROS WITH MULTIPLE ENTRIES (HAPPENS IF *ANY* KEY IS EVER ALL ZEROS)

# - FIX: IMPLEMENT LearningMechanism FOR RETRIEVAL WEIGHTS:

#        - what is learning_update: AFTER doing?  Use for scheduling execution of storage_node?

#        ?? implement derivative for concatenate

# - FIX: implement add_storage_pathway to handle addition of storage_node as learning mechanism

#        - in "_create_storage_learning_components()" assign "learning_update" arg

#          as BEORE OR DURING instead of AFTER (assigned to learning_enabled arg of LearningMechanism)

# - FIX: Add StorageMechanism LearningProjections to Composition? -> CAUSES TEST FAILURES; NEEDS INVESTIGATION

# - FIX: Thresholded version of SoftMax gain (per Kamesh)

# - FIX: DEAL WITH INDEXING IN NAMES FOR NON-CONTIGUOUS KEYS AND VALUES (reorder to keep all keys together?)

# - FIX: _import_composition:

#        - MOVE LearningProjections

#        - MOVE Condition? (e.g., AllHaveRun) (OR PUT ON MECHANISM?)

# - FIX: IMPLEMENT _integrate_into_composition METHOD THAT CALLS _import_composition ON ANOTHER COMPOSITION

# -      AND TRANSFERS RELEVANT ATTRIBUTES (SUCH AS MEMORY, query_input_nodeS, ETC., POSSIBLY APPENDING NAMES)

# - FIX: ADD Option to suppress field_weights when computing norm for weakest entry in EMStorageMechanism

# - FIX: GENERATE ANIMATION w/ STORAGE (uses Learning but not in usual way)

# - IMPLEMENT use OF multiple inheritance of EMComposition from AutoDiff and Composition

# - FIX: DOCUMENTATION:

#        - enable_learning vs. learning_field_weights

#        - USE OF EMStore.storage_location (NONE => LOCAL, SPECIFIED => GLOBAL)

#        - define "keys" and "values" explicitly

#        - define "key weights" explicitly as field_weights for all non-zero values

#        - make it clear that full size of memory is initialized (rather than "filling up" w/ use)

#        - write examples for run()

# - FIX: ADD NOISE

# - FIX: ?ADD add_memory() METHOD FOR STORING W/O RETRIEVAL, OR JUST ADD retrieval_prob AS modulable Parameter

# - FIX: CONFIDENCE COMPUTATION (USING SIGMOID ON DOT PRODUCTS) AND REPORT THAT (EVEN ON FIRST CALL)

# - FIX: ALLOW SOFTMAX SPEC TO BE A DICT WITH PARAMETERS FOR _get_softmax_gain() FUNCTION

# MISC:

# - WRITE TESTS FOR INPUT_PORT and MATRIX SPECS CORRECT IN LATEST BRANCHs

# - ACCESSIBILITY OF DISTANCES (SEE BELOW): MAKE IT A LOGGABLE PARAMETER (I.E., WITH APPROPRIATE SETTER)

#   ADD COMPILED VERSION OF NORMED LINEAR_COMBINATION FUNCTION TO LinearCombination FUNCTION: dot / (norm a * norm b)

# - DECAY WEIGHTS BY:

#   ? 1-SOFTMAX / N (WHERE N = NUMBER OF ITEMS IN MEMORY)

#   or

#      1/N (where N=number of items in memory, and thus gets smaller as N gets

#      larger) on each storage (to some asymptotic minimum value), and store the new memory to the unit with the

#      smallest weights (randomly selected among “ties" [i.e., within epsilon of each other]), I think we have a

#      mechanism that can adaptively use its limited capacity as sensibly as possible, by re-cycling the units

#      that have the least used memories.

# - MAKE "_store_memory" METHOD USE LEARNING INSTEAD OF ASSIGNMENT

#   - make LearningMechanism that, instead of error, simply adds relevant input to weights (with all others = 0)

#   - (relationship to Steven's Hebbian / DPP model?):

# - ADD ADDITIONAL PARAMETERS FROM CONTENTADDRESSABLEMEMORY FUNCTION

# - ADAPTIVE TEMPERATURE: KAMESH FOR FORMULA

# - ADD MEMORY_DECAY TO ContentAddressableMemory FUNCTION (and compiled version by Samyak)

# - MAKE memory_template A CONSTRUCTOR ARGUMENT FOR default_variable

# - FIX: PSYNEULINK:

#      - TESTS:

#        - WRITE TESTS FOR DriftOnASphere variable = scalar, 2d vector or 1d vector of correct and incorrect lengths

#        - WRITE TESTS FOR LEARNING WITH LinearCombination of 1, 2 and 3 inputs

#

#      - COMPILATION:

#        - Remove CIM projections on import to another composition

#        - Autodiff support for IdentityFunction

#        - LinearMatrix to add normalization

#        - _store() method to assign weights to memory

#        - LLVM problem with ComparatorMechanism

#

#      - pytorchcreator_function:

#           SoftMax implementation:  torch.nn.Softmax(dim=0) is not getting passed correctly

#           Implement LinearCombination

#        - LinearMatrix Function:

#

#      - LEARNING - Backpropagation LearningFunction / LearningMechanism

#        - DOCUMENTATION:

#           - weight_change_matrix = gradient (result of delta rule) * learning_rate

#           - ERROR_SIGNAL is OPTIONAL (only implemented when there is an error_source specified)

#        - Backprop: (related to above?) handle call to constructor with default_variable = None

#        - WRITE TESTS FOR USE OF COVARIATES AND RELATED VIOLATIONS: (see ScratchPad)

#          - Use of LinearCombination with PRODUCT in output_source

#          - Use of LinearCombination with PRODUCT in InputPort of output_source

#                  - Construction of LearningMechanism with Backprop:

#        - MappingProjection / LearningMechanism:

#          - Add learning_rate parameter to MappingProjection (if learnable is True)

#          - Refactor LearningMechanism to use MappingProjection learning_rate specification if present

#        - CHECK FOR EXISTING LM ASSERT IN pytests

#

#      - AutodiffComposition:

#         - replace handling / flattening of nested compositions with Pytorch.add_module (which adds "child" modules)

#         - Check that error occurs for adding a controller to an AutodiffComposition

#         - Check that if "epochs" is not in input_dict for Autodiff, then:

#           - set to num_trials as default,

#           - leave it to override num_trials if specified (add this to DOCUMENTATION)

#        - Input construction has to be:

#           - same for Autodiff in Python mode and PyTorch mode

#               (NOTE: used to be that autodiff could get left in Python mode

#                      so only where tests for Autodiff happened did it branch)

#           - AND different from Composition (in Python mode)

#        - support use of pathway argument in Autodff

#        - the following format doesn't work for LLVM (see test_identicalness_of_input_types:

#           xor = pnl.AutodiffComposition(nodes=[input_layer,hidden_layer,output_layer])

#           xor.add_projections([input_to_hidden_wts, hidden_to_output_wts])

#          - DOCUMENTATION: execution_mode=ExecutionMode.Python allowed

#          - Add warning of this on initial call to learn()

#

#      - Composition:

#        - Add default_execution_mode attribute to allow nested Compositions to be executed in

#              different model than outer Composition

#        - _validate_input_shapes_and_expand_for_all_trials: consolidate with get_input_format()

#        - Generalize treatment of FEEDBACK specification:

      #        - FIX: ADD TESTS FOR FEEDBACK TUPLE SPECIFICATION OF Projection, DIRECT SPECIFICATION IN CONSTRUCTOR

#              - FIX: why aren't FEEDBACK_SENDER and FEEDBACK_RECEIVER roles being assigned when feedback is specified?

#        - add property that keeps track of warnings that have been issued, and suppresses repeats if specified

#        - add property of Composition that lists it cycles

#        - Add warning if termination_condition is trigged (and verbosePref is set)

#        - Addition of projections to a ControlMechanism seems too dependent on the order in which the

#              the ControlMechanism is constructed with respect to its afferents (if it comes before one,

#              the projection to it (i.e., for monitoring) does not get added to the Composition

# -      - IMPLEMENTATION OF LEARNING: NEED ERROR IF TRY TO CALL LEARN ON A COMPOSITION THAT HAS NO LEARNING MECHANISMS

#          INCLUDING IN PYTHON MODE??  OR JUST ALLOW IT TO CONSTRUCT THE PATHWAY AUTOMATICALLY?

#        - Change size argument in constructor to use standard numpy shape format if tupe, and PNL format if list

#        - Write convenience Function for returning current time from context

#             - requires it be called from execution within aComposition, error otherwise)

#             - takes argument for time scale (e.g., TimeScale.TRIAL, TimeScale.RUN, etc.)

#             - Add TimeMechanism for which this is the function, and can be configured to report at a timescale

#        - Add Composition.run_status attribute assigned a context flag, with is_preparing property that checks it

#               (paralleling handling of is_initializing)

#        - Allow set of lists as specification for pathways in Composition

#        - Add support for set notation in add_backpropagation_learning_pathway (to match add_linear_processing_pathway)

#             see ScratchPad: COMPOSITION 2 INPUTS UNNESTED VERSION: MANY-TO-MANY

#        - Make sure that shadow inputs (see InputPort_Shadow_Inputs) uses the same matrix as shadowed input.

#        - composition.add_backpropagation_learning_pathway(): support use of set notation for multiple nodes that

#        project to a single one.

#        - add LearningProjections executed in EXECUTION_PHASE to self.projections

#          and then remove MODIFIED 8/1/23 in _check_for_unused_projections

#        - Why can't verbosePref be set directly on a composition?

#        - Composition.add_nodes():

#           - should check, on each call to add_node, to see if one that has a releavantprojection and, if so, add it.

#           - Allow [None] as argument and treat as []

#        - IF InputPort HAS default_input = DEFAULT_VARIABLE,

#           THEN IT SHOULD BE IGNORED AS AN INPUT NODE IN A COMPOSITION

#        - Add use of dict in pathways specification to map outputs from a set to inputs of another set

#            (including nested comps)

#

#      - ShowGraph:  (show_graph)

#        - don't show INPUT/OUTPUT Nodes for nested Comps in green/red

#                (as they don't really receive input or generate output on a run

#        - show feedback projections as pink (shouldn't that already be the case?)

#        - add mode for showing projections as diamonds without show_learning (e.g., "show_projections")

#        - figure out how to get storage_node to show without all other learning stuff

#        - show 'operation' parameter for LinearCombination in show_node_structure=ALL

#        - specify set of nodes to show and only show those

#        - fix: show_learning=ALL (or merge from EM branch)

#

#      - ControlMechanism

#        - refactor ControlMechanism per notes of 11/3/21, including:

#                FIX: 11/3/21 - MOVE _parse_monitor_specs TO HERE FROM ObjectiveMechanism

#      - EpisodicMemoryMechanism:

#        - make storage_prob and retrieval_prob parameters linked to function

#        - make distance_field_weights a parameter linked to function

#

#      - LinearCombination Function:

#        - finish adding derivative (for if exponents are specified)

#        - remove properties (use getter and setter for Parameters)

#

#      - ContentAddressableMemory Function:

#           - rename "cue" -> "query"

#           - add field_weights as parameter of EM, and make it a shared_parameter ?as well as a function_parameter?

#     - DDM:

#        - make reset_stateful_function_when a Parameter and arg in constructor

#          and align with reset Parameter of IntegratorMechanism)

#

#    - FIX: BUGS:

#      - composition:

#          - If any MappingProjection is specified from nested node to outer node,

#            then direct projections are instantiated to the output_CIM of the outer comp, and the

#            nested comp is treated as OUTPUT Node of outer comp even if all its projections are to nodes in outer comp

#            LOOK IN add_projections? for nested comps

#      - composition (?add_backpropagation_learning_pathway?):

#           THIS FAILS:

#             comp = Composition(name='a_outer')

#             comp.add_backpropagation_learning_pathway([input_1, hidden_1, output_1])

#             comp.add_backpropagation_learning_pathway([input_1, hidden_1, output_2])

#           BUT THE FOLLOWING WORKS (WITH IDENTICAL show_graph(show_learning=True)):

#             comp = Composition(name='a_outer')

#             comp.add_backpropagation_learning_pathway([input_1, hidden_1, output_1])

#             comp.add_backpropagation_learning_pathway([hidden_1, output_2])

#      - show_graph(): QUIRK (BUT NOT BUG?):

#           SHOWS TWO PROJECTIONS FROM a_inner.input_CIM -> hidden_x:

#            ?? BECAUSE hidden_x HAS TWO input_ports SINCE ITS FUNCTION IS LinearCombination?

#             a_inner = AutodiffComposition([hidden_x],name='a_inner')

#             a_outer = AutodiffComposition([[input_1, a_inner, output_1],

#                                            [a_inner, output_2]],

#             a_outer.show_graph(show_cim=True)

#      -LearningMechanism / Backpropagation LearningFunction:

#         - Construction of LearningMechanism on its own fails; e.g.:

#             lm = LearningMechanism(learning_rate=.01, learning_function=BackPropagation())

#             causes the following error:

#                TypeError("Logistic.derivative() missing 1 required positional argument: 'self'")

#      - Adding GatingMechanism after Mechanisms they gate fails to implement gating projections

#           (example:  reverse order of the following in _construct_pathways

#                      self.add_nodes(self.softmax_nodes)

#                      self.add_nodes(self.field_weight_nodes)

#           - add Normalize as option

#           - Anytime a row's norm is 0, replace with 1s

#      - WHY IS Concatenate NOT WORKING AS FUNCTION OF AN INPUTPORT (WASN'T THAT USED IN CONTEXT OF BUFFER?

#           SEE NOTES TO KATHERINE

#

#     - TESTS

#       For duplicate Projections (e.g., assign a Mechanism in **monitor** of ControlMechanism

#            and use comp.add_projection(MappingProjection(mointored, control_mech) -> should generate a duplicate

#            then search for other instances of the same error message

"""

Contents

--------

  * `EMComposition_Overview`

     - `Organization <EMComposition_Organization>`

     - `Operation <EMComposition_Operation>`

  * `EMComposition_Creation`

     - `Fields <EMComposition_Fields>`

     - `Capacity <EMComposition_Memory_Capacity>`

     - `Storage and Retrieval <EMComposition_Retrieval_Storage>`

     - `Learning <EMComposition_Learning>`

  * `EMComposition_Structure`

     - `Input <EMComposition_Input>`

     - `Memory <EMComposition_Memory>`

     - `Output <EMComposition_Output>`

  * `EMComposition_Execution`

     - `Processing <EMComposition_Processing>`

     - `Learning <EMComposition_Learning>`

  * `EMComposition_Examples`

     - `Memory Template and Fill <EMComposition_Example_Memory_Template>`

     - `Field Weights <EMComposition_Example_Field_Weights>`

  * `EMComposition_Class_Reference`

.. _EMComposition_Overview:

Overview

--------

The EMComposition implements a configurable, content-addressable form of episodic, or eternal memory, that emulates

an `EpisodicMemoryMechanism` -- reproducing all of the functionality of its `ContentAddressableMemory` `Function` --

in the form of an `AutodiffComposition` that is capable of learning how to differentially weight different cues used

for retrieval,, and that adds the capability for `memory_decay <EMComposition.memory_decay_rate>`. Its `memory

<EMComposition.memory>` is configured using two arguments of its constructor: **memory_template** argument, that defines

how each entry in `memory <EMComposition.memory>` is structured (the number of fields in each entry and the length

of each field); and **field_weights** argument, that defines which fields are used as cues for retrieval, i.e., "keys",

including whether and how they are differentially weighted in the match process used for retrieval); and which

fields are treated as "values" that are stored retrieved, but not used by the match process. The inputs to an

EMComposition, corresponding to each key ("query") and value field are assigned to each of its `INPUT <NodeRole.INPUT>`

`Nodes <Composition_Nodes>` (listed in its `query_input_nodes <EMComposition.query_input_nodes>` and `value_input_nodes

<EMComposition.value_input_nodes>` attributes, respectively), and the retrieved values are represented as `OUTPUT

<NodeRole.OUTPUT>` `Nodes <Composition_Nodes>` of the EMComposition.  The `memory <EMComposition.memory>` can be

accessed using its `memory <EMComposition.memory>` attribute.

    .. technical_note::

       The memories of an EMComposition are actually stored in the `matrix <EMComposition_Memory>` attribute of a

       set of `MappingProjections <MappingProjection>` (see `note below <EMComposition_Memory_Storage>`).  The `memory

       <EMComposition.memory>` attribute compiles and formats these as a single 3d array, the rows of which (axis 0)

       are each entry, the columns of which (axis 1) are the fields of each entry, and the items of which (axis 2)

       are the values of each field (see `EMComposition_Memory` for additional details).

.. _EMComposition_Organization:

**Organization**

.. _EMComposition_Entries_and_Fields:

*Entries and Fields*. Each entry in memory can have an arbitrary number of fields, and each field can have an arbitrary

length.  However, all entries must have the same number of fields, and the corresponding fields must all have the same

length across entries. Each field is treated as a separate "channel" for storage and retrieval, and is associated with

its own corresponding input (key or value) and output (retrieved value) `Node <Composition_Nodes>` some or all of

which can be used to compute the similarity of the input (key) to entries in memory, that is used for retreieval.

Fields can be differentially weighted to determine the influence they have on retrieval, using the `field_weights

<ContentAddressableMemory.memory>` parameter (see `retrieval <EMComposition_Retrieval_Storage>` below). The number and

shape of the fields in each entry is specified in the **memory_template** argument of the EMComposition's constructor

(see `memory_template <EMComposition_Fields>`). Which fields treated as keys (i.e., matched against queries during

retrieval) and which are treated as values (i.e., retrieved but not used for matching retrieval) is specified in the

**field_weights** argument of the EMComposition's constructor (see `field_weights <EMComposition_Field_Weights>`).

.. _EMComposition_Operation:

**Operation**

*Retrieval.*  The values retrieved from `memory <ContentAddressableMemory.memory>` (one for each field) are based

on the relative similarity of the keys to the entries in memory, computed as the dot product of each key and the

values in the corresponding field for each entry in memory.  These dot products are then softmaxed, and those

softmax distributions are weighted by the corresponding `field_weights <EMComposition.field_weights>` for each field

and then combined, to produce a single softmax distribution over the entries in memory. That is then used to generate

a weighted average of the retrieved values across all fields, which is returned as the `result <Composition.result>`

of the EMComposition's `execution <Composition_Execution>` (an EMComposition can also be configured to return the

entry with the highest dot product weighted by field, however then it is not compatible with learning;

see `softmax_choice <EMComposition_Softmax_Choice>`).

  COMMENT:

  TBD DISTANCE ATTRIBUTES:

  The distances used for the last retrieval is stored in XXXX and the distances of each of their corresponding fields

  (weighted by `distance_field_weights <ContentAddressableMemory.distance_field_weights>`), are returned in XXX,

  respectively.

  COMMENT

*Storage.*  The `inputs <Composition_Input_External_InputPorts>` to the EMComposition's fields are stored in `memory

<EMComposition.memory>` after each execution, with a probability determined by `storage_prob

<EMComposition.storage_prob>`.  If `memory_decay_rate <EMComposition.memory_decay_rate>` is specified, then the `memory

<EMComposition.memory>` is decayed by that amount after each execution.  If `memory_capacity

<EMComposition.memory_capacity>` has been reached, then each new memory replaces the weakest entry (i.e., the one

with the smallest norm across all of its fields) in `memory <EMComposition.memory>`.

.. _EMComposition_Creation:

Creation

--------

An EMComposition is created by calling its constructor, that takes the following arguments:

  .. _EMComposition_Fields:

*Field Specification*

.. _EMComposition_Memory_Template:

* **memory_template**: This specifies the shape of the entries to be stored in the EMComposition's `memory

  <EMComposition.memory>`, and can be used to initialize `memory <EMComposition.memory>` with pre-specified entries.

  The **memory_template** argument can be specified in one of three ways (see `EMComposition_Examples` for

  representative use cases):

  * **tuple**: interpreted as an np.array shape specification, that must be of length 2 or 3.  If it is a 3-item tuple,

    then the first item specifies the number of entries in memory, the 2nd the number of fields in each entry, and the

    3rd the length of each field.  If it is a 2-item tuple, this specifies the shape of an entry, and the number of

    entries is specified by `memory_capacity <EMComposition_Memory_Capacity>`).  All entries are

    filled with zeros or the value specified by `memory_fill <EMComposition_Memory_Fill>`.

    .. warning::

       If **memory_template** is specified with a 3-item tuple and `memory_capacity <EMComposition_Memory_Capacity>`

       is also specified with a value that does not match the first item of **memory_template**, and error is

       generated indicating the conflict in the number of entries specified.

    .. hint::

       To specify a single field, a list or array must be used (see below), as a 2-item tuple is interpreted as

       specifying the shape of an entry, and so it can't be used to specify the number of entries each of which

       has a single field.

  * **2d list or array**: interpreted as a template for memory entries. This can be used to specify fields of

    different lengths (i.e., entries that are ragged arrays), with each item in the list (axis 0 of the array) used

    to specify the length of the corresponding field.  The template is then used to initialze all entries in `memory

    <EMComposition.memory>`.  If the template includes any non-zero elements, then the array is replicated for all

    entries in `memory <EMComposition.memory>`; otherwise, they are filled with either zeros or the value specified

    in `memory_fill <EMComposition_Memory_Fill>`.

    .. hint::

       To specify a single entry, with all other entries filled with zeros

       or the value specified in **memory_fill**, use a 3d array as described below.

  * **3d list or array**: used to initialize `memory <EMComposition.memory>` directly with the entries specified in

    the outer dimension (axis 0) of the list or array.  If `memory_capacity <EMComposition_Memory_Capacity>` is not

    specified, then it is set to the number of entries in the list or array. If **memory_capacity** *is* specified,

    then the number of entries specified in **memory_template** must be less than or equal to **memory_capacity**.  If

    is less than **memory_capacity**, then the remaining entries in `memory <EMComposition.memory>` are filled with

    zeros or the value specified in **memory_fill** (see below):  if all of the entries specified contain only

    zeros, and **memory_fill** is specified, then the matrix is filled with the value specified in **memory_fill**;

    otherwise, zeros are used to fill all entries.

.. _EMComposition_Memory_Capacity:

*Memory Capacity*

* **memory_capacity**: specifies the number of items that can be stored in the EMComposition's memory; when

  `memory_capacity <EMComposition.memory_capacity>` is reached, each new entry overwrites the weakest entry (i.e., the

  one with the smallest norm across all of its fields) in `memory <EMComposition.memory>`.  If `memory_template

  <EMComposition_Memory_Template>` is specified as a 3-item tuple or 3d list or array (see above), then that is used

  to determine `memory_capacity <EMComposition.memory_capacity>` (if it is specified and conflicts with either of those

  an error is generated).  Otherwise, it can be specified using a numerical value, with a default of 1000.  The

  `memory_capacity <EMComposition.memory_capacity>` cannot be modified once the EMComposition has been constructed.

.. _EMComposition_Memory_Fill:

* **memory_fill**: specifies the value used to fill the `memory <EMComposition.memory>`, based on the shape specified

  in the **memory_template** (see above).  The value can be a scalar, or a tuple to specify an interval over which

  to draw random values to fill `memory <EMComposition.memory>` --- both should be scalars, with the first specifying

  the lower bound and the second the upper bound.  If **memory_fill** is not specified, and no entries are specified

  in **memory_template**, then `memory <EMComposition.memory>` is filled with zeros.

  .. hint::

     If memory is initialized with all zeros and **normalize_memories** set to ``True`` (see `below

     <EMComposition_Retrieval_Storage>`) then a numpy.linalg warning is issued about divide by zero.

     This can be ignored, as it does not affect the results of execution, but it can be averted by specifying

     `memory_fill <EMComposition_Memory_Fill>` to use small random values (e.g., ``memory_fill=(0,.001)``).

.. _EMComposition_Field_Weights:

* **field_weights**: specifies which fields are used as keys, and how they are weighted during retrieval. The

  number of entries specified must match the number of fields specified in **memory_template** (i.e., the size of

  of its first dimension (axis 0)). All non-zero entries must be positive; these designate *keys* -- fields

  that are used to match queries against entries in memory for retrieval (see `Match memories by field

  <EMComposition_Processing>`). Entries of 0 designate *values* -- fields that are ignored during the matching

  process, but the values of which are retrieved and assigned as the `value <Mechanism_Base.value>` of the

  corresponding `retrieved_node <EMComposition.retrieved_nodes>`. This distinction between keys and value corresponds

  to the format of a standard "dictionary," though in that case only a single key and value are allowed, whereas

  here there can be one or more keys and any number of values;  if all fields are keys, this implements a full form of

  content-addressable memory. If **learn_field_weight** is True (and `enable_learning<EMComposition.enable_learning>`

  is either True or a list with True for at least one entry), then the field_weights can be modified during training

  (this functions similarly to the attention head of a Transformer model, although at present the field can only be

  scalar values rather than vecdtors); if **learn_field_weight** is False, then the field_weights are fixed.

  The following options can be used to specify **field_weights**:

    * *None* (the default): all fields except the last are treated as keys, and are weighted equally for retrieval,

      while the last field is treated as a value field;

    * *single entry*: all fields are treated as keys (i.e., used for retrieval) and weighted equally for retrieval.

      if `normalize_field_weights <EMComposition_Normalize_Field_Weights>` is True, the value is ignored and all

      of keys are weighted by 1 / number of keys (i.e., normalized), whereas if `normalize_field_weights

      <EMComposition_Normalize_Field_Weights>` is False, then the value specified is used to weight the retrieval of

      every keys.

    * *multiple non-zero entries*: If all entries are identical, the value is ignored and the corresponding keys

      are weighted equally for retrieval; if the non-zero entries are non-identical, they are used to weight the

      corresponding fields during retrieval (see `Weight fields <EMComposition_Processing>`).  In either case,

      the remaining fields (with zero weights) are treated as value fields.

.. _EMComposition_Normalize_Field_Weights:

* **normalize_field_weights**: specifies whether the `field_weights <EMComposition.field_weights>` are normalized

    or their raw values are used.  If True, the `field_weights <EMComposition.field_weights>` are normalized so that

    they sum to 1.0, and are used to weight the corresponding fields during retrieval (see `Weight fields

    <EMComposition_Processing>`). If False, the raw values of the `field_weights <EMComposition.field_weights>` are

    used to weight (i.e., multiply) the retrieved value of each field.  This setting is ignored if **field_weights**

    is None or `concatenate_keys <EMComposition_Concatenate_Keys>` is in effect.

.. _EMComposition_Field_Names:

* **field_names**: specifies names that can be assigned to the fields.  The number of names specified must

  match the number of fields specified in the memory_template.  If specified, the names are used to label the

  nodes of the EMComposition.  If not specified, the fields are labeled generically as "Key 0", "Key 1", etc..

.. _EMComposition_Concatenate_Keys:

* **concatenate_keys**:  specifies whether keys are concatenated before a match is made to items in memory.

  This is False by default. It is also ignored if the `field_weights <EMComposition.field_weights>` for all keys are

  not all equal (i.e., all non-zero weights are not equal -- see `field_weights <EMComposition_Field_Weights>`) and/or

  `normalize_memories <EMComposition.normalize_memories>` is set to False. Setting concatenate_keys to True in either

  of those cases issues a warning, and the setting is ignored. If the key `field_weights <EMComposition.field_weights>`

  (i.e., all non-zero values) are all equal *and* **normalize_memories** is set to True, then setting

  **concatenate_keys** causes a `concatenate_keys_node <EMComposition.concatenate_keys_node>` to be created that

  receives input from all of the `query_input_nodes <EMComposition.query_input_nodes>` and passes them as a single

  vector to the `mactch_node <EMComposition.match_nodes>`.

      .. note::

         While this is computationally more efficient, it can affect the outcome of the `matching process

         <EMComposition_Processing>`, since computing the normalized dot product of a single vector comprised of the

         concatentated inputs is not identical to computing the normalized dot product of each field independently and

         then combining the results.

      .. note::

         All `query_input_nodes <EMComposition.query_input_nodes>` and `retrieved_nodes <EMComposition.retrieved_nodes>`

         are always preserved, even when `concatenate_keys <EMComposition.concatenate_keys>` is True, so that separate

         inputs can be provided for each key, and the value of each key can be retrieved separately.

.. _EMComposition_Memory_Decay_Rate

* **memory_decay_rate**: specifies the rate at which items in the EMComposition's memory decay;  the default rate

  is *AUTO*, which sets it to  1 / `memory_capacity <EMComposition.memory_capacity>`, such that the oldest memories

  are the most likely to be replaced when `memory_capacity <EMComposition.memory_capacity>` is reached.  If

  **memory_decay_rate** is set to 0 None or False, then memories do not decay and, when `memory_capacity

  <EMComposition.memory_capacity>` is reached, the weakest memories are replaced, irrespective of order of entry.

.. _EMComposition_Retrieval_Storage:

*Retrieval and Storage*

* **storage_prob** : specifies the probability that the inputs to the EMComposition will be stored as an item in

  `memory <EMComposition.memory>` on each execution.

* **normalize_memories** : specifies whether queries and keys in memory are normalized before computing their dot

  products.

.. _EMComposition_Softmax_Gain:

* **softmax_gain** : specifies the gain (inverse temperature) used for softmax normalizing the dot products of

  queries and keys in memory (see `EMComposition_Execution` below).  The following options can be used:

  * numeric value: the value is used as the gain of the `SoftMax` Function for the EMComposition's

    `softmax_nodes <EMComposition.softmax_nodes>`.

  * *ADAPTIVE*: the `adapt_gain <SoftMax.adapt_gain>` method of the `SoftMax` Function is used to adaptively set

    the `softmax_gain <EMComposition.softmax_gain>` based on the entropy of the dot products, in order to preserve

    the distribution over non- (or near) zero entries irrespective of how many (near) zero entries there are

    (see `Thresholding and Adaptive Gain <SoftMax_AdaptGain>` for additional details).

  * *CONTROL*: a `ControlMechanism` is created, and its `ControlSignal` is used to modulate the `softmax_gain

    <EMComposition.softmax_gain>` parameter of the `SoftMax` function of the EMComposition's `softmax_nodes

    <EMComposition.softmax_nodes>`.

  If *None* is specified, the default value for the `SoftMax` function is used.

.. _EMComposition_Softmax_Threshold:

* **softmax_threshold**: if this is specified, and **softmax_gain** is specified with a numeric value,

  then any values below the specified threshold are set to 0 before the dot products are softmaxed

  (see *mask_threhold* under `Thresholding and Adaptive Gain <SoftMax_AdaptGain>` for additional details).

.. _EMComposition_Softmax_Choice:

* **softmax_choice** : specifies how the `SoftMax` Function of each of the EMComposition's `softmax_nodes

  <EMComposition.softmax_nodes>` is used, with the dot products of queries and keys, to generate a retrieved item;

  the following are the options that can be used and the retrieved value they produce:

  * *WEIGHTED_AVG* (default): softmax-weighted average of entries, based on their dot products with the key(s).

  * *ARG_MAX*: entry with the largest dot product (one with lowest index in `memory <EMComposition.memory>`)\

               if there are identical ones).

  * *PROBABISTIC*: probabilistically chosen entry based on softmax-transformed distribution of dot products.

  .. warning::

     Use of the *ARG_MAX* and *PROBABILISTIC* options is not compatible with learning, as these implement a discrete

     choice and thus are not differentiable. Constructing an EMComposition with **softmax_choice** set to either of

     these options and **enable_learning** set to True (or a list with any True entries) will generate a warning, and

     calling the EMComposition's `learn <Composition.learn>` method will generate an error; it must be changed to

     *WEIGHTED_AVG* to execute learning.

  .. technical_note::

     The *WEIGHTED_AVG* option is passed as *ALL* to the **output** argument of the `SoftMax` Function, *ARG_MAX* is

     passed as *ARG_MAX_INDICATOR*; and *PROBALISTIC* is passed as *PROB_INDICATOR*; the other SoftMax options are

     not currently supported.

.. _EMComposition_Learning:

*Learning*

EMComposition supports two forms of learning -- error backpropagation and the learning of `field_weights

<EMComposition.field_weights>` -- that can be configured by the following arguments of the EMComposition's constructor:

* **enable_learning** : specifies whether learning is enabled for the EMComposition and, if so, which `retrieved_nodes

    <EMComposition.retrieved_nodes>` are used to compute errors, and propagate these back through the network. If

    **enable_learning** is False, then no learning occurs, including of `field_weights <EMComposition.field_weights>`).

    If it is True, then all of the `retrieved_nodes <EMComposition.retrieved_nodes>` participate in learning:  For

    those that do not project to an outer Composition (i.e., one in which the EMComposition is `nested

    <Composition_Nested>`), a `TARGET <NodeRole.TARGET>` node is constructed for each, and used to compute errors that

    are backpropagated through the network to its `query_input_nodes <EMComposition.query_input_nodes>` and

    `value_input_nodes <EMComposition.value_input_nodes>`, and on to any nodes that project to it from a composition

    in which the EMComposition is `nested <Composition_Nested>`; retrieved_nodes that *do* project to an outer

    Composition receive their errors from those nodes, which are also backpropagated through the EMComposition.

    If **enable_learning** is a list, then only the `retrieved_nodes <EMComposition.retrieved_nodes>` specified in the

    list participate in learning, and errors are computed only for those nodes.  The list must contain the same

    number of entries as there are `fields <EMComposition_Fields>` and corresponding `retreived_nodes

    <EMComposition.retrieved_nodes>`, and each entry must be a boolean that specifies whether the corresponding

    `retrieved_node <EMComposition.retrieved_nodes>` is used for learning.

* **learn_field_weight** : specifies whether `field_weights <EMComposition.field_weights>` are modifiable during

    learning (see `field_weights <EMComposition.field_weights>` and `Learning <EMComposition_Learning>` for additional

    information.  For learning of `field_weights <EMComposition.field_weights>` to occur, **enable_learning** must

    also be True, or it must be a list with at least one True entry.

* **learning_rate** : specifies the rate at which  `field_weights <EMComposition.field_weights>` are learned if

  **learn_field_weight** is True; see `Learning <EMComposition_Learning>` for additional information.

.. _EMComposition_Structure:

Structure

---------

.. _EMComposition_Input:

*Input*

~~~~~~~

The inputs corresponding to each key and value field are represented as `INPUT <NodeRole.INPUT>` `Nodes

<Composition_Nodes>` of the EMComposition, listed in its `query_input_nodes <EMComposition.query_input_nodes>`

and `value_input_nodes <EMComposition.value_input_nodes>` attributes, respectively,

.. _EMComposition_Memory:

*Memory*

~~~~~~~~

The `memory <EMComposition.memory>` attribute contains a record of the entries in the EMComposition's memory. This

is in the form of a 3d array, in which rows (axis 0) are entries, columns (axis 1) are fields, and items (axis 2) are

the values of an entry in a given field.  The number of fields is determined by the `memory_template

<EMComposition_Memory_Template>` argument of the EMComposition's constructor, and the number of entries is determined

by the `memory_capacity <EMComposition_Memory_Capacity>` argument.

  .. _EMComposition_Memory_Storage:

  .. technical_note::

     The memories are actually stored in the `matrix <MappingProjection.matrix>` parameters of the`MappingProjections`

     from the `combined_softmax_node <EMComposition.combined_softmax_node>` to each of the `retrieved_nodes

     <EMComposition.retrieved_nodes>`. Memories associated with each key are also stored (in inverted form)

     in the `matrix <MappingProjection.matrix>` parameters of the `MappingProjection <MappingProjection>`

     from the `query_input_nodes <EMComposition.query_input_nodes>` to each of the corresponding `match_nodes

     <EMComposition.match_nodes>`. This is done so that the match of each query to the keys in memory for the

     corresponding field can be computed simply by passing the input for each query through the Projection (which

     computes the dot product of the input with the Projection's `matrix <MappingProjection.matrix>` parameter) to

     the corresponding match_node; and, similarly, retrieivals can be computed by passing the softmax distributions

     and weighting for each field computed in the `combined_softmax_node <EMComposition.combined_softmax_node>`

     through its Projection to each `retrieved_node <EMComposition.retrieved_nodes>` (which are inverted versions

     of the matrices of the `MappingProjections <MappingProjection>` from the `query_input_nodes

     <EMComposition.query_input_nodes>` to each of the corresponding `match_nodes <EMComposition.match_nodes>`),

     to compute the dot product of the weighted softmax over entries with the corresponding field of each entry

     that yields the retreieved value for each field.

.. _EMComposition_Output:

*Output*

~~~~~~~~

The outputs corresponding to retrieved value for each field are represented as `OUTPUT <NodeRole.INPUT>` `Nodes

<Composition_Nodes>` of the EMComposition, listed in its `retrieved_nodes <EMComposition.retrieved_nodes>` attribute.

.. _EMComposition_Execution:

Execution

---------

The arguments of the `run <Composition.run>` , `learn <Composition.learn>` and `Composition.execute`

methods are the same as those of a `Composition`, and they can be passed any of the arguments valid for

an `AutodiffComposition`.  The details of how the EMComposition executes are described below.

.. _EMComposition_Processing:

*Processing*

~~~~~~~~~~~~

When the EMComposition is executed, the following sequence of operations occur

(also see `figure <EMComposition_Example_Fig>`):

* **Input**.  The inputs to the EMComposition are provided to the `query_input_nodes <EMComposition.query_input_nodes>`

  and `value_input_nodes <EMComposition.value_input_nodes>`.  The former are used for matching to the corresponding

  `fields <EMComposition_Fields>` of the `memory <EMComposition.memory>`, while the latter are retrieved but not used

  for matching.

* **Concatenation**. By default, the input to every `query_input_node <EMComposition.query_input_nodes>` is passed to a

  to its own `match_node <EMComposition.match_nodes>` through a `MappingProjection` that computes its

  dot product with the corresponding field of each entry in `memory <EMComposition.memory>`.  In this way, each

  match is normalized so that, absent `field_weighting <EMComposition_Field_Weights>`, all keys contribute equally to

  retrieval irrespective of relative differences in the norms of the queries or the keys in memory. However, if the

  `field_weights <EMComposition.field_weights>` are the same for all `keys <EMComposition_Field_Weights>` and

  `normalize_memories <EMComposition.normalize_memories>` is True, then the inputs provided to the `query_input_nodes

  <EMComposition.query_input_nodes>` are concatenated into a single vector (in the

  `concatenate_keys_node <EMComposition.concatenate_keys_node>`), which is passed to a single `match_node

  <EMComposition.match_nodes>`.  This may be more computationally efficient than passing each query through its own

  `match_node <EMComposition.match_nodes>`,

  COMMENT:

  FROM CodePilot: (OF HISTORICAL INTEREST?)

  and may also be more effective if the keys are highly correlated (e.g., if they are different representations of

  the same stimulus).

  COMMENT

  however it will not necessarily produce the same results as passing each query through its own `match_node

  <EMComposition.match_nodes>` (see `concatenate keys <`concatenate_keys_node

  >` for additional

  information).

* **Match memories by field**. The values of each `query_input_node <EMComposition.query_input_nodes>` (or the

  `concatenate_keys_node <EMComposition.concatenate_keys_node>` if `concatenate_keys <EMComposition_Concatenate_Keys>`

  attribute is True) are passed through a `MappingProjection` that computes the dot product of the input with each

  memory for the corresponding field, the result of which is passed to the corresponding `match_node

  <EMComposition.match_nodes>`.

* **Softmax normalize matches over fields**. The dot product for each key field is passed from the `match_node

  <EMComposition.match_nodes>` to the corresponding `softmax_node <EMComposition.softmax_nodes>`, which applies

  the `SoftMax` Function to normalize the dot products for each key field.  If a numerical value is specified for

  `softmax_gain <EMComposition.softmax_gain>`, that is used as the gain (inverse temperature) for the SoftMax Function;

  if *ADAPTIVE* is specified, then the `SoftMax.adapt_gain` function is used to adaptively set the gain based on the

  dot products in each field;  if *CONTROL* is specified, then the dot products are monitored by a `ControlMechanism`

  that uses the `adapt_gain <Softmax.adapt_gain>` method of the SoftMax Function to modulate its `gain <Softmax.gain>`

  parameter; if None is specified, the default value of the `Softmax` Function is used as the `gain <Softmax.gain>`

  parameter (see `Softmax_Gain <EMComposition_Softmax_Gain>` for additional  details).

* **Weight fields**. If `field weights <EMComposition_Field_Weights>` are specified, then the softmax normalized dot

  product for each key field is passed to the corresponding `field_weight_node <EMComposition.field_weight_nodes>`

  where it is multiplied by the corresponding `field_weight <EMComposition.field_weights>` (if

  `use_gating_for_weighting <EMComposition.use_gating_for_weighting>` is True, this is done by using the `field_weight

  <EMComposition.field_weights>` to output gate the `softmax_node <EMComposition.softmax_nodes>`). The weighted softamx

  vectors for all key fields are then passed to the `combined_softmax_node <EMComposition.combined_softmax_node>`,

  where they are haddamard summed to produce a single weighting for each memory.

* **Retrieve values by field**. The vector of softmax weights for each memory generated by the `combined_softmax_node

  <EMComposition.combined_softmax_node>` is passed through the Projections to the each of the `retrieved_nodes

  <EMComposition.retrieved_nodes>` to compute the retrieved value for each field.

* **Decay memories**.  If `memory_decay <EMComposition.memory_decay>` is True, then each of the memories is decayed

  by the amount specified in `memory_decay_rate <EMComposition.memory_decay_rate>`.

    .. technical_note::

       This is done by multiplying the `matrix <MappingProjection.matrix>` parameter of the `MappingProjection` from

       the `combined_softmax_node <EMComposition.combined_softmax_node>` to each of the `retrieved_nodes

       <EMComposition.retrieved_nodes>`, as well as the `matrix <MappingProjection.matrix>` parameter of the

       `MappingProjection` from each `query_input_node <EMComposition.query_input_nodes>` to the corresponding

       `match_node <EMComposition.match_nodes>` by `memory_decay <EMComposition.memory_decay_rate>`,

        by 1 - `memory_decay <EMComposition.memory_decay_rate>`.

.. _EMComposition_Storage:

* **Store memories**. After the values have been retrieved, the inputs to for each field (i.e., values in the

  `query_input_nodes <EMComposition.query_input_nodes>` and `value_input_nodes <EMComposition.value_input_nodes>`)

  are added by the `storage_node <EMComposition.storage_node>` as a new entry in `memory <EMComposition.memory>`,

  replacing the weakest one if `memory_capacity <EMComposition_Memory_Capacity>` has been reached.

    .. technical_note::

       This is done by adding the input vectors to the the corresponding rows of the `matrix <MappingProjection.matrix>`

       of the `MappingProjection` from the `retreival_weighting_node <EMComposition.combined_softmax_node>` to each

       of the `retrieved_nodes <EMComposition.retrieved_nodes>`, as well as the `matrix <MappingProjection.matrix>`

       parameter of the `MappingProjection` from each `query_input_node <EMComposition.query_input_nodes>` to the

       corresponding `match_node <EMComposition.match_nodes>` (see note `above <EMComposition_Memory_Storage>` for

       additional details). If `memory_capacity <EMComposition_Memory_Capacity>` has been reached, then the weakest

       memory (i.e., the one with the lowest norm across all fields) is replaced by the new memory.

COMMENT:

FROM CodePilot: (OF HISTORICAL INTEREST?)

inputs to its `query_input_nodes <EMComposition.query_input_nodes>` and

`value_input_nodes <EMComposition.value_input_nodes>` are assigned the values of the corresponding items in the

`input <Composition.input>` argument.  The `combined_softmax_node <EMComposition.field_weight_node>`

computes the dot product of each query with each key in memory, and then applies a softmax function to each row of the

resulting matrix.  The `retrieved_nodes <EMComposition.retrieved_nodes>` then compute the dot product of the

softmaxed values for each memory with the corresponding value for each memory, and the result is assigned to the

corresponding `output <Composition.output>` item.

COMMENT

.. _EMComposition_Training:

*Training*

~~~~~~~~~~

If `learn <Composition.learn>` is called, `enable_learning <EMComposition.enable_learning>` is True or a list with

any True entries, then errors will be computed for each of the `retrieved_nodes <EMComposition.retrieved_nodes>`

that is specified for learning (see `Learning <EMComposition_Learning>` for details about specification). These errors

are derived either from any errors backprpated to the EMComposition from an outer Composition in which it is `nested

<Composition_Nested>`, or locally by the difference between the `retrieved_nodes <EMComposition.retrieved_nodes>`

and the `target_nodes <EMComposition.target_nodes>` that are created for each of the `retrieved_nodes

<EMComposition.retrieved_nodes>` that do not project to an outer Composition.  These errors are then backpropagated

through the EMComposition to the `query_input_nodes <EMComposition.query_input_nodes>` and `value_input_nodes

<EMComposition.value_input_nodes>`, and on to any nodes that project to it from a composition in which the

EMComposition is `nested <Composition_Nested>`.

If `learn_field_weights <EMComposition.learn_field_weights>` is also True, then the `field_weights

<EMComposition.field_weights>` are modified to minimize the error passed to the EMComposition retrieved nodes, using the

`learning_rate <EMComposition.learning_rate>` specified in the `learning_rate <EMComposition.learning_rate>` attribute.

If `learn_field_weights <EMComposition.learn_field_weights>` is False (or `run <Composition.run>` is called, then the

If `learn_field_weights <EMComposition.learn_field_weights>` is False), then the `field_weights

<EMComposition.field_weights>` are not modified and the EMComposition is simply executed

without any modification, and error signals are passed to the nodes that project to its `query_input_nodes

<EMComposition.query_input_nodes>` and `value_input_nodes <EMComposition.value_input_nodes>`.

  .. note::

     The only parameters modifable by learning in the EMComposition are its `field_weights

     <EMComposition.field_weights>`; all other parameters (including all other Projection `matrices

     <MappingProjection.matrix>`) are fixed, and used only to compute gradients and backpropagate errors.

  .. technical_note::

     Although memory storage is implemented as a form of learning (though modification of MappingProjection

     `matrix <MappingProjection.matrix>` parameters; see `memory storage <EMComposition_Memory_Storage>`),

     this occurs irrespective of how EMComposition is run (i.e., whether `learn <Composition.learn>` or `run

     <Composition.run>` is called), and is not affected by the `learn_field_weights <EMComposition.learn_field_weights>`

     or `learning_rate <EMComposition.learning_rate>` attributes, which pertain only to whether the `field_weights

     <EMComposition.field_weights>` are modified during learning.  Furthermore, when run in PyTorch mode, storage

     is executed after the forward() and backward() passes are complete, and is not considered as part of the

     gradient calculations.

.. _EMComposition_Examples:

Examples

--------

The following are examples of how to configure and initialize the EMComposition's `memory <EMComposition.memory>`:

*Visualizing the EMComposition*

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The EMComposition can be visualized graphically, like any `Composition`, using its `show_graph

<ShowGraph_show_graph_Method>` method.  For example, the figure below shows an EMComposition that

implements a simple dictionary, with one key field and one value field, each of length 5::

    >>> import psyneulink as pnl

    >>> em = EMComposition(memory_template=(2,5))

    >>> em.show_graph()

    <BLANKLINE>

.. _EMComposition_Example_fig:

.. figure:: _static/EMComposition_Example_fig.svg

   :alt: Exxample of an EMComposition

   :align: left

       **Example of an EMComposition**

       .. note::

          The order in which the nodes at a given level (e.g., the `INPUT <NodeRole.INPUT>` or `OUTPUT

          <NodeRole.OUTPUT>` `Nodes <Composition_Nodes>`) are shown in the diagram is arbitrary, and does not necessarily

          reflect the order in which they are created or specied in the script.

.. _EMComposition_Example_Memory_Template:

*Memory Template*

~~~~~~~~~~~~~~~~~

The `memory_template <EMComposition_Memory_Template>` argument of a EMComposition's constructor is used to configure

it `memory <EMComposition.memory>`, which can be specified using either a tuple or a list or array.

.. _EMComposition_Example_Tuple_Spec:

**Tuple specification**

The simplest form of specification is a tuple, that uses the `numpy shape

<https://numpy.org/doc/stable/reference/generated/numpy.shape.html>`_ format.  If it has two elements (as in the

example above), the first specifies the number of fields, and the second the length of each field.  In this case,

a default number of entries (1000) is created:

    >>> em.memory_capacity

    1000

The number of entries can be specified explicitly in the EMComposition's constructor, using either the

`memory_capacity <EMComposition_Memory_Capacity>` argument, or by using a 3-item tuple to specify the

`memory_template <EMComposition_Memory_Template>` argument, in which case the first element specifies

the  number of entries, while the second and their specify the number of fields and the length of each field,

respectively.  The following are equivalent::

    >>> em = EMComposition(memory_template=(2,5), memory_capcity=4)

and

    >>> em = EMComposition(memory_template=(4,2,5))

both of which create a memory with 4 entries, each with 2 fields of length 5. The contents of `memory

<EMComposition_Memory>` can be inspected using the `memory <EMComposition.memory>` attribute::

    >>> em.memory

    [[array([0., 0., 0., 0., 0.]), array([0., 0., 0., 0., 0.])],

     [array([0., 0., 0., 0., 0.]), array([0., 0., 0., 0., 0.])],

     [array([0., 0., 0., 0., 0.]), array([0., 0., 0., 0., 0.])],

     [array([0., 0., 0., 0., 0.]), array([0., 0., 0., 0., 0.])]]

The default for `memory_capacity <EMComposition.memory_capacity>` is 1000, which is used if it is not otherwise

specified.

**List or array specification**

Note that in the example above the two fields have the same length (5). This is always the case when a tuple is used,

as it generates a regular array.  A list or numpy array can also be used to specify the **memory_template** argument.

For example, the following is equivalent to the examples above::

    >>> em = EMComposition(memory_template=[[0,0,0],[0,0,0]], memory_capacity=4)

However, a list or array can be used to specify fields of different length (i.e., as a ragged array).  For example,

the following specifies one field of length 3 and another of length 1::

    >>> em = EMComposition(memory_template=[[0,0,0],[0]], memory_capacity=4)

    >>> em.memory

    [[[array([0., 0., 0.]), array([0.])]],

     [[array([0., 0., 0.]), array([0.])]],

     [[array([0., 0., 0.]), array([0.])]],

     [[array([0., 0., 0.]), array([0.])]]]

.. _EMComposition_Example_Memory_Fill:

**Memory fill**

Note that the examples above generate a warning about the use of zeros to initialize the memory. This is

because the default value for **memory_fill** is ``0``, and the default value for `normalize_memories

<EMComposition.normalize_memories>` is True, which will cause a divide by zero warning when memories are

normalized. While this doesn't crash, it will result in nan's that are likely to cauase problems elsewhere.

This can be avoided by specifying a non-zero  value for **memory_fill**, such as small number::

    >>> em = EMComposition(memory_template=[[0,0,0],[0]], memory_capacity=4, memory_fill=.001)

    >>> em.memory

    [[[array([0.001, 0.001, 0.001]), array([0.001])]],

     [[array([0.001, 0.001, 0.001]), array([0.001])]],

     [[array([0.001, 0.001, 0.001]), array([0.001])]],

     [[array([0.001, 0.001, 0.001]), array([0.001])]]]

Here, a single value was specified for **memory_fill** (which can be a float or int), that is used to fill all values.

Random values can be assigned using a tuple to specify and internval between the first and second elements.  For

example, the following uses random values between 0 and 0.01 to fill all entries::

    >>> em = EMComposition(memory_template=[[0,0,0],[0]], memory_capacity=4, memory_fill=(0,0.01))

    >>> em.memory

    [[[array([0.00298981, 0.00563404, 0.00444073]), array([0.00245373])]],

     [[array([0.00148447, 0.00666486, 0.00228882]), array([0.00237541])]],

     [[array([0.00432786, 0.00035378, 0.00265932]), array([0.00980598])]],

     [[array([0.00151163, 0.00889032, 0.00899815]), array([0.00854529])]]]

.. _EMComposition_Example_Multiple_Entries:

**Multiple entries**

In the examples above, a single entry was specified, and that was used as a template for initializing the remaining

entries in memory. However, a list or array can be used to directly initialize any or all entries. For example, the

following initializes memory with two specific entries::

    >>> em = EMComposition(memory_template=[[[1,2,3],[4]],[[100,101,102],[103]]], memory_capacity=4)

    >>> em.memory

    [[[array([1., 2., 3.]), array([4.])]],

     [[array([100., 101., 102.]), array([103.])]],

     [[array([0., 0., 0.]), array([0.])]],

     [[array([0., 0., 0.]), array([0.])]]]