PrincetonUniversity / PsyNeuLink / 15917088825

# Princeton University licenses this file to You under the Apache License, Version 2.0 (the "License");

# you may not use this file except in compliance with the License.  You may obtain a copy of the License at:

#     http://www.apache.org/licenses/LICENSE-2.0

# Unless required by applicable law or agreed to in writing, software distributed under the License is distributed

# on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.

# See the License for the specific language governing permissions and limitations under the License.

# ********************************************* AutodiffComposition *************************************************

"""

Contents

--------

  * `AutodiffComposition_Overview`

  * `AutodiffComposition_Creation`

      - `AutodiffComposition`

          - `AutodiffComposition_Modulatory_Mechanisms`

          - `AutodiffComposition_Bias_Parameters`

          - `AutodiffComposition_Nesting`

          - `AutodiffComposition_Learning_Rates`

          - `AutodiffComposition_Exchange_With_Torch_Parameters`

          - `AutodiffComposition_Post_Construction_Modification`

      * `AutodiffComposition_Execution`

          - `AutodiffComposition_PyTorch`

          - `AutodiffComposition_LLVM`

          - `AutodiffComposition_Python`

          - `AutodiffComposition_Nested_Modulation`

          - `AutodiffComposition_Logging`

  * `AutodiffComposition_Examples`

  * `AutodiffComposition_Class_Reference`

.. _AutodiffComposition_Overview:

Overview

--------

AutodiffComposition is a subclass of `Composition` for constructing and training feedforward neural network

either, using either direct compilation (to LLVM) or automatic conversion to `PyTorch <https://pytorch.org/>`_,

both of which considerably accelerate training (by as much as three orders of magnitude) compared to the

`standard implementation of learning  <Composition_Learning_Standard>` in a Composition.  Although an

AutodiffComposition is constructed and executed in much the same way as a standard Composition, it largely restricted

to feedforward neural networks using `supervised learning <Composition_Learning_Supervised>`, and in particular the

the `backpropagation learning algorithm <https://en.wikipedia.org/wiki/Backpropagation>`_. although it can be used for

some forms of `unsupervised learning <Composition_Learning_Unsupervised>` that are supported in PyTorch (e.g.,

`self-organized maps <https://github.com/giannisnik/som>`_).

.. _AutodiffComposition_Creation:

Creating an AutodiffComposition

-------------------------------

An AutodiffComposition can be created by calling its constructor, and then adding `Components <Component>` using

the standard `Composition methods <Composition_Creation>` for doing so (e.g., `add_node <Composition.add_node>`,

`add_projection <Composition.add_projections>`,  `add_linear_processing_pathway

<Composition.add_linear_processing_pathway>`, etc.). The constructor also includes a number of parameters that are

specific to the AutodiffComposition (see `AutodiffComposition_Class_Reference` for a list of these parameters,

and `examples <AutodiffComposition_Examples>` below). While an AutodiffComposition can generally be created using the

same methods as a standard Composition, there are a few restrictions that apply to its construction, summarized below.

.. _AutodiffComposition_Restrictions:

*Only one OutputPort per Node*

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The `Nodes <Composition_Nodes>` of an AutodiffComposition currently can have only *one* `OutputPort`, though that

can have more than one `efferent <Port_Base.efferents>` `MappingProjection`.  Nodes can also have more than one

`InputPort`, that can receive more than one `afferent `path_afferent <Port_Base.path_afferents>` Projections.

.. _AutodiffComposition_Modulatory_Mechanisms:

*No Modulatory Components*

~~~~~~~~~~~~~~~~~~~~~~~~~~

All of the Components in an AutodiffComposition must be able to be subjected to `learning <Composition_Learning>`,

which means that no `ModulatoryMechanisms <ModulatoryMechanism>` can be included in an AutodiffComposition.

Specifically, this precludes any `learning components <Composition_Learning_Components>`, `ControlMechanisms

<ControlMechanism>`, or a `controller <Composition_Controller>`.

.. _Autodiff_Learning_Components_Warning:

*Learning Components*.  An AutodiffComposition **cannot include any** `learning components

<Composition_Learning_Components>` themselves (i.e., `LearningMechanisms <LearningMechanism>`, `LearningSignals

<LearningSignal>`, or `LearningProjections <LearningProjection>`, nor the `ComparatorMechanism`

or `ObjectiveMechanism` used to compute the loss for learning). These are constructed

automatically when learning is executed in `Python mode <AutodiffComposition_Python>` or `LLVM mode

<AutodiffComposition_LLVM>`, and PyTorch-compatible Components are constructed when it is executed in

`PyTorch mode <AutodiffComposition_PyTorch>`.

*Control Components*. An AutodiffComposition also cannot include any `ControlMechanisms <ControlMechanism>` or a

`controller <Composition_Controller>`.  However, it *can* include Mechanisms that are subject to modulatory control

(see `Figure <ModulatorySignal_Anatomy_Figure>`, and `modulation <ModulatorySignal_Modulation>`) by ControlMechanisms

*outside* the Composition, including the controller of a Composition within which the AutodiffComposition is nested.

That is, an AutodiffComposition can be `nested in a Composition <Composition_Nested>` that has other such Components

(see `AutodiffComposition_Nested_Modulation` below).

.. _AutodiffComposition_Bias_Parameters:

*No Bias Parameters*

~~~~~~~~~~~~~~~~~~~~

AutodiffComposition does not (currently) support the *automatic* construction of separate bias parameters.

Thus, when constructing the PyTorch version of an AutodiffComposition, the `bias

<https://www.pytorch.org/docs/stable/nn.html#torch.nn.Module>`_ parameter of any PyTorch modules are set to False.

However, biases can be implemented using `Composition_Bias_Nodes`.

.. _AutodiffComposition_Nesting:

*Nesting*

~~~~~~~~~

An AutodiffComposition can be `nested <Composition_Nested>` inside another Composition for learning, and there can

be any level of such nestings.  However, all of the nested Compositions must be AutodiffCompositions. Furthermore, all

nested Compositions use the `learning_rate <AutodiffComposition.learning_rate>` specified for the outermost Composition,

whether this is specified in the call to its `learn <AutodiffComposition.learn>` method, its constructor, or its

default value is being used (see `learning_rate <AutodiffComposition.learning_rate>` below for additional details).

.. technical_note::

   Projections from `Nodes <Composition_Nodes>` in an immediately enclosing outer Composition to the `input_CIM

   <Composition.input_CIM>` of a nested Composition, and from its `output_CIM <Composition.output_CIM>` to Nodes

   in the outer Composition are subject to learning;  however those within the nested Composition itself (i.e.,

   from its input_CIM to its INPUT Nodes and from its OUTPUT Nodes to its output_CIM) are *not* subject to learning,

   as they serve simply as conduits of information between the outer Composition and the nested one.

.. warning::

   Nested Compositions are supported for learning only in `PyTorch mode <AutodiffComposition_PyTorch>`, and will

   cause an error if the `learn <AutodiffComposition.learn>` method of an AutodiffComposition is executed in

   `Python mode <AutodiffComposition_Python>` or `LLVM mode <AutodiffComposition_LLVM>`.

.. _AutodiffComposition_Learning_Rates:

*Learning Rates and Optimizer Params*

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The **optimizer_params** argument of the constructor can be used to specify parameters for the optimizer used for

learning by the AutodiffComposition. At present, this is restricted to overriding the `learning_rate

<AutodiffComposition.learning_rate>` Parameter of the Composition (used as the default by the `optimizer

<AutodiffComposition.optimizer>`) to assign individual learning rates to specific Projections. This is done by

specifying **optimizer_params** as a dict, in which each key is a reference to a learnable `MappingProjection`

in the AutodiffComposition, and the value of which specifies its learning_rate. Sublcasses of AutodiffComposition may

involve different forms of specification and/or support other parameters for the optimizer. Projections that are not

sepcified in **optimizer_params** use, in order of precedence: the `learning_rate <AutodiffComposition.learning_rate>`

specified in the call to the AutodiffComposition's `learn <AutodiffComposition.learn>` method, the **learning_rate**

argument of its constructor, or the default value for the AutodiffComposition.

.. _AutodiffComposition_Exchange_With_Torch_Parameters:

*Exchanging Parameters with Pytorch Modules*

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The AutodiffComposition's `copy_torch_param_to_projection_matrix` and `copy_projection_matrix_to_torch_param` methods

can be used to exchange weight matrices between the parameters of a PyTorch module and the `matrix

<MappingProjection.matrix>` Parameter of a `MappingProjection` in the AutodiffComposition. Pytorch Parameters can

be referenced flexibly, either by the Parameter object itself, or by the module and either the name or index of the

Parameter in the module's state_dict or parameter list, respectively. Slices of PyTorch Parameters can also be used,

for cases in which the matrix of a Project corresponds to only a subpart of the PyTorch Parameter (e.g., for

`GRUComposition`). Both methods return the item assigned.

.. _AutodiffComposition_Post_Construction_Modification:

*No Post-construction Modification*

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

COMMENT:

IS THIS STILL TRUE?  TEST?

COMMENT

Mechanisms or Projections should not be added to or deleted from an AutodiffComposition after it has

been executed. Unlike an ordinary Composition, AutodiffComposition does not support this functionality.

.. _AutodiffComposition_Execution:

Execution

---------

An AutodiffComposition's `run <Composition.run>`, `execute <Composition.execute>`, and `learn <Composition.learn>`

methods are the same as for a `Composition`.  However, the **execution_mode** in the `learn <Composition.learn>`

method has different effects than for a standard Composition, that determine whether it uses `LLVM compilation

<AutodiffComposition_LLVM>` or `translation to PyTorch <AutodiffComposition_PyTorch>` to execute learning.

These are each described in greater detail below, and summarized in this `table <Composition_Compilation_Table>`

which provides a comparison of the different modes of execution for an AutodiffComposition and standard `Composition`.

.. _AutodiffComposition_PyTorch:

*PyTorch mode*

~~~~~~~~~~~~~~

COMMENT:

# 7/10/24 - FIX:

.. _AutodiffComposition_PyTorch_LearningScale:

   ADD DESCRIPTION OF HOW LearningScale SPECIFICATIONS MAP TO EXECUTION OF pytorch_rep:

      OPTIMIZATION STEP:

      for AutodiffCompositions, this corresponds to a single call to `foward()` and `backward()`

            methods of the Pytorch model

COMMENT

This is the default for an AutodiffComposition, but, can be specified explicitly by setting **execution_mode** =

`ExecutionMode.PyTorch` in the `learn <Composition.learn>` method (see `example <BasicsAndPrimer_Rumelhart_Model>`

in `BasicsAndPrimer`).  In this mode, the AutodiffComposition is automatically translated to a `PyTorch

<https://pytorch.org>`_ model for learning.  This is comparable in speed to `LLVM compilation

<_AutodiffComposition_LLVM>`, but provides greater flexiblity, including the ability to include nested

AutoDiffCompositions in learning. Although it is best suited for use with `supervised learning

<Composition_Learning_Supervised>`, it can also be used for some forms of `unsupervised learning

<Composition_Learning_Unsupervised>` that are supported in PyTorch (e.g., `self-organized maps

<https://github.com/giannisnik/som>`_).

    .. _AutodiffComposition_PyTorch_Note:

    .. note::

       While specifying `ExecutionMode.PyTorch` in the `learn <Composition.learn>`  method of an AutodiffComposition

       causes it to use PyTorch for training, specifying this in the `run <Compositon.run>` method causes it to be

       executed using the *Python* interpreter (and not PyTorch);  this is so that any modulation can take effect

       during execution (see `AutodiffComposition_Nested_Modulation` below), which is not supported by PyTorch.

    .. warning::

      * Specifying `ExecutionMode.LLVMRun` or `ExecutionMode.PyTorch` in the learn() method of a standard

        `Composition` causes an error.

COMMENT:

FIX: ADD MENTION OF TARGET NODES AND PYTORCH WRAPPERS

COMMENT

.. _AutodiffComposition_LLVM:

*LLVM mode*

~~~~~~~~~~~

This is specified by setting **execution_mode** = `ExecutionMode.LLVMRun` in the `learn <Composition.learn>` method

of an AutodiffCompositon.  This provides the fastest performance, but is limited to `supervised learning

<Composition_Learning_Supervised>` using the `BackPropagation` algorithm. This can be run using standard forms of

loss, including mean squared error (MSE) and cross entropy, by specifying this in the **loss_spec** argument of

the constructor (see `AutodiffComposition <AutodiffComposition_Class_Reference>` for additional details, and

`Compilation Modes <Composition_Compiled_Modes>` for more information about executing a Composition in compiled mode.

    .. note::

       Specifying `ExecutionMode.LLVMRun` in either the `learn <Composition.learn>` and `run <Composition.run>`

       methods of an AutodiffComposition causes it to (attempt to) use compiled execution in both cases; this is

       because LLVM compilation supports the use of modulation in PsyNeuLink models (as compared to `PyTorch mode

       <AutodiffComposition_PyTorch>`; see `note <AutodiffComposition_PyTorch_Note>` below).

COMMENT:

FIX: 8/13/23 - COMPLETE DOCS HERE

COMMENT

.. _AutodiffComposition_Python:

*Python mode*

~~~~~~~~~~~~~

An AutodiffComposition can also be run using the standard PsyNeuLink learning components.  However, this cannot

be used if the AutodiffComposition has any nested Compositions, irrespective of whether they are ordinary

Compositions or AutodiffCompositions.

.. _AutodiffComposition_Nested_Modulation:

*Nested Execution and Modulation*

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

# FIX:

Like any other `Composition`, an AutodiffComposition may be `nested <Composition_Nested>` inside another

(see `example <AutodiffComposition_Nested_Example>` below).  However, during learning, none of the internal

Components of the AutodiffComposition (e.g., intermediate layers of a neural network model) are accessible to the

other Components of the outer Composition, (e.g., as sources of information, or for `modulation

<ModulatorySignal_Modulation>`).  However, when

COMMENT:

learning turned off,

COMMENT

it is executed using its `run <Composition.run>` method, then the  AutodiffComposition functions like any other,

and all of its internal Components are accessible to other Components of the outer Composition. Thus, as long as access

to its internal Components is not needed during learning, an `AutodiffComposition` can be trained, and then used to

execute the trained Composition like any other.

.. _AutodiffComposition_Logging:

*Logging*

~~~~~~~~~

Logging in AutodiffCompositions follows the same procedure as `logging in a Composition <Log>`.

However, since an AutodiffComposition internally converts all of its Mechanisms either to LLVM

or to an equivalent PyTorch model, then its inner components are not actually executed. This means that there is

limited support for logging parameters of components inside an AutodiffComposition; Currently, the only supported

parameters are:

1) the `matrix` parameter of Projections

2) the `value` parameter of its inner components

.. _AutodiffComposition_Examples:

Examples

--------

.. _AutodiffComposition_Creation_Example:

The following is an example showing how to create a simple AutodiffComposition, specify its inputs and targets,

and run it with learning enabled and disabled:

    >>> import psyneulink as pnl

    >>> # Set up PsyNeuLink Components

    >>> my_mech_1 = pnl.TransferMechanism(function=pnl.Linear, input_shapes = 3)

    >>> my_mech_2 = pnl.TransferMechanism(function=pnl.Linear, input_shapes = 2)

    >>> my_projection = pnl.MappingProjection(matrix=np.random.randn(3,2),

    ...                     sender=my_mech_1,

    ...                     receiver=my_mech_2)

    >>> # Create AutodiffComposition

    >>> my_autodiff = pnl.AutodiffComposition()

    >>> my_autodiff.add_node(my_mech_1)

    >>> my_autodiff.add_node(my_mech_2)

    >>> my_autodiff.add_projection(sender=my_mech_1, projection=my_projection, receiver=my_mech_2)

    >>> # Specify inputs and targets

    >>> my_inputs = {my_mech_1: [[1, 2, 3]]}

    >>> my_targets = {my_mech_2: [[4, 5]]}

    >>> input_dict = {"inputs": my_inputs, "targets": my_targets, "epochs": 2}

    >>> # Run Composition in learnng mode

    >>> my_autodiff.learn(inputs = input_dict)

    >>> # Run Composition in test mode

    >>> my_autodiff.run(inputs = input_dict['inputs'])

.. _AutodiffComposition_Nested_Example:

The following shows how the AutodiffComposition created in the previous example can be nested and run inside another

Composition::

    >>> # Create outer composition

    >>> my_outer_composition = pnl.Composition()

    >>> my_outer_composition.add_node(my_autodiff)

    >>> # Specify dict containing inputs and targets for nested Composition

    >>> training_input = {my_autodiff: input_dict}

    >>> # Run in learning mode

    >>> result1 = my_outer_composition.learn(inputs=training_input)

    COMMENT:

    >>> # Run with learning disabled (and standard input format)

    >>> no_training_input = {my_autodiff: my_inputs}

    >>> result2 = parentmy_outer_compositionComposition.run(inputs=no_training_input)

    COMMENT

.. _AutodiffComposition_Class_Reference:

Class Reference

---------------

"""

else:

                                                 ReportDevices, EXECUTE_REPORT, LEARN_REPORT, PROGRESS_REPORT)

    AUTODIFF_COMPOSITION, EXECUTION_MODE,

    LEARNING_SCALE_LITERALS, LEARNING_SCALE_NAMES, LEARNING_SCALE_VALUES,

    Loss, LOSSES, MATRIX_WEIGHTS, MINIBATCH, NODE_VALUES, NODE_VARIABLES,

    OPTIMIZATION_STEP, RESULTS, RUN, SOFT_CLAMP, SYNCH_WITH_PNL_OPTIONS,

    RETAIN_IN_PNL_OPTIONS, TARGETS, TRAINED_OUTPUTS, TRIAL, DEFAULT,

)

    'AutodiffComposition'

]

    """

    AutodiffComposition(                        \

        optimizer_type='sgd',

        loss_spec=Loss.MSE,

        weight_decay=0,

        learning_rate=0.001,

        optimizer_params=None,

        disable_learning=False,

        synch_projection_matrices_with_torch=RUN,

        synch_node_variables_with_torch=None,

        synch_node_values_with_torch=RUN,

        synch_results_with_torch=RUN,

        retain_torch_trained_outputs=MINIBATCH,

        retain_torch_targets=MINIBATCH,

        retain_torch_losses=MINIBATCH,

        device=CPU

        )

    Subclass of `Composition` that trains models using either LLVM compilation or `PyTorch <https://pytorch.org>`_;

    see and `Composition <Composition_Class_Reference>` for additional arguments and attributes.  See `Composition`

    for additional arguments to constructor.

    Arguments

    ---------

    optimizer_type : str : default 'sgd'

        the kind of optimizer used in training. The current options are 'sgd' or 'adam'.

    loss_spec : Loss or PyTorch loss function : default Loss.MSE

        specifies the loss function for training; see `Loss` for arguments.

    weight_decay : float : default 0

        specifies the L2 penalty (which discourages large weights) used by the optimizer.

    learning_rate : float : default 0.001

        specifies the learning rate passed to the optimizer if none is specified in the `learn

        <AutdodiffComposition.learn>` method of the AutodiffComposition;

        see `learning_rate <AutodiffComposition.learning_rate>` for additional details.

    optimizer_params : Dict[str: value]

        specifies parameters for the optimizer used for learning by the GRUComposition

        (see `AutodiffComposition_Learning_Rates` for details of specification.

    disable_learning : bool: default False

        specifies whether the AutodiffComposition should disable learning when run in `learning mode

        <Composition.learn>`.

    synch_projection_matrices_with_torch : `LearningScale` : default RUN

        specifies the default for the AutodiffComposition for when to copy Pytorch parameters to PsyNeuLink

        `Projection matrices <MappingProjection.matrix>` (connection weights), which can be overridden by specifying

        the **synch_projection_matrices_with_torch** argument in the `learn <Composition.learn>` method;

        see `synch_projection_matrices_with_torch <AutodiffComposition.synch_projection_matrices_with_torch>`

        for additional details.

    synch_node_variables_with_torch : `LearningScale` : default None

        specifies the default for the AutodiffComposition for when to copy the current input to Pytorch nodes

        to the PsyNeuLink `variable <Mechanism_Base.value>` attribute of the corresponding PsyNeuLink `nodes

        <Composition_Node>`, which can be overridden by specifying the **synch_node_variables_with_torch** argument

        in the `learn <Composition.learn>` method; see `synch_node_variables_with_torch

        <AutodiffComposition.synch_node_variables_with_torch>` for additional details.

    synch_node_values_with_torch : `LearningScale` : default RUN

        specifies the default for the AutodiffComposition for when to copy the current output of Pytorch nodes to the

        PsyNeuLink `value <Mechanism_Base.value>` attribute of the corresponding PsyNeuLink `nodes <Composition_Node>`,

        which can be overridden by specifying the **synch_node_values_with_torch** argument in the `learn

        <Composition.learn>` method; see `synch_node_values_with_torch

        <AutodiffComposition.synch_node_values_with_torch>` for additional details.

    synch_results_with_torch : `LearningScale` : default RUN

        specifies the default for the AutodiffComposition for when to copy the outputs of the Pytorch model

        to the AutodiffComposition's `results <Composition.results>` attribute, which can be overridden by

        specifying the **synch_results_with_torch** argument in the `learn <Composition.learn>` method.

        Note that this differs from **retain_torch_trained_outputs**, which specifies the frequency at which

        the outputs of the PyTorch model are tracked, all of which are stored in the AutodiffComposition's

        `torch_trained_outputs <AutodiffComposition.torch_trained_outputs>` attribute at the end of the run;

        see `synch_results_with_torch <AutodiffComposition.synch_results_with_torch>` for

        additional details.

    retain_torch_trained_outputs : `LearningScale` : default MINIBATCH

        specifies the default for the AutodiffComposition for scale at which the outputs of the Pytorch

        model are tracked, all of which are stored in the AutodiffComposition's `torch_trained_outputs

        <AutodiffComposition.torch_trained_outputs>` attribute at the end of the run; this can be overridden

        by specifying the **retain_torch_trained_outputs** argument in the `learn <Composition.learn>` method.

        Note that this differs from **synch_results_with_torch**, which specifies the frequency with

        which values are called to the AutodiffComposition's `results` attribute; see `retain_torch_trained_outputs

        <AutodiffComposition.retain_torch_trained_outputs>` for additional details.

    retain_torch_targets : `LearningScale` : default MINIBATCH

        specifies the default for the AutodiffComposition for when to copy the targets used for training the

        Pytorch model to the AutodiffComposition's `torch_targets <Composition.torch_targets>` attribute, which can be

        overridden by specifying the **retain_torch_targets** argument in the `learn <Composition.learn>` method;

        see `retain_torch_targets <AutodiffComposition.retain_torch_targets>` for additional details.

    retain_torch_losses : `LearningScale` : default MINIBATCH

        specifies the default for the AutodiffComposition for the scale at which the losses of the Pytorch model

        are tracked, all of which are stored in the AutodiffComposition's `torch_losses <Composition.torch_losses>`

        attribute at the end of the run; see `retain_torch_losses <AutodiffComposition.retain_torch_losses>` for

        additional details.

    device : torch.device : default device-dependent

        specifies the device on which the model is run. If None, the device is set to 'cuda' if available,

        then 'mps`, otherwise 'cpu'.

    Attributes

    ----------

    pytorch_representation : PytorchCompositionWrapper : default None

        represents the PyTorch model of the AutodiffComposition, which is created when the AutodiffComposition is

        run in `PyTorch mode <AutodiffComposition_PyTorch>`.

    optimizer : PyTorch optimizer function

        the optimizer used for training. Depends on the **optimizer_type**, **learning_rate**, and **weight_decay**

        arguments from initialization.

    loss : PyTorch loss function

        the loss function used for training. Depends on the **loss_spec** argument from initialization.

    learning_rate : float or bool

        determines the default learning_rate passed the optimizer, that is applied to all `Projections <Projection>`

        in the AutodiffComposition that are `learnable <MappingProjection.learnable>`, and for which individual rates

        have not been specified (for how to do the latter, see `AutodiffComposition_Learning_Rates`).

        .. note::

           At present, an outermost Compositon's learning rate is applied to any `nested Compositions

           <AutodiffComposition_Nesting>`, whether this is specified in the call to its `learn

           <AutodiffComposition.learn>` method, its constructor, or its default value is being used.

        .. hint::

           To disable updating of a particular `MappingProjection` in an AutodiffComposition, specify either the

           **learnable** parameter of its constructor or its learning_rate specification in the **optimizer_params**

           argument of the AutodiffComposition's constructor to False  (see `AutodiffComposition_Learning_Rates`);

           this applies to MappingProjections at any level of `nesting <AutodiffComposition_Nesting>`

    synch_projection_matrices_with_torch : OPTIMIZATION_STEP, MINIBATCH, EPOCH or RUN

        determines when to copy PyTorch parameters to PsyNeuLink `Projection matrices <MappingProjection.matrix>`

        (connection weights) if this is not specified in the call to `learn <AutodiffComposition.learn>`. Copying more

        frequently keeps the PsyNeuLink representation more closely synchronized with parameter updates in Pytorch,

        but slows performance (see `AutodiffComposition_PyTorch_LearningScale` for information about settings).

    synch_node_variables_with_torch : OPTIMIZATION_STEP, TRIAL, MINIBATCH, EPOCH, RUN or None

        determines when to copy the current input to Pytorch functions to the PsyNeuLink `variable

        <Mechanism_Base.value>` attribute of the corresponding PsyNeuLink `nodes <Composition_Node>`,

        if this is not specified in the call to `learn <AutodiffComposition.learn>`.

        COMMENT:

        8/8/24 - FIX: 3/15/25 ADD EXPLANATION OF WHY THIS IS NOT GENERALLY USEFUL ALONG THE LINES OF THE FOLLOWING

                 ALSO RELATE TO EXECUTE_NODES OPTION ONCE IMPLEMENTED

        This is supported for inspection and debugging, but is not generally useful, as PsyNeuLink uses `Lazy

        Evaluation <Component_Lazy_Updating>`, in which the variable of a node is determined by the input it receives

        during execution.

        COMMENT

        Copying more frequently keeps the PsyNeuLink representation more closely copying more frequently

        keeps them synchronized with parameter updates in Pytorch, but can slow performance (see

        `AutodiffComposition_PyTorch_LearningScale` for information about settings).

    synch_node_values_with_torch : OPTIMIZATION_STEP, MINIBATCH, EPOCH or RUN

        determines when to copy the current output of Pytorch functions to the PsyNeuLink `value

        <Mechanism_Base.value>` attribute of the corresponding PsyNeuLink `nodes <Composition_Node>`,

        if this is not specified in the call to `learn <AutodiffComposition.learn>`. Copying more

        frequently keeps the PsyNeuLink representation more closely synchronized with parameter

        updates in Pytorch, but can also slow performance (see `AutodiffComposition_PyTorch_LearningScale`

        for information about settings).

    synch_results_with_torch : OPTIMIZATION_STEP, TRIAL, MINIBATCH, EPOCH or RUN

        determines when to copy the current outputs of Pytorch nodes to the PsyNeuLink `results

        <Composition.results>` attribute of the AutodiffComposition if this is not specified in

        the call to `learn <AutodiffComposition.learn>`. Copying more frequently keeps the PsyNeuLink

        representation more closely synchronized with parameter updates in Pytorch, but slows performance

        (see `AutodiffComposition_PyTorch_LearningScale` for information about settings).

    retain_torch_trained_outputs : OPTIMIZATION_STEP, MINIBATCH, EPOCH, RUN or None

        determines the scale at which the outputs of the Pytorch model are tracked, all of which are stored in

        the AutodiffComposition's `results <Composition.results>` attribute at the end of the run if this is not

        specified in the call to `learn <AutodiffComposition.learn>`(see `AutodiffComposition_PyTorch_LearningScale`

        for information about settings)

    retain_torch_targets : OPTIMIZATION_STEP, TRIAL, MINIBATCH, EPOCH, RUN or None

        determines the scale at which the targets used for training the Pytorch model are tracked, all of which

        are stored in the AutodiffComposition's `targets <Composition.targets>` attribute at the end of the run

        if this is not specified in the call to `learn <AutodiffComposition.learn>`

        (see `AutodiffComposition_PyTorch_LearningScale` for information about settings).

    retain_torch_losses : OPTIMIZATION_STEP, MINIBATCH, EPOCH, RUN or None

        determines the scale at which the losses of the Pytorch model are tracked, all of which are stored in

        the AutodiffComposition's `torch_losses <Composition.torch_losses>` attribute at the end of the run

        if this is nota specified in the call to `learn <AutodiffComposition.learn>`

        (see `AutodiffComposition_PyTorch_LearningScale` for information about settings).

    torch_trained_outputs : List[ndarray]

        stores the outputs (converted to np arrays) of the Pytorch model trained during learning, at the frequency

        specified by `retain_torch_trained_outputs <AutodiffComposition.retain_torch_trained_outputs>` if it is set

        to *MINIBATCH*, *EPOCH*, or *RUN*; see `retain_torch_trained_outputs

        <AutodiffComposition.retain_torch_trained_outputs>` for additional details.

    torch_targets : List[ndarray]

        stores the targets used for training the Pytorch model during learning at the frequency specified by

        `retain_torch_targets <AutodiffComposition.retain_torch_targets>` if it is set to *MINIBATCH*, *EPOCH*,

        or *RUN*; see `retain_torch_targets <AutodiffComposition.retain_torch_targets>` for additional details.

    torch_losses : list of floats

        stores the average loss after each weight update (i.e. each minibatch) during learning, at the frequency

        specified by `retain_torch_trained_outputs <AutodiffComposition.retain_torch_trained_outputs>` if it is set to *MINIBATCH*,

        *EPOCH*, or *RUN*; see `retain_torch_losses <AutodiffComposition.retain_torch_losses>` for additonal details.

    COMMENT:  FIX: NOT CURRENTLY BEING POPULTED, BUT SEEMS TO BE USED BY _get_total_loss() and early_stopper

    trial_losses = Parameter([])

    COMMENT

    last_saved_weights : path

        path for file to which weights were last saved.

    last_loaded_weights : path

        path for file from which weights were last loaded.

    device : torch.device

        the device on which the model is run.

    """

        # def _validate_memory_template(self, device):

        #     if isinstance(device, str) and device not in [CPU, CUDA, MPS]:

        #         raise AutodiffCompositionError(f"Device must be one of {CPU}, {CUDA}, or {MPS}")

        #

                raise AutodiffCompositionError(f"Value of 'synch_projection_matrices_with_torch' arg "

                                               f"must be one of the following keywords: "

                                               f"{', '.join(LEARNING_SCALE_NAMES)}")

                raise AutodiffCompositionError(f"Value of 'synch_node_variables_with_torch' arg "

                                               f"must be one of the following keywords: "

                                               f"{', '.join(LEARNING_SCALE_NAMES)}")

                raise AutodiffCompositionError(f"Value of 'synch_node_values_with_torch' arg "

                                               f"must be one of the following keywords: "

                                               f"{', '.join(LEARNING_SCALE_NAMES)}")

                raise AutodiffCompositionError(f"Value of 'synch_results_with_torch' arg "

                                               f"must be one of the following keywords: "

                                               f"{', '.join(LEARNING_SCALE_NAMES)}")

                raise AutodiffCompositionError(f"'OPTIMIZATION_STEP can't be used with 'synch_results_with_torch';"

                                               f"use another value of {', '.arg_vals}")

                raise AutodiffCompositionError(f"Value of `retain_torch_trained_outputs` arg "

                                               f"must be one of the following keywords: "

                                               f"{', '.join(LEARNING_SCALE_NAMES)}")

                raise AutodiffCompositionError(f"Value of `retain_torch_targets` arg "

                                               f"must be one of the following keywords: "

                                               f"{', '.join(LEARNING_SCALE_NAMES)}")

                raise AutodiffCompositionError(f"Value of `retain_torch_losses` arg "

                                               f"must be one of the following keywords: "

                                               f"{', '.join(LEARNING_SCALE_NAMES)}")

    # TODO (CW 9/28/18): add compositions to registry so default arg for name is no longer needed

                 pathways=None,

                 optimizer_type='sgd',

                 loss_spec=Loss.MSE,

                 weight_decay=0,

                 learning_rate=None,

                 optimizer_params:dict=None,

                 disable_learning=False,

                 force_no_retain_graph=False,

                 refresh_losses=False,

                 synch_projection_matrices_with_torch:Optional[str]=RUN,

                 synch_node_variables_with_torch:Optional[str]=None,

                 synch_node_values_with_torch:Optional[str]=RUN,

                 synch_results_with_torch:Optional[str]=RUN,

                 retain_torch_trained_outputs:Optional[str]=MINIBATCH,

                 retain_torch_targets:Optional[str]=MINIBATCH,

                 retain_torch_losses:Optional[str]=MINIBATCH,

                 device=None,

                 disable_cuda=True,

                 cuda_index=None,

                 name="autodiff_composition",

                 **kwargs):

        # if not torch_available:

        #     raise AutodiffCompositionError('Pytorch python module (torch) is not installed. Please install it with '

        #                                    '`pip install torch` or `pip3 install torch`')

        #

            name = name,

            pathways=pathways,

            optimizer_type = optimizer_type,

            loss_spec = loss_spec,

            weight_decay = weight_decay,

            learning_rate = learning_rate,

            synch_projection_matrices_with_torch = synch_projection_matrices_with_torch,

            synch_node_variables_with_torch = synch_node_variables_with_torch,

            synch_node_values_with_torch = synch_node_values_with_torch,

            synch_results_with_torch = synch_results_with_torch,

            retain_torch_trained_outputs = retain_torch_trained_outputs,

            retain_torch_targets = retain_torch_targets,

            retain_torch_losses = retain_torch_losses,

            **kwargs)

        # keeps track of average loss per epoch

        # ordered execution sets for the pytorch model

        # # MODIFIED 7/10/24 OLD:

            else:

        else:

        # # MODIFIED 7/10/24 NEW: NEEDED FOR torch MPS SUPPORT

        #  FIX: ADD AFTER USE OF utilities.get_torch_tensor() AND COMPATIBLITY WITH MPS IS VALIDATED

        # if device is None:

        #     # Try setting device by default

        #     if not disable_cuda and torch.cuda.is_available():

        #         if cuda_index is None:

        #             self.device = torch.device(CUDA)

        #         else:

        #             self.device = torch.device('cuda:' + str(cuda_index))

        #     elif torch_available:

        #         if torch.backends.mps.is_available():

        #             from psyneulink.core.components.functions.nonstateful.transferfunctions import Linear

        #             try:

        #                 self.device = torch.device(MPS)

        #                 test_pytorch_fct_with_mps = Linear()._gen_pytorch_fct(self.device, Context())

        #             except AssertionError:

        #                 self.device = torch.device(CPU)

        #         else:

        #             self.device = torch.device(CPU)

        # else:

        #     self.device = device

        # # MODIFIED 7/10/24 END

        # Set to True after first warning about failure to specify execution mode so warning is issued only once

        # torch params added when warned in copy_projection_matrix_to_torch_param() to avoid repeats for same param

        # return self.infer_backpropagation_learning_pathways(pnlvm.ExecutionMode.PyTorch)

        # ShowGraph

        """Override to replace assignment of ShowGraph class with PytorchShowGraph if torch is available"""

        else:

        """Create backpropagation learning pathways for every Input Node --> Output Node pathway

        Flattens nested compositions:

          - only includes the Projections in outer Composition to/from the CIMs of the nested Composition

            (i.e., to input_CIMs and from output_CIMs) -- the ones that should be learned;

          - excludes Projections from/to CIMs in the nested Composition

            (from input_CIMs and to output_CIMs), as those should remain identity Projections;

          see `PytorchCompositionWrapper` for table of how Projections are handled and further details.

        Returns list of target nodes for each pathway

        """

        # Construct a pathway(s) for each INPUT Node (including BIAS Nodes), except the TARGET Node)

            # For PyTorch mode, only need to construct dummy TARGET Nodes, to allow targets to be:

            #  - specified in the same way as for other execution_modes

            #  - trial-by-trial values kept aligned with inputs in batch / minibatch construction

            #  - tracked for logging (as mechs of a Composition)

            # IMPLEMENTATION NOTE:

            #    only add target nodes if not already present

            #    (to avoid duplication in multiple calls, including from command line;

            #     see test_xor_training_identicalness_standard_composition_vs_PyTorch_and_LLVM for example)

            # output_mechs_for_learning = self.get_nested_output_nodes_at_all_levels()

            # assert set([mech for mech in [pathway[-1] for pathway in pathways]]) == set(output_mechs_for_learning)