Evolvable Multi-Input Neural Network (Dict / Tuple Observations)

Parameters

class agilerl.modules.multi_input.EvolvableMultiInput(*args, **kwargs)

Evolvable multi-input network composed of an evolvable feature extractor for each observation key, and an optional MLP for the concatenated vector inputs. The extracted features are concatenated and passed through a final MLP to produce the output tensor.

Note

The user can inherit from this class and override the forward method to define a custom forward pass for the network, manipulating the observation dictionary as needed, and adding any non-evolvable additional layers needed for their specific problem.

Parameters:

  • observation_space (spaces.Dict or spaces.Tuple) – Dictionary or Tuple space of observations.

  • num_outputs (int) – Dimension of the output tensor.

  • channel_size (List[int]) – List of channel sizes for the convolutional layers.

  • kernel_size (List[int]) – List of kernel sizes for the convolutional layers.

  • stride_size (List[int]) – List of stride sizes for the convolutional layers.

  • latent_dim (int, optional) – Dimension of the latent space representation. Default is 16.

  • cnn_block_type (Literal["Conv2d", "Conv3d"], optional) – Type of convolutional block to use. Default is “Conv2d”.

  • sample_input (Optional[dict[str, torch.Tensor]], optional) – Sample input tensor for the CNN. Default is None.

  • vector_space_mlp (bool, optional) – Whether to use an MLP for the vector spaces. This is done by concatenating the flattened observations and passing them through an EvolvableMLP. Default is False, whereby the observations are concatenated directly to the feature encodings before the final MLP.

  • hidden_size (List[int], optional) – List of hidden sizes for the MLP. Default is None.

  • init_dicts (Dict[str, Dict[str, Any]], optional) – Dictionary of initialization dictionaries for the feature extractors. Default is {}.

  • output_activation (Optional[str], optional) – Activation function for the output layer. Default is None.

  • activation (str, optional) – Activation function for the module layers. Default is “ReLU”.

  • min_hidden_layers (int, optional) – Minimum number of hidden layers for the MLP. Default is 1.

  • max_hidden_layers (int, optional) – Maximum number of hidden layers for the MLP. Default is 3.

  • min_mlp_nodes (int, optional) – Minimum number of nodes for the MLP. Default is 64.

  • max_mlp_nodes (int, optional) – Maximum number of nodes for the MLP. Default is 1024.

  • min_cnn_hidden_layers (int, optional) – Minimum number of hidden layers for the CNN. Default is 1.

  • max_cnn_hidden_layers (int, optional) – Maximum number of hidden layers for the CNN. Default is 6.

  • min_channel_size (int, optional) – Minimum channel size for the CNN. Default is 32.

  • max_channel_size (int, optional) – Maximum channel size for the CNN. Default is 256.

  • layer_norm (bool, optional) – Whether to use layer normalization. Default is False.

  • noise_std (float, optional) – Standard deviation of the noise. Default is 0.5.

  • noisy (bool, optional) – Whether to use noisy layers. Default is False.

  • init_layers (bool, optional) – Whether to initialize the layers. Default is True.

  • device (str, optional) – Device to use for the network. Default is “cpu”.

property activation: str

Returns the activation function for the network.

Returns:

Activation function

Return type:

str

add_latent_node(numb_new_nodes: int | None = None) → Dict[str, Any]

Add a latent node to the network.

Parameters:

numb_new_nodes (int, optional) – Number of new nodes to add, defaults to None

Returns:

Configuration for adding a latent node.

Return type:

Dict[str, Any]

property base_init_dict: Dict[str, Any]

Returns dictionary of base information.

Returns:

Base information

Return type:

Dict[str, Any]

build_feature_extractor() → Dict[str, EvolvableCNN | EvolvableMLP]

Creates the feature extractor and final MLP networks.

Returns:

Dictionary of feature extractors.

Return type:

Dict[str, Union[EvolvableCNN, EvolvableMLP]]

change_activation(activation: str, output: bool = False) → None

Set the activation function for the network.

Parameters:

  • activation (str) – Activation function to use.

  • output (bool, optional) – Flag indicating whether to set the output activation function, defaults to False

Returns:

Activation function

Return type:

str

property cnn_init_dict: Dict[str, Any]

Returns dictionary of CNN information.

Returns:

CNN information

Return type:

Dict[str, Any]

forward(x: Dict[str, _SupportsArray[dtype[Any]] | _NestedSequence[_SupportsArray[dtype[Any]]] | bool | int | float | complex | str | bytes | _NestedSequence[bool | int | float | complex | str | bytes] | Tensor] | Tuple[_SupportsArray[dtype[Any]] | _NestedSequence[_SupportsArray[dtype[Any]]] | bool | int | float | complex | str | bytes | _NestedSequence[bool | int | float | complex | str | bytes] | Tensor]) → Tensor

Forward pass of the composed network. Extracts features from each observation key and concatenates them with the corresponding observation key if specified. The concatenated features are then passed through the final MLP to produce the output tensor.

Parameters:

  • x (Dict[str, ArrayOrTensor], Tuple[ArrayOrTensor]) – Dictionary of observations.

  • xc (Optional[ArrayOrTensor], optional) – Optional additional input tensor for critic network, defaults to None.

  • q (bool, optional) – Flag to indicate if Q-values should be computed, defaults to True.

Returns:

Output tensor.

Return type:

torch.Tensor

get_inner_init_dict(key: str, default: Literal['cnn', 'mlp']) → Dict[str, Any]

Returns the initialization dictionary for the specified key.

Arguments:

key (str): Key of the observation space.

Returns:

Dict[str, Any]: Initialization dictionary.

property init_dicts: Dict[str, Dict[str, Any]]

Returns the initialization dictionaries for the network.

Returns:

Initialization dictionaries

Return type:

Dict[str, Dict[str, Any]]

init_weights_gaussian(std_coeff: float = 4, output_coeff: float = 4) → None

Initialise weights of linear layers using Gaussian distribution.

property mlp_init_dict: Dict[str, Any]

Returns dictionary of MLP information.

Returns:

MLP information

Return type:

Dict[str, Any]

property net_config: Dict[str, Any]

Returns the configuration of the network.

Returns:

Network configuration

Return type:

Dict[str, Any]

recreate_network() → None

Recreates the network with the new latent dimension.

remove_latent_node(numb_new_nodes: int | None = None) → Dict[str, Any]

Remove a latent node from the network.

Parameters:

numb_new_nodes (int, optional) – Number of nodes to remove, defaults to None

Returns:

Configuration for removing a latent node.

Return type:

Dict[str, Any]