Evolvable Multi-layer Perceptron (MLP)

Parameters

class agilerl.modules.mlp.EvolvableMLP(*args, **kwargs)

The Evolvable Multi-layer Perceptron class.

Parameters:

  • num_inputs (int) – Input layer dimension

  • num_outputs (int) – Output layer dimension

  • hidden_size (List[int]) – Hidden layer(s) size

  • activation (str, optional) – Activation layer, defaults to ‘ReLU’

  • output_activation (str, optional) – Output activation layer, defaults to None

  • min_hidden_layers (int, optional) – Minimum number of hidden layers the network will shrink down to, defaults to 1

  • max_hidden_layers (int, optional) – Maximum number of hidden layers the network will expand to, defaults to 3

  • min_mlp_nodes (int, optional) – Minimum number of nodes a layer can have within the network, defaults to 64

  • max_mlp_nodes (int, optional) – Maximum number of nodes a layer can have within the network, defaults to 500

  • layer_norm (bool, optional) – Normalization between layers, defaults to True

  • output_layernorm (bool, optional) – Normalization for the output layer, defaults to False

  • output_vanish (bool, optional) – Vanish output by multiplying by 0.1, defaults to True

  • init_layers (bool, optional) – Initialise network layers, defaults to True

  • noise_std (float, optional) – Noise standard deviation, defaults to 0.5

  • noisy (bool, optional) – Add noise to network, defaults to False

  • new_gelu (bool, optional) – Use new GELU activation function, defaults to False

  • device (str, optional) – Device for accelerated computing, ‘cpu’ or ‘cuda’, defaults to ‘cpu’

  • name (str, optional) – Name of the network, defaults to ‘mlp’

property activation: str

Returns activation function.

add_layer() → None

Adds a hidden layer to neural network. Falls back on add_node() if max hidden layers reached.

add_node(hidden_layer: int | None = None, numb_new_nodes: int | None = None) → Dict[str, int]

Adds nodes to hidden layer of neural network.

Parameters:

  • hidden_layer (int, optional) – Depth of hidden layer to add nodes to, defaults to None

  • numb_new_nodes (int, optional) – Number of nodes to add to hidden layer, defaults to None

change_activation(activation: str, output: bool = False) → None

Set the activation function for the network.

Parameters:

  • activation (str) – Activation function to use.

  • output (bool, optional) – Flag indicating whether to set the output activation function, defaults to False

forward(x: _SupportsArray[dtype[Any]] | _NestedSequence[_SupportsArray[dtype[Any]]] | bool | int | float | complex | str | bytes | _NestedSequence[bool | int | float | complex | str | bytes] | Tensor) → Tensor

Returns output of neural network.

Parameters:

x (torch.Tensor()) – Neural network input

Returns:

Neural network output

Return type:

torch.Tensor

get_output_dense() → Module

Returns output layer of neural network.

init_weights_gaussian(std_coeff: float = 4, output_coeff: float = 4) → None

Initialise weights of neural network using Gaussian distribution.

property net_config: Dict[str, Any]

Returns model configuration in dictionary.

recreate_network() → None

Recreates neural networks.

Parameters:

shrink_params (bool, optional) – Shrink parameters of neural networks, defaults to False

remove_layer() → None

Removes a hidden layer from neural network. Falls back on add_node() if min hidden layers reached.

remove_node(hidden_layer: int | None = None, numb_new_nodes: int | None = None) → Dict[str, int]

Removes nodes from hidden layer of neural network.

Parameters:

  • hidden_layer (int, optional) – Depth of hidden layer to remove nodes from, defaults to None

  • numb_new_nodes (int, optional) – Number of nodes to remove from hidden layer, defaults to None