Evolvable SimBa

An evolvable version of the SimBa architecture following the paper SimBa: Simplicity Bias for Scaling Up Parameters in Deep Reinforcement Learning .

Parameters

class agilerl.modules.simba.EvolvableSimBa(*args: Any, **kwargs: Any)

Evolvable module that implements the architecture presented in ‘SimBa: Simplicity Bias for Scaling Up Parameters in Deep Reinforcement Learning’. Designed to avoid overfitting by integrating components that induce a simplicity bias, guiding models toward simple and generalizable solutions. Supports the following types of architecture mutations during training:

  • Adding or removing residual blocks

  • Adding or removing nodes from residual blocks

  • Changing the activation function between layers

  • Changing the activation function for the output layer

Paper: https://arxiv.org/abs/2410.09754

Parameters:

  • num_inputs (int) – Input layer dimension

  • num_outputs (int) – Output layer dimension

  • hidden_size (int) – Hidden layer(s) size

  • num_blocks (int) – Number of residual blocks that compose the network

  • output_activation (str, optional) – Output activation layer, defaults to None

  • scale_factor (int, optional) – Scale factor for the network, defaults to 4

  • min_blocks (int, optional) – Minimum number of residual blocks that compose the network, defaults to 1

  • max_blocks (int, optional) – Maximum number of residual blocks that compose the network, defaults to 4

  • min_mlp_nodes (int, optional) – Minimum number of nodes a layer can have within the network, defaults to 16

  • max_mlp_nodes (int, optional) – Maximum number of nodes a layer can have within the network, defaults to 500

  • device (str, optional) – Device for accelerated computing, ‘cpu’ or ‘cuda’, defaults to ‘cpu’

  • name (str, optional) – Name of the network, defaults to ‘mlp’

  • random_seed (int | None) – Random seed to use for the network. Defaults to None.

add_block() → None

Add a hidden layer to neural network. Falls back on add_node if max hidden layers reached.

add_node(numb_new_nodes: int | None = None) → dict[str, int]

Add nodes to residual blocks of the neural network.

Parameters:

numb_new_nodes (int, optional) – Number of nodes to add, defaults to None

change_activation(activation: str, output: bool = False) → None

Use ReLU activations by default for SimBa architecture; this shouldn’t be changed during training.

Returns:

Activation function

Return type:

str

forward(x: ndarray | dict[str, ndarray] | tuple[ndarray, ...] | Tensor | TensorDict | tuple[Tensor, ...] | dict[str, Tensor] | Number | list[ReasoningPrompts] | ReasoningPrompts) → Tensor

Return output of neural network.

Parameters:

x (torch.Tensor) – Neural network input

Returns:

Neural network output

Return type:

torch.Tensor

get_output_dense() → Module

Return output layer of neural network.

init_weights_gaussian(std_coeff: float = 4, output_coeff: float = 4) → None

Initialise weights of neural network using Gaussian distribution.

property net_config: dict[str, Any]

Return model configuration in dictionary.

recreate_network() → None

Recreates neural networks.

Parameters:

shrink_params (bool, optional) – Shrink parameters of neural networks, defaults to False

remove_block() → None

Remove a hidden layer from neural network. Falls back on remove_node if min hidden layers reached.

remove_node(numb_new_nodes: int | None = None) → dict[str, int]

Remove nodes from hidden layer of neural network.

Parameters:

  • hidden_layer (int, optional) – Depth of hidden layer to remove nodes from, defaults to None

  • numb_new_nodes (int, optional) – Number of nodes to remove from hidden layer, defaults to None