Evolvable SimBa

An evolvable version of the SimBa architecture following the paper SimBa: Simplicity Bias for Scaling Up Parameters in Deep Reinforcement Learning .

Parameters

class agilerl.modules.simba.EvolvableSimBa(*args, **kwargs)

Evolvable module that implements the architecture presented in ‘SimBa: Simplicity Bias for Scaling Up Parameters in Deep Reinforcement Learning’. Designed to avoid overfitting by integrating components that induce a simplicity bias, guiding models toward simple and generalizable solutions.

Paper: https://arxiv.org/abs/2410.09754

Parameters:

  • num_inputs (int) – Input layer dimension

  • num_outputs (int) – Output layer dimension

  • hidden_size (int) – Hidden layer(s) size

  • num_blocks (int) – Number of residual blocks that compose the network

  • output_activation (str, optional) – Output activation layer, defaults to None

  • scale_factor (int, optional) – Scale factor for the network, defaults to 4

  • min_blocks (int, optional) – Minimum number of residual blocks that compose the network, defaults to 1

  • max_blocks (int, optional) – Maximum number of residual blocks that compose the network, defaults to 4

  • min_mlp_nodes (int, optional) – Minimum number of nodes a layer can have within the network, defaults to 16

  • max_mlp_nodes (int, optional) – Maximum number of nodes a layer can have within the network, defaults to 500

  • device (str, optional) – Device for accelerated computing, ‘cpu’ or ‘cuda’, defaults to ‘cpu’

  • name (str, optional) – Name of the network, defaults to ‘mlp’

add_block() → None

Adds a hidden layer to neural network. Falls back on add_node if max hidden layers reached.

add_node(numb_new_nodes: int | None = None) → Dict[str, int]

Adds nodes to residual blocks of the neural network.

Parameters:

numb_new_nodes (int, optional) – Number of nodes to add, defaults to None

change_activation(activation: str, output: bool = False) → None

The SimBa architecture uses ReLU activations by default and this shouldn’t be changed during training.

Returns:

Activation function

Return type:

str

forward(x: ndarray | Dict[str, ndarray] | Tuple[_SupportsArray[dtype[Any]] | _NestedSequence[_SupportsArray[dtype[Any]]] | bool | int | float | complex | str | bytes | _NestedSequence[bool | int | float | complex | str | bytes], ...] | Tensor | Dict[str, Tensor] | Tuple[Tensor, ...]) → Tensor

Returns output of neural network.

Parameters:

x (torch.Tensor) – Neural network input

Returns:

Neural network output

Return type:

torch.Tensor

get_output_dense() → Module

Returns output layer of neural network.

init_weights_gaussian(std_coeff: float = 4, output_coeff: float = 4) → None

Initialise weights of neural network using Gaussian distribution.

property net_config: Dict[str, Any]

Returns model configuration in dictionary.

recreate_network() → None

Recreates neural networks.

Parameters:

shrink_params (bool, optional) – Shrink parameters of neural networks, defaults to False

remove_block() → None

Removes a hidden layer from neural network. Falls back on remove_node if min hidden layers reached.

remove_node(numb_new_nodes: int | None = None) → Dict[str, int]

Removes nodes from hidden layer of neural network.

Parameters:

  • hidden_layer (int, optional) – Depth of hidden layer to remove nodes from, defaults to None

  • numb_new_nodes (int, optional) – Number of nodes to remove from hidden layer, defaults to None