Evolvable Networks Utils

Model Compilation

agilerl.utils.evolvable_networks.compile_model(model: Module | ModuleDict[EvolvableModule], mode: str | None = 'default') → OptimizedModule | ModuleDict[EvolvableModule]

Compiles torch model if not already compiled.

Parameters:
Returns:

compiled model

Return type:

OptimizedModule | ModuleDict[OptimizedModule]

Space Utilities

agilerl.utils.evolvable_networks.is_image_space(space: Space) → bool

Check if the space is an image space. We ignore dtype and number of channels checks.

Parameters:

space (spaces.Space) – Input space

Returns:

True if the space is an image space, False otherwise

Return type:

bool

agilerl.utils.evolvable_networks.is_box_space_ndim(space: Space, ndim: int) → bool

Check if the space is a Box space with the given number of dimensions.

Parameters:

  • space (spaces.Space) – Input space

  • ndim (int) – Number of dimensions

Returns:

True if the space is a Box space with the given number of dimensions, False otherwise

agilerl.utils.evolvable_networks.is_vector_space(space: Space) → bool

Check if the space is a vector space.

Parameters:

space (spaces.Space) – Input space

Returns:

True if the space is a vector space, False otherwise

Return type:

bool

agilerl.utils.evolvable_networks.tuple_to_dict_space(tuple_space: Tuple) → Dict

Convert a Tuple observation space to a Dict observation space.

Parameters:

tuple_space (spaces.Tuple) – Tuple observation space

Returns:

Dictionary observation space

Return type:

spaces.Dict

agilerl.utils.evolvable_networks.tuple_to_dict_obs(tuple_obs: tuple) → dict

Convert a tuple observation to a Python dictionary.

Parameters:

tuple_obs (tuple) – Tuple observation

Returns:

Dictionary observation

Return type:

dict

Configuration Utilities

agilerl.utils.evolvable_networks.config_from_dict(config_dict: dict[str, dict[str, Any] | Any]) → NetConfig

Get the class of the net config from the dictionary.

Parameters:

config_dict (NetConfigType) – The dictionary to get the class from.

Returns:

The net config class.

Return type:

NetConfig

agilerl.utils.evolvable_networks.get_default_encoder_config(observation_space: Space, simba: bool = False, recurrent: bool = False, layer_norm: bool = True) → dict[str, dict[str, Any] | Any]

Get the default configuration for the encoder network based on the observation space.

Parameters:

  • observation_space (spaces.Space) – Observation space of the environment.

  • simba (bool) – Whether to use SimBA encoder.

  • recurrent (bool) – Whether to use recurrent encoder.

  • layer_norm (bool) – Whether to use layer normalization in MLP encoders.

Returns:

Default configuration for the encoder network.

Return type:

NetConfigType

Module Inspection

agilerl.utils.evolvable_networks.contains_moduledict(module: Module) → bool

Check if a module contains a ModuleDict.

Parameters:

module (nn.Module) – Input module

Returns:

True if module contains a ModuleDict, False otherwise

Return type:

bool

agilerl.utils.evolvable_networks.get_module_dict(module: Module) → ModuleDict

Get the ModuleDict from a module.

Parameters:

module (nn.Module) – Input module

Returns:

ModuleDict from module

Return type:

dict[str, nn.Module]

Layer Factory Functions

agilerl.utils.evolvable_networks.get_batch_norm_layer(name: str, num_features: int, device: str | device = 'cpu') → Module

Return batch normalization layer for corresponding batch normalization name.

Parameters:

  • name (str) – Batch normalization layer name

  • layer_size (int) – The layer after which the batch normalization layer will be applied

Returns:

Batch normalization layer

Return type:

nn.Module

agilerl.utils.evolvable_networks.get_conv_layer(conv_layer_name: Literal['Conv2d', 'Conv3d'], in_channels: int, out_channels: int, kernel_size: tuple[int, ...] | int, stride: tuple[int, ...] | int = 1, padding: tuple[int, ...] | int = 0, device: str | device = 'cpu') → Module

Return convolutional layer for corresponding convolutional layer name.

Parameters:

  • conv_layer_name (str) – Convolutional layer name

  • in_channels (int) – Number of input channels to convolutional layer

  • out_channels (int) – Number of output channels from convolutional layer

  • kernel_size (int or tuple[int]) – Kernel size of convolutional layer

  • stride (int or tuple[int]) – Stride size of convolutional layer

  • padding (int or tuple[int]) – Convolutional layer padding

Returns:

Convolutional layer

Return type:

nn.Module

agilerl.utils.evolvable_networks.get_normalization(normalization_name: str, layer_size: int, device: str | device = 'cpu') → Module

Return normalization layer for corresponding normalization name.

Parameters:

  • normalization_names (str) – Normalization layer name

  • layer_size – The layer after which the normalization layer will be applied

  • layer_size – int

Returns:

Normalization layer

Return type:

nn.Module

agilerl.utils.evolvable_networks.get_activation(activation_name: str | None, new_gelu: bool = False) → Module

Return activation function for corresponding activation name.

Parameters:

activation_names (str) – Activation function name

agilerl.utils.evolvable_networks.get_pooling(pooling_name: str, kernel_size: tuple[int, ...] | int, stride: tuple[int, ...] | int, padding: tuple[int, ...] | int) → Module

Return pooling layer for corresponding activation name.

Parameters:

  • pooling_names (str) – Pooling layer name

  • kernel_size (int or tuple[int]) – Pooling layer kernel size

  • stride (int or tuple[int]) – Pooling layer stride

  • padding (int or tuple[int]) – Pooling layer padding

Returns:

Pooling layer

Return type:

nn.Module

Weight Initialization

agilerl.utils.evolvable_networks.layer_init(layer: Module | GumbelSoftmax | NoisyLinear, std: float = np.float64(1.4142135623730951), bias_const: float = 0.0) → Module

Initialize the weights and biases of a layer.

Parameters:

  • layer (nn.Module) – The layer to initialize.

  • std (float, optional) – The standard deviation for weight initialization. Defaults to np.sqrt(2).

  • bias_const (float, optional) – The constant value for bias initialization. Defaults to 0.0.

Returns:

The initialized layer.

Return type:

nn.Module

agilerl.utils.evolvable_networks.init_weights_gaussian(m: Module, mean: float, std: float) → None

Initialize weights of a module using Gaussian distribution.

Parameters:

  • m (nn.Module) – Module to initialize

  • mean (float) – Mean of the Gaussian distribution

  • std (float) – Standard deviation of the Gaussian distribution

Network Creation

agilerl.utils.evolvable_networks.create_cnn(block_type: Literal['Conv2d', 'Conv3d'], in_channels: int, channel_size: list[int], kernel_size: list[tuple[int, ...] | int], stride_size: list[tuple[int, ...] | int], name: str = 'cnn', init_layers: bool = True, layer_norm: bool = False, activation_fn: str = 'ReLU', device: str | device = 'cpu') → dict[str, Module]

Build a convolutional block.

Parameters:

  • block_type (Literal["Conv2d", "Conv3d"]) – Type of convolutional block.

  • in_channels (int) – Number of input channels.

  • channel_size (list[int]) – List of channel sizes for each layer.

  • kernel_size (list[int]) – List of kernel sizes for each layer.

  • stride_size (list[int]) – List of stride sizes for each layer.

  • name (str) – Name of the block.

  • init_layers (bool, optional) – Whether to initialize the layers. Defaults to True.

  • layer_norm (bool, optional) – Whether to use layer normalization. Defaults to False.

  • activation_fn (str, optional) – Activation function to use. Defaults to “ReLU”.

  • device (DeviceType, optional) – Device to use. Defaults to “cpu”.

Returns:

Convolutional block.

Return type:

dict[str, nn.Module]

agilerl.utils.evolvable_networks.create_mlp(input_size: int, output_size: int, hidden_size: list[int], output_vanish: bool, output_activation: str | None = None, noisy: bool = False, init_layers: bool = True, layer_norm: bool = False, output_layernorm: bool = False, activation: str = 'ReLU', noise_std: float = 0.1, device: str | device = 'cpu', new_gelu: bool = False, name: str = 'mlp') → Sequential

Create and returns multi-layer perceptron.

Parameters:

  • input_size (int) – Number of input features.

  • output_size (int) – Number of output features.

  • hidden_size (list[int]) – List of hidden layer sizes.

  • output_vanish (bool) – Whether to initialize output layer weights to a small value.

  • output_activation (str | None) – Activation function for output layer.

  • noisy (bool, optional) – Whether to use noisy layers.

  • init_layers (bool, optional) – Whether to initialize the layers.

  • layer_norm (bool, optional) – Whether to use layer normalization.

  • output_layernorm (bool, optional) – Whether to use layer normalization for the output layer.

  • activation (str, optional) – Activation function for hidden layers.

  • noise_std (float, optional) – Standard deviation of noise for noisy layers.

  • name (str, default "mlp") – Name of the network.

Returns:

Multi-layer perceptron.

Return type:

nn.Sequential

agilerl.utils.evolvable_networks.create_simba(input_size: int, output_size: int, hidden_size: int, num_blocks: int, output_activation: str | None = None, scale_factor: float = 4.0, device: str | device = 'cpu', name: str = 'simba') → Sequential

Create a number of SimBa residual blocks.

Paper: https://arxiv.org/abs/2410.09754.

Parameters:

  • input_size (int) – Number of input features.

  • output_size (int) – Number of output features.

  • hidden_size (int) – Number of hidden units.

  • num_blocks (int) – Number of residual blocks.

  • output_activation (str | None) – Activation function for output layer.

  • scale_factor (float, optional) – Scale factor for the hidden layer.

  • device (DeviceType, optional) – Device to use. Defaults to “cpu”.

  • name (str, default "simba") – Name of the network.

Returns:

Residual block.

Return type:

nn.Sequential

agilerl.utils.evolvable_networks.create_resnet(input_channels: int, channel_size: int, kernel_size: int, stride_size: int, num_blocks: int, scale_factor: int = 4, device: str = 'cpu', name: str = 'resnet') → dict[str, Module]

Create a number of residual blocks for image-based inputs.

Paper: https://arxiv.org/abs/1512.03385.

Parameters:

  • input_channels (int) – Number of input channels.

  • channel_size (int) – Number of output channels.

  • kernel_size (int) – Kernel size of the convolutional layer.

  • stride_size (int) – Stride size of the convolutional layer.

  • num_blocks (int) – Number of residual blocks.

  • scale_factor (int, optional) – Scale factor for the hidden layer.

  • device (DeviceType, optional) – Device to use. Defaults to “cpu”.

  • name (str, default "resnet") – Name of the network.

Returns:

Residual block.

Return type:

nn.Sequential