Make Evolvable

Parameters

class agilerl.wrappers.make_evolvable.MakeEvolvable(*args: Any, **kwargs: Any)

Wrapper to make a neural network evolvable.

Parameters:

  • network (nn.Module) – Input neural network

  • input_tensor (torch.Tensor) – Example input tensor so forward pass can be made to detect the network architecture

  • num_atoms (int, optional) – Number of atoms for Rainbow DQN, defaults to 51

  • secondary_input_tensor (torch.Tensor, optional) – Second input tensor if network performs forward pass with two tensors, for example, off-policy algorithms that use a critic(s) with environments that have RGB image observations and thus require CNN architecture, defaults to None

  • min_hidden_layers (int, optional) – Minimum number of hidden layers the fully connected layer will shrink down to, defaults to 1

  • max_hidden_layers (int, optional) – Maximum number of hidden layers the fully connected layer will expand to, defaults to 3

  • min_mlp_nodes (int, optional) – Minimum number of nodes a layer can have within the fully connected layer, defaults to 64

  • max_mlp_nodes (int, optional) – Maximum number of nodes a layer can have within the fully connected layer, defaults to 1024

  • min_cnn_hidden_layers (int, optional) – Minimum number of hidden layers the convolutional layer will shrink down to, defaults to 1

  • max_cnn_hidden_layers (int, optional) – Maximum number of hidden layers the convolutional layer will expand to, defaults to 6

  • min_channel_size (int, optional) – Minimum number of channels a convolutional layer can have, defaults to 32

  • max_channel_size (int, optional) – Maximum number of channels a convolutional layer can have, defaults to 256

  • output_vanish (bool, optional) – Vanish output by multiplying by 0.1, defaults to False

  • init_layers (bool, optional) – Initialise network layers, defaults to False

  • support (torch.Tensor(), optional) – Atoms support tensor, defaults to None

  • rainbow (bool, optional) – Using Rainbow DQN, defaults to False

  • device (str, optional) – Device for accelerated computing, ‘cpu’ or ‘cuda’, defaults to ‘cpu’

  • accelerator (accelerate.Accelerator(), optional) – Accelerator for distributed computing, defaults to None

property activation: str

Return the activation function.

add_cnn_channel(hidden_layer: int | None = None, numb_new_channels: int | None = None) → dict[str, int]

Add channel to hidden layer of Convolutional Neural Network.

Parameters:

  • hidden_layer (int, optional) – Depth of hidden layer to add channel to, defaults to None

  • numb_new_channels (int, optional) – Number of channels to add to hidden layer, defaults to None

add_cnn_layer() → None

Add a hidden layer to convolutional neural network.

add_mlp_layer() → None

Add a hidden layer to value network.

add_mlp_node(hidden_layer: int | None = None, numb_new_nodes: int | None = None) → dict[str, int]

Add nodes to hidden layer of value network.

Parameters:

  • hidden_layer (int, optional) – Depth of hidden layer to add nodes to, defaults to None

  • numb_new_nodes (int, optional) – Number of nodes to add to hidden layer, defaults to None

build_networks() → tuple[Module, Module, Module | None]

Create and returns the feature and value net.

calc_max_kernel_sizes() → list[tuple[int, int]]

Calculate the max kernel size for each convolutional layer of the feature net.

calc_stride_size_ranges() → list[tuple[int, int]]

Calculate a range of stride sizes for each convolutional layer of the feature net.

change_activation(activation: str, output: bool = False) → None

Set the activation function for the network.

Parameters:

  • activation (str) – Activation function to use.

  • output (bool, optional) – Flag indicating whether to set the output activation function, defaults to False

Returns:

Activation function

Return type:

str

change_cnn_kernel() → None

Randomly alters convolution kernel of random CNN layer.

create_cnn(input_size: int, channel_size: list[int], kernel_size: list[int], stride_size: list[int], padding: list[int], name: str) → Sequential

Create and returns convolutional neural network.

Parameters:
create_mlp(input_size: int, output_size: int, hidden_size: list[int], name: str, mlp_activation: str, mlp_output_activation: str | None, noisy: bool = False, rainbow_feature_net: bool = False) → Sequential

Create and returns multi-layer perceptron.

Parameters:

  • input_size (int) – Input dimensions to first MLP layer

  • output_size (int) – Output dimensions from last MLP layer

  • hidden_size (list[int]) – Hidden layer sizes

  • name (str) – Layer name

  • mlp_activation (str) – Activation function for hidden layers

  • mlp_output_activation (str | None) – Activation function for output layer

  • noisy (bool) – Whether to use NoisyLinear layers

  • rainbow_feature_net (bool) – Whether this is a Rainbow DQN feature network

detect_architecture(network: Module, input_tensor: Tensor, secondary_input_tensor: Tensor | None = None) → None

Detect the architecture of a neural network.

Parameters:

  • network (nn.Module) – Neural network whose architecture is being detected

  • input_tensor (torch.Tensor) – Tensor used to perform forward pass to detect layers

  • secondary_input_tensor (torch.Tensor, optional) – Second tensor used to perform forward pass if forward method of neural network takes two tensors as arguments, defaults to None

forward(x: ndarray | Tensor, xc: ndarray | Tensor | None = None, q: bool = True) → Tensor

Return output of neural network.

Parameters:

  • x (torch.Tensor() or np.array) – Neural network input

  • xc (torch.Tensor() or np.array, optional) – Actions to be evaluated by critic, defaults to None

  • q (bool, optional) – Return Q value if using rainbow, defaults to True

get_init_dict() → dict[str, Any]

Return model information in dictionary.

get_output_dense() → Module

Return the output dense layer.

init_weights_gaussian(std_coeff: float = 4.0, output_coeff: float = 2.0) → None

Initialise network weights using Gaussian distribution.

Parameters:

  • std_coeff (float, optional) – Standard deviation coefficient, defaults to 4.0

  • output_coeff (float, optional) – Output coefficient, defaults to 2.0

property output_activation: str

Return the output activation function.

recreate_network(shrink_params: bool = False) → None

Recreates neural networks.

Parameters:

shrink_params (bool) – Boolean flag to shrink parameters

remove_cnn_channel(hidden_layer: int | None = None, numb_new_channels: int | None = None) → dict[str, int]

Remove channel from hidden layer of convolutional neural network.

Parameters:

  • hidden_layer (int, optional) – Depth of hidden layer to add channel to, defaults to None

  • numb_new_channels (int, optional) – Number of channels to add to hidden layer, defaults to None

remove_cnn_layer() → None

Remove a hidden layer from the convolutional neural network.

remove_mlp_layer() → None

Remove a hidden layer from value network.

remove_mlp_node(hidden_layer: int | None = None, numb_new_nodes: int | None = None) → dict[str, int]

Remove nodes from hidden layer of neural network.

Parameters:

  • hidden_layer (int, optional) – Depth of hidden layer to remove nodes from, defaults to None

  • numb_new_nodes (int, optional) – Number of nodes to remove from hidden layer, defaults to None