Evolvable Convolutional Neural Network (CNN)

Parameters

class agilerl.networks.evolvable_cnn.EvolvableCNN(input_shape: List[int], channel_size: List[int], kernel_size: List[int], stride_size: List[int], hidden_size: List[int], num_actions: int, num_atoms=51, mlp_output_activation=None, mlp_activation='ReLU', cnn_activation='ReLU', min_hidden_layers=1, max_hidden_layers=3, min_mlp_nodes=64, max_mlp_nodes=1024, min_cnn_hidden_layers=1, max_cnn_hidden_layers=6, min_channel_size=32, max_channel_size=256, n_agents=None, multi=False, layer_norm=False, support=None, rainbow=False, critic=False, normalize=True, init_layers=True, device='cpu', accelerator=None, arch='cnn')

The Evolvable Convolutional Neural Network class.

Parameters:

  • input_shape (list[int]) – Input shape

  • channel_size (list[int]) – CNN channel size

  • kernel_size (list[int]) – Comvolution kernel size

  • stride_size (list[int]) – Convolution stride size

  • hidden_size (list[int]) – Hidden layer(s) size

  • num_actions (int) – Action dimension

  • num_atoms (int, optional) – Number of atoms for Rainbow DQN, defaults to 51

  • mlp_output_activation (str, optional) – MLP output activation layer, defaults to None

  • mlp_activation (str, optional) – MLP activation layer, defaults to ‘relu’

  • cnn_activation (str, optional) – CNN activation layer, defaults to ‘relu’

  • min_hidden_layers (int, optional) – Minimum number of hidden layers the fully connected layer will shrink down to, defaults to 1

  • max_hidden_layers (int, optional) – Maximum number of hidden layers the fully connected layer will expand to, defaults to 3

  • min_mlp_nodes (int, optional) – Minimum number of nodes a layer can have within the fully connected layer, defaults to 64

  • max_mlp_nodes (int, optional) – Maximum number of nodes a layer can have within the fully connected layer, defaults to 1024

  • min_cnn_hidden_layers (int, optional) – Minimum number of hidden layers the convolutional layer will shrink down to, defaults to 1

  • max_cnn_hidden_layers (int, optional) – Maximum number of hidden layers the convolutional layer will expand to, defaults to 6

  • min_channel_size (int, optional) – Minimum number of channels a convolutional layer can have, defaults to 32

  • max_channel_size (int, optional) – Maximum number of channels a convolutional layer can have, defaults to 256

  • n_agents (int, optional) – Number of agents, defaults to None

  • multi (bool, optional) – Boolean flag to indicate if this is a multi-agent problem, defaults to False

  • layer_norm (bool, optional) – Normalization between layers, defaults to False

  • support (torch.Tensor(), optional) – Atoms support tensor, defaults to None

  • rainbow (bool, optional) – Using Rainbow DQN, defaults to False

  • critic (bool, optional) – CNN is a critic network, defaults to False

  • normalize (bool, optional) – Normalize CNN inputs, defaults to True

  • init_layers (bool, optional) – Initialise network layers, defaults to True

  • device (str, optional) – Device for accelerated computing, ‘cpu’ or ‘cuda’, defaults to ‘cpu’

  • accelerator (accelerate.Accelerator(), optional) – Accelerator for distributed computing, defaults to None

add_cnn_channel(hidden_layer=None, numb_new_channels=None)

Adds channel to hidden layer of convolutional neural network.

Parameters:

  • hidden_layer (int, optional) – Depth of hidden layer to add channel to, defaults to None

  • numb_new_channels (int, optional) – Number of channels to add to hidden layer, defaults to None

add_cnn_layer()

Adds a hidden layer to convolutional neural network.

add_mlp_layer()

Adds a hidden layer to fully connected layer.

add_mlp_node(hidden_layer=None, numb_new_nodes=None)

Adds nodes to hidden layer of Multi-layer Perceptron.

Parameters:

  • hidden_layer (int, optional) – Depth of hidden layer to add nodes to, defaults to None

  • numb_new_nodes (int, optional) – Number of nodes to add to hidden layer, defaults to None

calc_max_kernel_sizes(channel_size, kernel_size, stride_size, input_shape)

Calculates the max kernel size for each convolutional layer of the feature net.

calc_stride_size_ranges()

Calculates a range of stride sizes for each convolutional layer of the feature net.

change_cnn_kernel()

Randomly alters convolution kernel of random CNN layer.

clone()

Returns clone of neural net with identical parameters.

create_cnn(input_size, channel_size, kernel_size, stride_size, name, features_dim=None)

Creates and returns convolutional neural network.

create_mlp(input_size, output_size, hidden_size, name, output_activation, noisy=False)

Creates and returns multi-layer perceptron.

create_nets()

Creates and returns neural networks.

forward(x, xc=None, q=True)

Returns output of neural network.

Parameters:

  • x (torch.Tensor() or np.array) – Neural network input

  • xc (torch.Tensor() or np.array, optional) – Actions to be evaluated by critic, defaults to None

  • q (bool, optional) – Return Q value if using rainbow, defaults to True

get_activation(activation_names)

Returns activation function for corresponding activation name.

Parameters:

activation_names (str) – Activation function name

property init_dict

Returns model information in dictionary.

preserve_parameters(old_net, new_net)

Returns new neural network with copied parameters from old network.

Parameters:

  • old_net (nn.Module()) – Old neural network

  • new_net (nn.Module()) – New neural network

recreate_nets(shrink_params=False)

Recreates neural networks.

remove_cnn_channel(hidden_layer=None, numb_new_channels=None)

Remove channel from hidden layer of convolutional neural network.

Parameters:

  • hidden_layer (int, optional) – Depth of hidden layer to add channel to, defaults to None

  • numb_new_channels (int, optional) – Number of channels to add to hidden layer, defaults to None

remove_cnn_layer()

Removes a hidden layer from convolutional neural network.

remove_mlp_layer()

Removes a hidden layer from fully connected layer.

remove_mlp_node(hidden_layer=None, numb_new_nodes=None)

Removes nodes from hidden layer of fully connected layer. :param hidden_layer: Depth of hidden layer to remove nodes from, defaults to None :type hidden_layer: int, optional :param numb_new_nodes: Number of nodes to remove from hidden layer, defaults to None :type numb_new_nodes: int, optional

reset_noise()

Resets noise of value and advantage networks.

shrink_preserve_parameters(old_net, new_net)

Returns shrunk new neural network with copied parameters from old network.

Parameters:

  • old_net (nn.Module()) – Old neural network

  • new_net (nn.Module()) – New neural network

class agilerl.networks.evolvable_cnn.NoisyLinear(in_features, out_features, std_init=0.5)

The Noisy Linear Neural Network class.

Parameters:

  • in_features (int) – Input features size

  • out_features (int) – Output features size

  • std_init (float, optional) – Standard deviation, defaults to 0.5

forward(x)

Returns output of neural network.

Parameters:

x (torch.Tensor()) – Neural network input

reset_noise()

Resets neural network noise.

reset_parameters()

Resets neural network parameters.