Make Evolvable¶
Parameters¶
- class agilerl.wrappers.make_evolvable.MakeEvolvable(network, input_tensor, secondary_input_tensor=None, num_atoms=51, min_hidden_layers=1, max_hidden_layers=3, min_mlp_nodes=64, max_mlp_nodes=1024, min_cnn_hidden_layers=1, max_cnn_hidden_layers=6, min_channel_size=32, max_channel_size=256, output_vanish=False, init_layers=False, support=None, rainbow=False, device='cpu', accelerator=None, **kwargs)¶
Wrapper to make a neural network evolvable
- Parameters:
network (nn.Module) – Input neural network
input_tensor (torch.Tensor) – Example input tensor so forward pass can be made to detect the network architecture
num_atoms (int, optional) – Number of atoms for Rainbow DQN, defaults to 51
secondary_input_tensor (torch.Tensor, optional) – Second input tensor if network performs forward pass with two tensors, for example, off-policy algorithms that use a critic(s) with environments that have RGB image observations and thus require CNN architecture, defaults to None
min_hidden_layers (int, optional) – Minimum number of hidden layers the fully connected layer will shrink down to, defaults to 1
max_hidden_layers (int, optional) – Maximum number of hidden layers the fully connected layer will expand to, defaults to 3
min_mlp_nodes (int, optional) – Minimum number of nodes a layer can have within the fully connected layer, defaults to 64
max_mlp_nodes (int, optional) – Maximum number of nodes a layer can have within the fully connected layer, defaults to 1024
min_cnn_hidden_layers (int, optional) – Minimum number of hidden layers the convolutional layer will shrink down to, defaults to 1
max_cnn_hidden_layers (int, optional) – Maximum number of hidden layers the convolutional layer will expand to, defaults to 6
min_channel_size (int, optional) – Minimum number of channels a convolutional layer can have, defaults to 32
max_channel_size (int, optional) – Maximum number of channels a convolutional layer can have, defaults to 256
output_vanish (bool, optional) – Vanish output by multiplying by 0.1, defaults to False
init_layers (bool, optional) – Initialise network layers, defaults to False
support (torch.Tensor(), optional) – Atoms support tensor, defaults to None
rainbow (bool, optional) – Using Rainbow DQN, defaults to False
device (str, optional) – Device for accelerated computing, ‘cpu’ or ‘cuda’, defaults to ‘cpu’
accelerator (accelerate.Accelerator(), optional) – Accelerator for distributed computing, defaults to None
- add_cnn_channel(hidden_layer=None, numb_new_channels=None)¶
Adds channel to hidden layer of Convolutional Neural Network.
- add_cnn_layer()¶
Adds a hidden layer to convolutional neural network.
- add_mlp_layer()¶
Adds a hidden layer to value network.
- add_mlp_node(hidden_layer=None, numb_new_nodes=None)¶
Adds nodes to hidden layer of value network.
- calc_max_kernel_sizes()¶
Calculates the max kernel size for each convolutional layer of the feature net.
- calc_stride_size_ranges()¶
Calculates a range of stride sizes for each convolutional layer of the feature net.
- change_cnn_kernel()¶
Randomly alters convolution kernel of random CNN layer.
- clone()¶
Returns clone of neural net with identical parameters.
- create_cnn(input_size, channel_size, kernel_size, stride_size, padding, name)¶
Creates and returns convolutional neural network.
- create_mlp(input_size, output_size, hidden_size, name, mlp_activation, mlp_output_activation, noisy=False, rainbow_feature_net=False)¶
Creates and returns multi-layer perceptron.
- Parameters:
- Parameters:
rainbow_feature_net
- create_nets()¶
Creates and returns the feature and value net.
- detect_architecture(network, input_tensor, secondary_input_tensor=None)¶
Detect the architecture of a neural network.
- Parameters:
network (nn.Module) – Neural network whose architecture is being detected
input_tensor (torch.Tensor) – Tensor used to perform forward pass to detect layers
secondary_input_tensor – Second tensor used to perform forward pass if forward
method of neural network takes two tensors as arguments, defaults to None :type secondary_input_tensor: torch.Tensor, optional
- forward(x, xc=None, q=True)¶
Returns output of neural network.
- Parameters:
x (torch.Tensor() or np.array) – Neural network input
xc (torch.Tensor() or np.array, optional) – Actions to be evaluated by critic, defaults to None
q (bool, optional) – Return Q value if using rainbow, defaults to True
- get_activation(activation_names)¶
Returns activation function for corresponding activation name.
- Parameters:
activation_names (str) – Activation function name
- get_conv_layer(conv_layer_name, in_channels, out_channels, kernel_size, stride, padding)¶
Return convolutional layer for corresponding convolutional layer name.
- Parameters:
conv_layer_name (str) – Convolutional layer name
in_channels (int) – Number of input channels to convolutional layer
out_channels (int) – Number of output channels from convolutional layer
kernel_size (int or Tuple[int]) – Kernel size of convolutional layer
stride (int or Tuple[int]) – Stride size of convolutional layer
- get_normalization(normalization_name, layer_size)¶
Returns normalization layer for corresponding normalization name.
- Parameters:
normalization_names (str) – Normalization layer name
layer_size – The layer after which the normalization layer will be applied
layer_size – int
- get_pooling(pooling_names, kernel_size, stride, padding)¶
Returns pooling layer for corresponding activation name.
- property init_dict¶
Returns model information in dictionary.
- layer_init(layer, std=1.4142135623730951, bias_const=0.0)¶
Initialize the weights of a neural network layer using orthogonal initialization and set the biases to a constant value.
- preserve_parameters(old_net, new_net)¶
Returns new neural network with copied parameters from old network.
- Parameters:
old_net (nn.Module()) – Old neural network
new_net (nn.Module()) – New neural network
- recreate_nets(shrink_params=False)¶
Recreates neural networks.
- Parameters:
shrink_params (bool) – Boolean flag to shrink parameters
- remove_cnn_channel(hidden_layer=None, numb_new_channels=None)¶
Remove channel from hidden layer of convolutional neural network.
- remove_cnn_layer()¶
Removes a hidden layer from the convolutional neural network.
- remove_mlp_layer()¶
Removes a hidden layer from value network.
- remove_mlp_node(hidden_layer=None, numb_new_nodes=None)¶
Removes nodes from hidden layer of neural network.
- reset_noise()¶
Resets noise of value and advantage networks.
- shrink_preserve_parameters(old_net, new_net)¶
Returns shrunk new neural network with copied parameters from old network.
- Parameters:
old_net (nn.Module()) – Old neural network
new_net (nn.Module()) – New neural network