Evolvable Multi-layer Perceptron (MLP)

Parameters

class agilerl.networks.evolvable_mlp.EvolvableMLP(num_inputs: int, num_outputs: int, hidden_size: List[int], feature_hidden_size=[128], num_atoms=51, mlp_activation='ReLU', mlp_output_activation=None, min_hidden_layers=1, max_hidden_layers=3, min_mlp_nodes=64, max_mlp_nodes=500, layer_norm=True, output_vanish=True, init_layers=True, support=None, rainbow=False, device='cpu', accelerator=None, arch='mlp')

The Evolvable Multi-layer Perceptron class.

Parameters:

  • num_inputs (int) – Input layer dimension

  • num_outputs (int) – Output layer dimension

  • hidden_size (list[int]) – Hidden layer(s) size

  • feature_hidden_size (list[int]) – Hidden size for the feature network when using Rainbow DQN, defaults to [128]

  • num_atoms (int, optional) – Number of atoms for Rainbow DQN, defaults to 51

  • mlp_activation (str, optional) – Activation layer, defaults to ‘relu’

  • mlp_output_activation (str, optional) – Output activation layer, defaults to None

  • min_hidden_layers (int, optional) – Minimum number of hidden layers the network will shrink down to, defaults to 1

  • max_hidden_layers (int, optional) – Maximum number of hidden layers the network will expand to, defaults to 3

  • min_mlp_nodes (int, optional) – Minimum number of nodes a layer can have within the network, defaults to 64

  • max_mlp_nodes (int, optional) – Maximum number of nodes a layer can have within the network, defaults to 500

  • layer_norm (bool, optional) – Normalization between layers, defaults to True

  • output_vanish (bool, optional) – Vanish output by multiplying by 0.1, defaults to True

  • init_layers (bool, optional) – Initialise network layers, defaults to True

  • support (torch.Tensor(), optional) – Atoms support tensor, defaults to None

  • rainbow (bool, optional) – Using Rainbow DQN, defaults to False

  • device (str, optional) – Device for accelerated computing, ‘cpu’ or ‘cuda’, defaults to ‘cpu’

  • accelerator (accelerate.Accelerator(), optional) – Accelerator for distributed computing, defaults to None

add_mlp_layer()

Adds a hidden layer to neural network.

add_mlp_node(hidden_layer=None, numb_new_nodes=None)

Adds nodes to hidden layer of neural network.

Parameters:

  • hidden_layer (int, optional) – Depth of hidden layer to add nodes to, defaults to None

  • numb_new_nodes (int, optional) – Number of nodes to add to hidden layer, defaults to None

clone()

Returns clone of neural net with identical parameters.

create_mlp(input_size, output_size, hidden_size, output_vanish, output_activation, noisy=False, rainbow_feature_net=False)

Creates and returns multi-layer perceptron.

create_net()

Creates and returns neural network.

forward(x, q=True)

Returns output of neural network.

Parameters:

  • x (torch.Tensor() or np.array) – Neural network input

  • q (bool, optional) – Return Q value if using rainbow, defaults to True

get_activation(activation_names)

Returns activation function for corresponding activation name.

Parameters:

activation_names (str) – Activation function name

property init_dict

Returns model information in dictionary.

preserve_parameters(old_net, new_net)

Returns new neural network with copied parameters from old network.

Parameters:

  • old_net (nn.Module()) – Old neural network

  • new_net (nn.Module()) – New neural network

recreate_nets(shrink_params=False)

Recreates neural networks.

remove_mlp_layer()

Removes a hidden layer from neural network.

remove_mlp_node(hidden_layer=None, numb_new_nodes=None)

Removes nodes from hidden layer of neural network.

Parameters:

  • hidden_layer (int, optional) – Depth of hidden layer to remove nodes from, defaults to None

  • numb_new_nodes (int, optional) – Number of nodes to remove from hidden layer, defaults to None

reset_noise()

Resets noise of value and advantage networks.

shrink_preserve_parameters(old_net, new_net)

Returns shrunk new neural network with copied parameters from old network.

Parameters:

  • old_net (nn.Module()) – Old neural network

  • new_net (nn.Module()) – New neural network