EvolvableModule¶

Parameters¶

class agilerl.modules.base.EvolvableModule(*args: Any, **kwargs: Any)¶

Base class for evolvable neural networks. Inheriting from this class allows us to dynamically keep track of the available mutation methods of the network and its nested evolvable modules. During initialization, registered mutation methods are wrapped to use a context manager that automatically calls the recreate_network method of the network after the outermost mutation method has been called. This avoids redundant recreations of the network when multiple mutation methods are applied in sequence.

Parameters:

device (str) – The device to run the network on.
random_seed (int | None) – The random seed to use for the network. Determines the random number generator stored in self.rng that can be used to sample random numbers in e.g. mutation methods.

change_activation(activation: str, output: bool) → None¶

Set the activation function for the network.

Parameters:

activation (str) – Activation function to use.
output (bool) – Whether to set the activation function for the output layer.

clone() → Self¶

Return clone of an EvolvableModule with identical parameters.

Returns:: A clone of the EvolvableModule.
Return type:: SelfEvolvableModule

disable_mutations(mut_type: MutationType | None = None) → None¶

Disable all or some mutation methods from the evolvable module. It recursively disables the mutation methods of nested evolvable modules as well.

Parameters:: mut_type (MutationType | None) – The type of mutation method to disable.

filter_mutation_methods(remove: str) → None¶

Filter out mutation methods that contain the specified string in their name.

Parameters:: remove (str) – The string to remove.

forward(*args: Any, **kwargs: Any) → Tensor¶

Define the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

get_init_dict() → dict[str, Any]¶

Get the dictionary of constructor arguments for the network.

Returns:: The dictionary of constructor arguments.
Return type:: dict[str, Any]

get_mutation_methods() → dict[str, MutationMethodProtocol]¶

Get all mutation methods for the network as dictionary of method names to mutation methods.

Returns:: A dictionary of mutation methods.
Return type:: dict[str, MutationMethodProtocol]

get_mutation_probs(new_layer_prob: float) → list[float]¶

Get the mutation probabilities for each mutation method.

Parameters:: new_layer_prob (float) – The probability of selecting a layer mutation method.

return: A list of probabilities for each mutation method. rtype: list[float]

get_output_dense() → Module | None¶

Get the output dense layer of the network.

Returns:: The output dense layer.
Return type:: nn.Module

static init_weights_gaussian(module: Module, std_coeff: float) → None¶

Initialize the weights of the neural network using a Gaussian distribution.

Parameters:

module (nn.Module) – The neural network module.
std_coeff (float) – The standard deviation coefficient.

modules() → dict[str, EvolvableModule]¶

Return the nested evolvable modules in the network.

Warning

This overrides the behavior of nn.Module.modules() and only returns the evolvable modules. If you need the torch modules, use torch_modules() instead.

Returns:: A dictionary of network attributes.
Return type:: dict[str, EvolvableModule]

static preserve_parameters(old_net: Module, new_net: Module) → Module¶

Return new neural network with copied parameters from old network. Specifically, it handles tensors with different sizes by copying the minimum number of elements.

Parameters:

old_net (nn.Module) – Old neural network
new_net (nn.Module) – New neural network

Returns:

New neural network with copied parameters

Return type:

nn.Module

recreate_network(**kwargs) → None¶

Recreate the network after a mutation has been applied. If the mutation methods of an EvolvableModule are only attributed to its nested modules, then the recreate_network method should be implemented in the nested modules and it is not required on the parent.

Parameters:: kwargs (dict[str, Any]) – Keyword arguments to pass to the network constructor.

register_mutation_hook(hook: Callable) → None¶

Register a hook to be called after a mutation has been applied to a nested evolvable module. The hook function should not take any arguments.

Parameters:: hook (Callable) – The hook function.

reset_noise() → None¶: Reset noise for all NoisyLinear layers in the network.

sample_mutation_method(new_layer_prob: float, rng: Generator | None = None) → MutationMethodProtocol¶

Sample a mutation method based on the mutation probabilities.

Parameters:: new_layer_prob – The probability of selecting a layer mutation method.

type new_layer_prob: float :param rng: The random number generator. type rng: Generator | None return: The sampled mutation method. rtype: MutationMethodProtocol

torch_modules() → Iterator[Module]¶

Return an iterator over the `nn.Module`s in the network.

Returns:: An iterator over the `nn.Module`s in the network.
Return type:: Iterator[nn.Module]

EvolvableWrapper¶

Parameters¶

class agilerl.modules.base.EvolvableWrapper(*args: Any, **kwargs: Any)¶

Wrapper class for evolvable neural networks. Can be used to provide some additional functionality to an EvolvableModule while maintaining its mutation methods at the top-level.

Parameters:: module (EvolvableModule) – The evolvable module to wrap.

change_activation(activation: str, output: bool) → None¶

Change the activation function for the network.

Parameters:

activation (str) – The activation function to use.
output (bool) – Whether to set the activation function for the output layer.

ModuleDict¶

Parameters¶

class agilerl.modules.base.ModuleDict(*args: Any, **kwargs: Any)¶

Analogous to nn.ModuleDict, but allows for the inheritance of the mutation methods of nested evolvable modules.

Parameters:

modules (dict[str, EvolvableModule] | None) – The modules to add to the dictionary.
device (str) – The device to use for the modules.

change_activation(activation: str, output: bool) → None¶

Change the activation function for the network.

Parameters:

activation (str) – The activation function to use.
output (bool) – Whether to set the activation function for the output layer.

clone() → ModuleDict¶

Return clone of an ModuleDict with identical parameters.

Returns:: A clone of the ModuleDict.
Return type:: ModuleDict

filter_mutation_methods(remove: str) → None¶

Filter out mutation methods that contain the specified string in their name.

param remove: The string to remove. type remove: str

get_mutation_methods() → dict[str, MutationMethodProtocol]¶

Get all mutation methods for the network.

Returns:: A dictionary of mutation methods.
Return type:: dict[str, MutationMethodProtocol]

items() → Iterable[tuple[str, ModuleType]]¶: Return an iterable of the ModuleDict key/value pairs.

modules() → dict[str, EvolvableModule]¶

Return the nested evolvable modules in the network.

Warning

This overrides the behavior of nn.Module.modules() and only returns the evolvable modules. If you need the torch modules, use torch_modules() instead.

Returns:: A dictionary of network attributes.
Return type:: dict[str, Any]

values() → Iterable[ModuleType]¶: Return an iterable of the ModuleDict values.

Mutation Decorator¶

agilerl.modules.base.mutation(mutation_type: MutationType, **recreate_kwargs) → Callable[[Callable], MutationMethodProtocol]¶

Register a method as a mutation function of a specific type (decorator). This signals that the module should be recreated after the function has been called on the module.

Parameters:: mutation_type (MutationType) – The type of mutation function.
Returns:: The decorator function.
Return type:: Callable[[Callable], MutationMethodProtocol]