Mutation¶

Mutation is periodically used to explore the hyperparameter space, allowing different hyperparameter combinations to be trialled during training. If certain hyperparameters prove relatively beneficial to training, then that agent is more likely to be preserved in the next generation, and so those characteristics are more likely to remain in the population.

The Mutations() class is used to mutate agents with pre-set probabilities. The available mutations currently implemented are:

No mutation
Network architecture mutation - adding layers or nodes. Trained weights are reused and new weights are initialized randomly.
Network parameters mutation - mutating weights with Gaussian noise.
Network activation layer mutation - change of activation layer.
RL algorithm mutation - mutation of learning hyperparameter, such as learning rate or batch size.

Mutations.mutation() returns a mutated population.

Tournament selection and mutation should be applied sequentially to fully evolve a population between evaluation and learning cycles.

from agilerl.hpo.mutation import Mutations
import torch

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

mutations = Mutations(
              no_mutation=0.4,                      # No mutation
              architecture=0.2,                     # Architecture mutation
              new_layer_prob=0.2,                   # New layer mutation
              parameters=0.2,                       # Network parameters mutation
              activation=0,                         # Activation layer mutation
              rl_hp=0.2,                            # Learning HP mutation
              mutation_sd=0.1,                      # Mutation strength
              rand_seed=1,                          # Random seed
              device=device
            )

Parameters¶

class agilerl.hpo.mutation.Mutations(no_mutation: float, architecture: float, new_layer_prob: float, parameters: float, activation: float, rl_hp: float, mutation_sd: float = 0.1, activation_selection: List[str] = ['ReLU', 'ELU', 'GELU'], mutate_elite: bool = True, rand_seed: int | None = None, device: str = 'cpu', accelerator: Accelerator | None = None)¶

The Mutations class for evolutionary hyperparameter optimization.

Parameters:

no_mutation (float) – Relative probability of no mutation
architecture (float) – Relative probability of architecture mutation
new_layer_prob (float) – Relative probability of new layer mutation (type of architecture mutation)
parameters (float) – Relative probability of network parameters mutation
activation (float) – Relative probability of activation layer mutation
rl_hp (float) – Relative probability of learning hyperparameter mutation
rl_hp_selection (list[str]) – Learning hyperparameter mutations to choose from
mutation_sd (float) – Mutation strength
activation_selection (list[str], optional) – Activation functions to choose from, defaults to [“ReLU”, “ELU”, “GELU”]
mutate_elite (bool, optional) – Mutate elite member of population, defaults to True
rand_seed (int, optional) – Random seed for repeatability, defaults to None
device (str, optional) – Device for accelerated computing, ‘cpu’ or ‘cuda’, defaults to ‘cpu’
accelerator (accelerate.Accelerator(), optional) – Accelerator for distributed computing, defaults to None

activation_mutation(individual: T) → T¶

Returns individual from population with activation layer mutation.

Parameters:: individual (EvolvableAlgorithm) – Individual agent from population

architecture_mutate(individual: T) → T¶

Returns individual from population with network architecture mutation, which adds either layers or nodes to different types of network architectures.

Parameters:: individual (object) – Individual agent from population

classic_parameter_mutation(network: EvolvableModule) → EvolvableModule¶

Returns network with mutated weights, with a vectorized inner loop for efficiency.

Parameters:: network (EvolvableModule) – Neural network to mutate.
Returns:: Mutated network.
Return type:: EvolvableModule

compile_modules(modules: List[EvolvableModule] | EvolvableModule, compiler: str) → List[EvolvableModule] | EvolvableModule¶

Compile the modules using the given compiler.

Parameters:

modules (List[ModuleType]) – The modules to compile
compiler (Optional[str]) – The compiler to use

get_mutations_options(pretraining: bool = False) → Tuple[List[Callable], List[float]]¶

Get the mutation options and probabilities for the given mutation configuration.

Parameters:: pretraining (bool) – Boolean flag indicating if the mutation is before the training loop
Returns:: Mutation functions and their respective relative probabilities
Return type:: Tuple[List[Callable], List[float]]

load_state_dicts(modules: List[OptimizedModule | EvolvableModule], state_dicts: List[Dict[str, Any]], remove_prefix: bool = False) → None¶

Load the state dictionary into the module.

Parameters:

module (ModuleType) – The module to load the state dictionary into
state_dict (Dict[str, Any]) – The state dictionary to load
remove_prefix (bool, optional) – Boolean flag indicating if the compile prefix should be removed, defaults to False

mutation(population: Iterable[T], pre_training_mut: bool = False) → Iterable[T]¶

Returns mutated population.

Parameters:

population (list[EvolvableAlgorithm]) – Population of agents
pre_training_mut (bool, optional) – Boolean flag indicating if the mutation is before the training loop

no_mutation(individual: T)¶

Returns individual from population without mutation.

Parameters:: individual (object) – Individual agent from population

parameter_mutation(individual: T) → T¶

Returns individual from population with network parameters mutation.

Parameters:: individual (EvolvableAlgorithm) – Individual agent from population

reinit_from_mutated(offspring: List[EvolvableModule] | EvolvableModule, remove_compile_prefix: bool = False) → List[EvolvableModule] | EvolvableModule¶

Reinitialize the mutated offspring with their state dictionary.

Parameters:: offspring (OffspringType) – The offspring to reinitialize
Returns:: The reinitialized offspring
Return type:: OffspringType

reinit_module(module: EvolvableModule, init_dict: Dict[str, Any]) → EvolvableModule¶

Reinitialize the module with the given initialization dictionary.

Parameters:

module (EvolvableModule) – The module to reinitialize
init_dict (Dict[str, Any]) – The initialization dictionary

reinit_opt(individual: T, optimizer: OptimizerConfig | None = None) → None¶

Reinitialize the optimizers of an individual.

Parameters:: individual (EvolvableAlgorithm) – The individual to reinitialize the optimizers for

rl_hyperparam_mutation(individual: T) → T¶

Returns individual from population with RL hyperparameter mutation.

Parameters:: individual (object) – Individual agent from population

to_device(offsprings: List[EvolvableModule] | EvolvableModule) → List[EvolvableModule] | EvolvableModule¶

Move offspring to device.

Parameters:: offsprings (OffspringType) – The offspring to move to device

to_device_and_set_individual(individual: T, name: str, networks: List[EvolvableModule] | EvolvableModule) → None¶

Moves networks to the device and assigns them back to the individual.

Parameters:

individual (EvolvableAlgorithm) – The individual to assign the networks to
name (str) – The name of the attribute to assign the networks to
networks (OffspringType) – The networks to move to the device