Algorithms Mutations Registry

Parameters

class agilerl.algorithms.core.registry.RLParameter(min: float, max: float, shrink_factor: float = 0.8, grow_factor: float = 1.2, dtype: ~typing.Type[float] | ~typing.Type[int] = <class 'float'>)

Dataclass for storing the configuration of a hyperparameter that will be mutated during training. The hyperparameter is defined by a range of values that it can take, and the shrink and grow factors that will be used to mutate the hyperparameter value.

Parameters:

  • min (float) – The minimum value that the hyperparameter can take.

  • max (float) – The maximum value that the hyperparameter can take.

  • shrink_factor (float) – The factor by which the hyperparameter will be shrunk during mutation. Default is 0.8.

  • grow_factor (float) – The factor by which the hyperparameter will be grown during mutation. Default is 1.2.

  • dtype (Union[Type[float], Type[int]]) – The data type of the hyperparameter. Default is float.

  • value (Optional[Number]) – The current value of the hyperparameter. Default is None.

dtype

alias of float

mutate() → Number

Mutate the hyperparameter value by either growing or shrinking it.

Returns:

The mutated hyperparameter value.

Return type:

Number

class agilerl.algorithms.core.registry.HyperparameterConfig(**kwargs: Dict[str, RLParameter])

Stores the RL hyperparameters that will be mutated during training. For each hyperparameter, we store the name of the attribute where the hyperparameter is stored, and the range of values that the hyperparameter can take.

sample() → Tuple[str, RLParameter]

Sample a hyperparameter from the configuration.

Returns:

The name of the hyperparameter and its configuration.

Return type:

Tuple[str, RLHyperparameter]

class agilerl.algorithms.core.registry.NetworkGroup(eval: EvolvableModule, shared: EvolvableModule | List[EvolvableModule] | None = None, policy: bool = False, multiagent: bool = False)

Dataclass for storing a group of networks. This consists of an evaluation network (i.e. a network that is optimized during training) and, optionally, some other networks that share parameters with the evaluation network (e.g. the target network in DQN).

Parameters:

  • eval (str) – The evaluation network.

  • shared (str, List[str]) – The list of shared networks.

  • policy (bool) – Whether the network is a policy (e.g. the network used to get the actions of the agent). There must be one network group in an algorithm which sets this to True. Default is False.

  • multiagent (bool) – Whether the network group is used in a multiagent setting. Default is False.

class agilerl.algorithms.core.registry.MutationRegistry(hp_config: HyperparameterConfig | None = None)

Registry to keep track of the components of an algorithms that may evolve during training in a structured way to be interpreted by a Mutations object when performing evolutionary hyperparameter optimization. This includes:

  1. The hyperparameter configuration of the algorithm.

  2. The network groups of the algorithm.

  3. The optimizers of the algorithm.

  4. The mutation hooks of the algorithm (i.e. functions that are called when a mutation is performed).

Parameters:

hp_config (HyperparameterConfig) – The hyperparameter configuration of the algorithm.

all_registered() → List[str]

Returns all of the members in the registry.

networks() → List[NetworkConfig]

Get a list of network configurations in the registry.

Returns:

A list of network configurations in the registry.

Return type:

List[NetworkConfig]

property optimizer_networks: Dict[str, List[str]]

Get a dictionary of optimizer names and the network attribute names that they update.

Returns:

A dictionary of optimizer names and the network attribute names that they update.

Return type:

Dict[str, List[str]]

property policy: str | None

Get the name of the policy network in the registry.

Returns:

The name of the policy network in the registry.

Return type:

Optional[str]

register_group(group: NetworkGroup) → None

Register a network configuration in the registry.

Parameters:

config (NetworkConfig) – The network configuration to be registered.

register_hook(hook: Callable) → None

Register a hook in the registry as its name.

Parameters:

hook (Callable) – The hook to be registered.

register_optimizer(optimizer: OptimizerConfig) → None

Register an optimizer configuration in the registry.

Parameters:

config (OptimizerConfig) – The optimizer configuration to be registered.