Network Specifications

Pydantic models that configure the encoder and head architecture for evolvable networks. The encoder_config field uses a discriminated union on the arch literal so YAML manifests can select the encoder type with a single key.

Encoder Specs

class agilerl.models.networks.MlpSpec(*, hidden_size: Annotated[list[int], MinLen(min_length=1)], activation: Literal['Tanh', 'GELU', 'ReLU', 'ELU', 'LeakyReLU', 'Sigmoid', 'Softplus', 'Softmax', 'GumbelSoftmax', 'PReLU', 'Identity'] = 'ReLU', output_activation: Literal['Tanh', 'GELU', 'ReLU', 'ELU', 'LeakyReLU', 'Sigmoid', 'Softplus', 'Softmax', 'GumbelSoftmax', 'PReLU', 'Identity'] | None = None, layer_norm: bool = True, init_layers: bool = True, output_vanish: bool = True, min_hidden_layers: Annotated[int, Gt(gt=0)] = 1, max_hidden_layers: Annotated[int, Gt(gt=1)] = 6, min_mlp_nodes: Annotated[int, Gt(gt=0)] = 8, max_mlp_nodes: Annotated[int, Gt(gt=1)] = 256, arch: Literal['mlp'] = 'mlp')

Model specification for evolvable MLP networks.

model_config: ClassVar[ConfigDict] = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_dump(**kwargs: Any) → dict[str, Any]

Dump the model to a dictionary. Remove the output_activation field if it is None.

Parameters:

kwargs (Any) – Keyword arguments to pass to the parent method.

Returns:

The model as a dictionary.

Return type:

dict[str, Any]

class agilerl.models.networks.CnnSpec(*, channel_size: Annotated[list[int], MinLen(min_length=1)], kernel_size: Annotated[list[int], MinLen(min_length=1)], stride_size: Annotated[list[int], MinLen(min_length=1)], min_hidden_layers: Annotated[int, Gt(gt=0)] = 1, max_hidden_layers: Annotated[int, Gt(gt=1)] = 6, min_channel_size: Annotated[int, Gt(gt=0)] = 8, max_channel_size: Annotated[int, Gt(gt=1)] = 256, layer_norm: bool = False, init_layers: bool = True, activation: Literal['Tanh', 'GELU', 'ReLU', 'ELU', 'LeakyReLU', 'Sigmoid', 'Softplus', 'Softmax', 'GumbelSoftmax', 'PReLU', 'Identity'] = 'ReLU', arch: Literal['cnn'] = 'cnn')

Model specification for evolvable convolutional networks.

model_config: ClassVar[ConfigDict] = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class agilerl.models.networks.LstmSpec(*, hidden_state_size: Annotated[int, Gt(gt=0)], num_layers: Annotated[int, Gt(gt=0)] = 1, min_hidden_state_size: Annotated[int, Gt(gt=0)] = 8, max_hidden_state_size: Annotated[int, Gt(gt=1)] = 256, min_layers: Annotated[int, Ge(ge=0)] = 1, max_layers: Annotated[int, Ge(ge=1)] = 6, dropout: Annotated[float, Ge(ge=0.0), Le(le=1.0)] = 0.0, arch: Literal['lstm'] = 'lstm')

Model specification for evolvable LSTM networks.

model_config: ClassVar[ConfigDict] = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class agilerl.models.networks.SimbaSpec(*, hidden_size: Annotated[int, Gt(gt=0)], num_blocks: Annotated[int, Gt(gt=0)], output_activation: Literal['Tanh', 'GELU', 'ReLU', 'ELU', 'LeakyReLU', 'Sigmoid', 'Softplus', 'Softmax', 'GumbelSoftmax', 'PReLU', 'Identity'] | None = None, scale_factor: Annotated[int, Gt(gt=0)] = 4, min_blocks: Annotated[int, Gt(gt=0)] = 1, max_blocks: Annotated[int, Gt(gt=1)] = 4, min_mlp_nodes: Annotated[int, Gt(gt=0)] = 8, max_mlp_nodes: Annotated[int, Gt(gt=1)] = 256, arch: Literal['simba'] = 'simba')

Model specification for evolvable SimBA networks.

Reference: https://arxiv.org/abs/2410.09754

model_config: ClassVar[ConfigDict] = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class agilerl.models.networks.MultiInputSpec(*, latent_dim: Annotated[int, Gt(gt=0)] = 32, vector_space_mlp: bool = False, min_latent_dim: Annotated[int, Gt(gt=0)] = 8, max_latent_dim: Annotated[int, Gt(gt=1)] = 128, mlp_config: MlpSpec | None = None, cnn_config: CnnSpec | None = None, arch: Literal['multiinput'] = 'multiinput')

Model specification for evolvable multi-input networks.

model_config: ClassVar[ConfigDict] = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

Network Specs

class agilerl.models.networks.NetworkSpec(*, latent_dim: Annotated[int, Gt(gt=0)] = 32, min_latent_dim: Annotated[int, Gt(gt=0)] = 8, max_latent_dim: Annotated[int, Gt(gt=1)] = 128, encoder_config: MlpSpec | CnnSpec | LstmSpec | MultiInputSpec | SimbaSpec, head_config: MlpSpec, random_seed: int | None = None, simba: bool = False)

Base model specification for nested AgileRL evolvable networks.

model_config: ClassVar[ConfigDict] = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class agilerl.models.networks.QNetworkSpec(*, latent_dim: Annotated[int, Gt(gt=0)] = 32, min_latent_dim: Annotated[int, Gt(gt=0)] = 8, max_latent_dim: Annotated[int, Gt(gt=1)] = 128, encoder_config: MlpSpec | CnnSpec | LstmSpec | MultiInputSpec | SimbaSpec, head_config: MlpSpec, random_seed: int | None = None, simba: bool = False)

Model specification for evolvable Q networks.

model_config: ClassVar[ConfigDict] = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class agilerl.models.networks.ContinuousQNetworkSpec(*, latent_dim: Annotated[int, Gt(gt=0)] = 32, min_latent_dim: Annotated[int, Gt(gt=0)] = 8, max_latent_dim: Annotated[int, Gt(gt=1)] = 128, encoder_config: MlpSpec | CnnSpec | LstmSpec | MultiInputSpec | SimbaSpec, head_config: MlpSpec, random_seed: int | None = None, simba: bool = False)

Model specification for evolvable continuous Q networks.

model_config: ClassVar[ConfigDict] = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class agilerl.models.networks.DeterministicActorSpec(*, latent_dim: Annotated[int, Gt(gt=0)] = 32, min_latent_dim: Annotated[int, Gt(gt=0)] = 8, max_latent_dim: Annotated[int, Gt(gt=1)] = 128, encoder_config: MlpSpec | CnnSpec | LstmSpec | MultiInputSpec | SimbaSpec, head_config: MlpSpec, random_seed: int | None = None, simba: bool = False)

Model specification for evolvable deterministic actor networks.

model_config: ClassVar[ConfigDict] = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class agilerl.models.networks.StochasticActorSpec(*, latent_dim: Annotated[int, Gt(gt=0)] = 32, min_latent_dim: Annotated[int, Gt(gt=0)] = 8, max_latent_dim: Annotated[int, Gt(gt=1)] = 128, encoder_config: MlpSpec | CnnSpec | LstmSpec | MultiInputSpec | SimbaSpec, head_config: MlpSpec, random_seed: int | None = None, simba: bool = False, squash_output: bool = False)

Model specification for evolvable stochastic actor networks.

model_config: ClassVar[ConfigDict] = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class agilerl.models.networks.ValueNetworkSpec(*, latent_dim: Annotated[int, Gt(gt=0)] = 32, min_latent_dim: Annotated[int, Gt(gt=0)] = 8, max_latent_dim: Annotated[int, Gt(gt=1)] = 128, encoder_config: MlpSpec | CnnSpec | LstmSpec | MultiInputSpec | SimbaSpec, head_config: MlpSpec, random_seed: int | None = None, simba: bool = False)

Model specification for evolvable value networks.

model_config: ClassVar[ConfigDict] = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class agilerl.models.networks.RainbowQNetworkSpec(*, latent_dim: Annotated[int, Gt(gt=0)] = 32, min_latent_dim: Annotated[int, Gt(gt=0)] = 8, max_latent_dim: Annotated[int, Gt(gt=1)] = 128, encoder_config: MlpSpec | CnnSpec | LstmSpec | MultiInputSpec | SimbaSpec, head_config: MlpSpec, random_seed: int | None = None, simba: bool = False)

Model specification for evolvable Rainbow Q networks.

model_config: ClassVar[ConfigDict] = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

LLM Finetuning Specs

Used by the network section of manifests for LLM algorithms (GRPO, DPO, etc.) instead of NetworkSpec.

class agilerl.models.networks.LoraConfigDict(*, lora_r: Annotated[int, Ge(ge=1)] = 16, lora_alpha: Annotated[int, Ge(ge=1)] = 32, target_modules: list[str] | str | set[str] = 'all-linear', task_type: Annotated[str, MinLen(min_length=1)] = 'CAUSAL_LM', lora_dropout: Annotated[float, Ge(ge=0.0), Le(le=1.0)] = 0.05)

Model specification for LLM LoRA configuration.

model_config: ClassVar[ConfigDict] = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class agilerl.models.networks.FinetuningNetworkSpec(*, pretrained_model_name_or_path: Annotated[str, MinLen(min_length=1)], max_context_length: Annotated[int, Ge(ge=1)], lora_config: LoraConfigDict | None = None)

Model specification for LLM finetuning networks.

model_config: ClassVar[ConfigDict] = {}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].