Releases

v2.59.0: train vision-language checkpoints with their own architecture

Released on 2026-10-28 - GitHub - PyPI

Features

• support Qwen3.5 models in agilerl-arena, deprecate Qwen3

  Adds Qwen/Qwen3.5-0.8B, Qwen/Qwen3.5-2B, Qwen/Qwen3.5-4B, and Qwen/Qwen3.5-9B to SUPPORTED_MODEL_INFO with their LoRA targets, allowed ranks, and bundled config.json. Marks Qwen/Qwen3-1.7B and Qwen/Qwen3-4B deprecated. The memory estimator now counts the fp32 query upcast from Qwen3.5 attention QK norms. The Qwen3.5 language-tower weight mapper also covers the adapter-form language_model infix vLLM sees when loading LoRA, so text adapters sync onto tower engines.

• pin library workarounds to the releases they were checked against

  Add agilerl.utils.patching.library_shim, a decorator that records which release range of vLLM, transformers, peft, liger-kernel or torch a workaround was checked against. Calling a decorated shim with an installed release outside that range raises UnsupportedLibraryVersionError, naming the shim, the range and the installed version. Shims are listed in agilerl.utils.patching.LIBRARY_SHIMS. The vLLM, transformers, peft, Liger and torch workarounds in agilerl.architectures, agilerl.lora.fused, agilerl.distributed.fsdp_blocks and the colocated vLLM helpers now carry their ranges.

• expose expert-parallel support and routed expert count

  ModelInfo reports supports_expert_parallel and num_routed_experts. The flag is true when the model has packed expert modules. The count is read from the language config (n_routed_experts, num_experts, or num_local_experts) and is null when the model has no packed experts.

• load CLI plugins from the agilerl.arena.cli entry-point group

  Installed packages can add commands to the arena CLI by declaring an entry point in the agilerl.arena.cli group that names a callable taking the root click group, e.g. [project.entry-points."agilerl.arena.cli"] my = "pkg.cli:register". With no plugins installed the CLI is unchanged.

• price tensor parallel, expert parallel and NCCL process groups, and tighten LLM training and vLLM engine estimates

  The agilerl-arena memory estimator now sizes fsdp.tp and fsdp.ep. Under tensor parallel an untied lm_head sits at 1/tp outside the FSDP gather; families with a training tensor-parallel plan (nemotron_h) also shard their decoder linears and per-layer activations. Under expert parallel each rank keeps its share of the routed experts and their LoRA, each MoE backward adds token-exchange buffers, and tensor-parallel ranks route a share of the tokens each. NCCL memory is charged per process group, by size and by whether the group spans nodes.

  Training estimates now cover HF's unfused Mamba path on compute capability 8.9 GPUs, hybrids that run one mixer per layer (Granite 4), gpt-oss's expert intermediates, DPO's Liger loss, and Liger above token level. vLLM engine estimates keep the start-up profile's peak, add a compiling engine's per-layer embedding copy, size hybrid KV capacity by vLLM's block pool and SSM state dtype, and hold sliding-window KV for the scheduler's in-flight tokens. The H100 80GB catalogue entry uses its CUDA-visible 79.11 GiB. DeviceSpec.nccl_bytes, NCCL_BYTES_BY_DEVICE and NCCL_BYTES_DEFAULT are replaced by per-group NCCL bytes.

Fixes

• do not save the routed EP counts copy in packed-expert block checkpoints

  Selective checkpointing on packed-expert blocks saves every device-to-host copy. Routed expert-parallel counts skip that copy on recompute, so the next host copy (grouped-matmul fallback .tolist(), or actor/critic adapter ids) replayed a stale counts tensor and crashed with split_with_sizes(): found element of type list at pos 0.

  The routed counts copy is now flagged and not saved, so the saved copies stay in order and the recompute adds no host syncs. This hits expert-parallel training with gradient checkpointing on packed-expert blocks when the grouped-matmul fallback runs (GPUs other than compute capability 9/10, non-bf16, or unaligned sizes) or with actor/critic rows. Peak GPU memory is unchanged.

• resolve omitted training and generation settings in one place

  Omitted training and generation settings are chosen by one function each. The estimate and training call the same function. A set value is kept.

  An omitted mini-batch is the per-rank trajectory count, max(batch_size // data_parallel_size, 1) times group_size. GRPO-family uses the spec group size. Other algorithms use 1.

  An omitted micro_batch_size_per_gpu is the largest divisor of that mini-batch whose training estimate fits. On a CUDA training build, an omitted value is that search, including a single GPU. The search uses the launcher world size. Before the process group exists, the process rank is the launcher RANK. With no CUDA device, or a model outside the catalog, the omitted micro-batch is that mini-batch.

  An unset LLMPPO fuse_actor_critic_pass fuses when the fused training peak fits the device, and fuses when there is no device budget.

  An unset FSDP chunk is the largest that fits on a device. With no device it is the smallest chunk, and an unset optimizer sits on the CPU unless cpu_offload is set.

  An omitted fused-logprob chunk is the row count of a 256 MiB fp32 logit tile, clamped to 128..4096. An explicit count stays.

  An omitted max_num_seqs is 8 when generation shares the trainer GPU. Otherwise it is the prompts one engine receives times the group size. auto counts as one engine.

  An async engine max_lora_rank snaps up to the smallest accepted rank (8, 16, 32, 64, 128, 256, 320, 512) that covers the adapter. Colocated generation keeps max(configured, adapter rank) and does not snap.

  An omitted gpu_memory_utilization is 0.3. An omitted max_lora_rank starts at 16.

  An omitted LLMPPO fuse bit and an omitted FSDP chunk are priced on 95% of the card, including job overhead.

• point scoped-mismatch error at causal-LM scope fix

  Scoped LoRA errors now show example module keys present on the model alongside the empty in-scope sample, and suggest leaving the scope unset for causal LM layouts under consecutive model.layers path segments.

• lock flash-linear-attention on the llm extra

  The published lock now includes flash-linear-attention 0.5.2 for the Linux llm and all extras, matching the dependency already declared for those extras. Tests treat that pin as part of the llm extra, not cpu-llm, and cover GPU GatedDeltaNet setup when both fused kernels resolve.

• start vLLM engines for Granite 4 hybrid models on vLLM 0.32

  vLLM 0.32 reads Granite 4 hybrid layer types under the names transformers uses, so the catalog no longer renames them. Engines for granitemoehybrid checkpoints such as ibm-granite/granite-4.0-micro failed to start with KeyError: 'attention'. Removes granitemoehybrid_vllm_hf_override from agilerl.architectures.catalog.

• MoE routers under activation checkpointing, vision models, FSDP2, rollout logprobs and memory estimates

  • MoE: top-k router renormalization runs out of place for every catalog MoE family that renormalizes in place (Qwen3-MoE, Qwen3-Next, Qwen3-Omni, Qwen3.5/3.6-MoE, Qwen3-VL-MoE, Gemma 4, Qwen2-MoE, OLMoE, FlexOlmo, Mellum, Qwen4Exp, Mixtral, MiniMax). Selective activation checkpointing of MoE blocks no longer fails in backward.
  • FSDP2: units that have not run forward keep gradient sync on during gradient accumulation, which avoids a torch 2.13 post-backward crash (for example a frozen vision tower in text-only training). wrap_every_n_blocks > 1 keeps checkpoint parameter names. Gemma 4 RoPE buffers are built per layer type.
  • Qwen3.5/3.6 gated delta net: seq_idx is computed once in the text-model forward, which fixes a crash at micro-batch 1 under checkpointing. Multi-row packed batches keep document boundaries. The expert-parallel count exchange runs outside dispatch modes. Class-patched LigerExperts run split expert LoRA with SiLU.
  • Logprobs: the final-logit softcap (Gemma 4) is applied in fused logprobs and in the Liger losses that support it; GRPO-family objectives fall back from Liger with a warning otherwise.
  • Rollout: RolloutSampling carries every vLLM sampling field (top_k, min_p, penalties, min_tokens). vLLM engines return processed logprobs, so importance ratios match the trainer when temperature is not 1. With top_k > 0 and temperature > 0, rollouts record each token's sampling support and GRPO, LLMPPO and REINFORCE normalize policy, old and reference logprobs over it, so top_p, min_p and answer grammars keep the importance ratio exact; Arena manifests that use those filters without that support are rejected. Turns made only of forced tokens keep their reward; PPO/REINFORCE micro-batches with no action tokens give zero loss instead of NaN; REINFORCE turn KL averages over action tokens.
  • Vision: each model's own image placeholder, per-image vision rows for Qwen3.5/3.6 and Gemma 4, a tower lockstep so ranks without images don't hang, a frozen-tower feature cache per learn step, vLLM-served tower LoRA for Qwen3.5 and Gemma 4 (skipped on towers the engine doesn't build), and pixel staging through pinned buffers. Qwen3.5, Qwen3.6, Qwen3.8 and Gemma 4 E4B are marked vision-capable.
  • Envs: RemoteEnvClient.reset and the in-process client raise when the reset reports an error. Hosted reset images are sent as base64 PNG. The harness restarts the context at the model budget when restarts are on. New repeat_system_prompt setting.
  • vLLM compile cache directories are keyed by the engine's GPU ids (apply_vllm_compile_cache_env).
  • agilerl-arena manifests: env specs take modality (text / vision) and read-only host_volumes under /var/cache/agilerl-. Text envs strip vLLM vision towers by default when the model has towers and no tower LoRA; image envs keep them. Every rollout manifest without vllm_config gets the default engine. stop_sequences with async rollout is rejected at validation.
  • agilerl-arena memory estimator: CUDA-visible card sizes, vLLM LoRA stacks, encoder profile and non-torch memory in the KV budget, engine layout from the run's GPUs (--num-gpus), expert-LoRA backward chunks, gated delta net, held FSDP grads, Gemma 4 norms, FSDP slack, and vision host and GPU terms.
  • Breaking: agilerl.arena.models.default_colocated_vllm_config is renamed to default_rollout_vllm_config, with no alias.

• Qwen3.5 reasoning parser, padded GatedDeltaNet forward, GRPO Liger memory sizing

  Qwen3.5 architecture families now set the qwen3 vLLM reasoning parser in their runtime config. vLLM needs a reasoning parser to enforce thinking_token_budget, so rollouts that set a thinking budget on a Qwen3.5 model failed when the engine started.

  packed_layout now starts a document only at each row start and where position_ids is 0, the rule FlashAttention varlen uses. A padded batch passed with position_ids and no attention mask (pads at position 1) used to give every pad token its own document. On Qwen3.5/3.6 that sent thousands of sequences to the FLA varlen scan, whose launch fails with Triton Error [CUDA]: invalid argument once sequences x value heads exceeds 65535. Padded rows now stay one document each; packed rows are unchanged.

  The Arena memory estimator no longer refuses GSPO or trajectory-level GRPO runs with use_liger_loss=True when GRPO runs them on the standard path, which it does whenever the vLLM sampling correction or either off-policy mask is on. It still refuses the fused trajectory-level case, which is not memory-bounded.

• reduce split actor/critic FSDP gradients in parameter dtype

  Non-fused actor/critic training runs a policy backward and a value backward over different parameters. Deferred gradient sync keeps the first pass's gradients in reduce_dtype until the optimizer step. The second pass's gradients stay in param_dtype. FSDP reduce-scatter then rejects the unit when those dtypes differ.

  Split passes that defer gradient sync now reduce in param_dtype, so both passes use one dtype. Fused passes, single-pass runs, and configs that already match are left as configured.

• FSDP DPO and SFT fill unset memory settings from the spec

  Unset FSDP chunk size and optimizer placement for DPO and SFT are filled from the algorithm spec's context length before the algorithm is built. Constructing either on a GPU with those fields unset raises ValueError.

• save router top-k for checkpointed gpt-oss blocks

  Activation checkpointing saves each MoE router's top-k so backward replays the forward routing. Blocks with the transposed packed-expert layout (gpt-oss) were not detected, so their router was recomputed in backward. A token that switches expert then changes the expert loop shapes and raises CheckpointError, or silently uses different routing when shapes happen to match.

  Router saving now covers every packed-expert layout.

• charge saved router top-k in the memory estimator

  FSDP activation checkpointing saves each MoE router's top-k scores and expert indices for backward, so recompute replays the forward routing. The memory estimator charges those tensors as saved_router_topk when FSDP is on: tokens x top_k x MoE layers, at fp32 scores plus int64 indices. They stay on the GPU under activation offload. Flat data parallel recomputes the router, so it is not charged.

Breaking Changes

• train vision-language checkpoints with their own architecture

  Vision-language checkpoints (Qwen3.5/3.6, Gemma 3/4, Nemotron omni) now train with their full multimodal class, with frozen towers, so merged exports keep the original architecture, config.json and vision weights. The trainer class per family lives in the architecture catalog (TrainerRuntimeConfig.model_class, trainer_model_class).

  LoRA target_modules (named or all-linear) always land on the language model of a vision-language model. The trainer memory estimate classifies that list after the language-model regex rewrite, so attention-only adapters are not sized as all-linear. Tower or connector modules are adapted only when named in the new lora_config.tower_target_modules, which requires the towers to be served. The model catalog lists each vision-language checkpoint's language-model prefix and default text-only LoRA modules.

  strip_multimodal_towers: true now means vLLM language_model_only for every family; the Qwen3.5 language-tower vLLM classes are removed.

  Unset target_parameters on a packed-MoE catalog model resolves through the public resolve_lora_defaults in agilerl.arena.models.networks.

  Breaking: lora_target_scope is removed from the LLM algorithms, INIT_HP and the manifest. build_vllm_llm_init_kwargs(lora_target_modules=...) is replaced by tower_lora_target_modules, and apply_multimodal_engine_kwargs no longer takes runtime.

Tests

• speed up multi-rank tests and xdist runs

  Multi-rank tests share one launcher, tests/rank_launch.run_ranks. On Linux
  CPU hosts ranks fork from a forkserver that imports torch, transformers, peft
  and agilerl once per pytest worker instead of re-importing them per rank, and a
  crashed rank fails the test immediately. A new multi_rank marker replaces the
  per-file gloo skips.

  pytest-xdist workers run with one OpenMP thread each and -n logical, so worker
  thread pools stop contending for CPU cores.

  CI collects coverage only on the Python 3.13 jobs that are combined, and macOS
  skips multi_rank tests, which Linux runs. GPU tests run on Python 3.13 for
  pull requests and on every supported version in a nightly scheduled run.

Other

• bump vllm to 0.32.0 and torch to 2.13.0

  Bump the Linux llm extra to vLLM 0.32.0 and torch 2.13, with transformers 5.19.0, triton 3.7.1, huggingface-hub 2.2.0, datasets 5.1.0, liger-kernel 0.8.5, peft 0.21.2, and bitsandbytes 0.50.2. The vllm extra now requires Python >=3.11. Adapt to the new stack: Nemotron-H mamba patches follow the transformers 5.19 single-forward mixer, RoPE buffers (including axial vision rotaries) initialize after a meta load, the EP checkpoint replay assert is SAC-safe, and vLLM-side bitsandbytes quantization is removed (upstream dropped it; trainer-side bnb is unchanged). Hybrid Mamba families keep dense prefix-cache states through a new VllmRuntimeConfig.dense_mamba_prefix_cache and engine_kwargs(). The Nemotron-H Liger RMSNorm matches transformers 5.19 numerics and drops its unused RoPE option, and Qwen3.5 packed rows use the varlen kernels whenever causal-conv1d and FLA are installed. Super-VL loads natively without checkpoint code; tower LoRA targets use the native weight names. Nemotron-H and Nemotron-H Omni no longer load checkpoint code in the trainer or vLLM. Image observations from processors that return image_grid_hw fold into one (h, w, patch_dim) row per image, and the native Super-VL image features unfold those rows when no grid is given.

• start import sync when a labeled pull request changes

  Add a GitHub Actions workflow that runs when a pull request into main is opened, updated, labeled, or reopened. When the pull request has the hub-sync-import label, the job requests an import of that pull request. The workflow does not check out the pull request.

Full Changelog: v2.58.0...v2.59.0

agilerl-arena/v1.30.0: agilerl-arena v1.30.0: train vision-language checkpoints with their own architecture

Released on 2026-10-28 - GitHub - PyPI

Features

• support Qwen3.5 models in agilerl-arena, deprecate Qwen3

  Adds Qwen/Qwen3.5-0.8B, Qwen/Qwen3.5-2B, Qwen/Qwen3.5-4B, and Qwen/Qwen3.5-9B to SUPPORTED_MODEL_INFO with their LoRA targets, allowed ranks, and bundled config.json. Marks Qwen/Qwen3-1.7B and Qwen/Qwen3-4B deprecated. The memory estimator now counts the fp32 query upcast from Qwen3.5 attention QK norms. The Qwen3.5 language-tower weight mapper also covers the adapter-form language_model infix vLLM sees when loading LoRA, so text adapters sync onto tower engines.

• pin library workarounds to the releases they were checked against

  Add agilerl.utils.patching.library_shim, a decorator that records which release range of vLLM, transformers, peft, liger-kernel or torch a workaround was checked against. Calling a decorated shim with an installed release outside that range raises UnsupportedLibraryVersionError, naming the shim, the range and the installed version. Shims are listed in agilerl.utils.patching.LIBRARY_SHIMS. The vLLM, transformers, peft, Liger and torch workarounds in agilerl.architectures, agilerl.lora.fused, agilerl.distributed.fsdp_blocks and the colocated vLLM helpers now carry their ranges.

• expose expert-parallel support and routed expert count

  ModelInfo reports supports_expert_parallel and num_routed_experts. The flag is true when the model has packed expert modules. The count is read from the language config (n_routed_experts, num_experts, or num_local_experts) and is null when the model has no packed experts.

• load CLI plugins from the agilerl.arena.cli entry-point group

  Installed packages can add commands to the arena CLI by declaring an entry point in the agilerl.arena.cli group that names a callable taking the root click group, e.g. [project.entry-points."agilerl.arena.cli"] my = "pkg.cli:register". With no plugins installed the CLI is unchanged.

• price tensor parallel, expert parallel and NCCL process groups, and tighten LLM training and vLLM engine estimates

  The agilerl-arena memory estimator now sizes fsdp.tp and fsdp.ep. Under tensor parallel an untied lm_head sits at 1/tp outside the FSDP gather; families with a training tensor-parallel plan (nemotron_h) also shard their decoder linears and per-layer activations. Under expert parallel each rank keeps its share of the routed experts and their LoRA, each MoE backward adds token-exchange buffers, and tensor-parallel ranks route a share of the tokens each. NCCL memory is charged per process group, by size and by whether the group spans nodes.

  Training estimates now cover HF's unfused Mamba path on compute capability 8.9 GPUs, hybrids that run one mixer per layer (Granite 4), gpt-oss's expert intermediates, DPO's Liger loss, and Liger above token level. vLLM engine estimates keep the start-up profile's peak, add a compiling engine's per-layer embedding copy, size hybrid KV capacity by vLLM's block pool and SSM state dtype, and hold sliding-window KV for the scheduler's in-flight tokens. The H100 80GB catalogue entry uses its CUDA-visible 79.11 GiB. DeviceSpec.nccl_bytes, NCCL_BYTES_BY_DEVICE and NCCL_BYTES_DEFAULT are replaced by per-group NCCL bytes.

Fixes

• do not save the routed EP counts copy in packed-expert block checkpoints

  Selective checkpointing on packed-expert blocks saves every device-to-host copy. Routed expert-parallel counts skip that copy on recompute, so the next host copy (grouped-matmul fallback .tolist(), or actor/critic adapter ids) replayed a stale counts tensor and crashed with split_with_sizes(): found element of type list at pos 0.

  The routed counts copy is now flagged and not saved, so the saved copies stay in order and the recompute adds no host syncs. This hits expert-parallel training with gradient checkpointing on packed-expert blocks when the grouped-matmul fallback runs (GPUs other than compute capability 9/10, non-bf16, or unaligned sizes) or with actor/critic rows. Peak GPU memory is unchanged.

• resolve omitted training and generation settings in one place

  Omitted training and generation settings are chosen by one function each. The estimate and training call the same function. A set value is kept.

  An omitted mini-batch is the per-rank trajectory count, max(batch_size // data_parallel_size, 1) times group_size. GRPO-family uses the spec group size. Other algorithms use 1.

  An omitted micro_batch_size_per_gpu is the largest divisor of that mini-batch whose training estimate fits. On a CUDA training build, an omitted value is that search, including a single GPU. The search uses the launcher world size. Before the process group exists, the process rank is the launcher RANK. With no CUDA device, or a model outside the catalog, the omitted micro-batch is that mini-batch.

  An unset LLMPPO fuse_actor_critic_pass fuses when the fused training peak fits the device, and fuses when there is no device budget.

  An unset FSDP chunk is the largest that fits on a device. With no device it is the smallest chunk, and an unset optimizer sits on the CPU unless cpu_offload is set.

  An omitted fused-logprob chunk is the row count of a 256 MiB fp32 logit tile, clamped to 128..4096. An explicit count stays.

  An omitted max_num_seqs is 8 when generation shares the trainer GPU. Otherwise it is the prompts one engine receives times the group size. auto counts as one engine.

  An async engine max_lora_rank snaps up to the smallest accepted rank (8, 16, 32, 64, 128, 256, 320, 512) that covers the adapter. Colocated generation keeps max(configured, adapter rank) and does not snap.

  An omitted gpu_memory_utilization is 0.3. An omitted max_lora_rank starts at 16.

  An omitted LLMPPO fuse bit and an omitted FSDP chunk are priced on 95% of the card, including job overhead.

• point scoped-mismatch error at causal-LM scope fix

  Scoped LoRA errors now show example module keys present on the model alongside the empty in-scope sample, and suggest leaving the scope unset for causal LM layouts under consecutive model.layers path segments.

• lock flash-linear-attention on the llm extra

  The published lock now includes flash-linear-attention 0.5.2 for the Linux llm and all extras, matching the dependency already declared for those extras. Tests treat that pin as part of the llm extra, not cpu-llm, and cover GPU GatedDeltaNet setup when both fused kernels resolve.

• start vLLM engines for Granite 4 hybrid models on vLLM 0.32

  vLLM 0.32 reads Granite 4 hybrid layer types under the names transformers uses, so the catalog no longer renames them. Engines for granitemoehybrid checkpoints such as ibm-granite/granite-4.0-micro failed to start with KeyError: 'attention'. Removes granitemoehybrid_vllm_hf_override from agilerl.architectures.catalog.

• MoE routers under activation checkpointing, vision models, FSDP2, rollout logprobs and memory estimates

  • MoE: top-k router renormalization runs out of place for every catalog MoE family that renormalizes in place (Qwen3-MoE, Qwen3-Next, Qwen3-Omni, Qwen3.5/3.6-MoE, Qwen3-VL-MoE, Gemma 4, Qwen2-MoE, OLMoE, FlexOlmo, Mellum, Qwen4Exp, Mixtral, MiniMax). Selective activation checkpointing of MoE blocks no longer fails in backward.
  • FSDP2: units that have not run forward keep gradient sync on during gradient accumulation, which avoids a torch 2.13 post-backward crash (for example a frozen vision tower in text-only training). wrap_every_n_blocks > 1 keeps checkpoint parameter names. Gemma 4 RoPE buffers are built per layer type.
  • Qwen3.5/3.6 gated delta net: seq_idx is computed once in the text-model forward, which fixes a crash at micro-batch 1 under checkpointing. Multi-row packed batches keep document boundaries. The expert-parallel count exchange runs outside dispatch modes. Class-patched LigerExperts run split expert LoRA with SiLU.
  • Logprobs: the final-logit softcap (Gemma 4) is applied in fused logprobs and in the Liger losses that support it; GRPO-family objectives fall back from Liger with a warning otherwise.
  • Rollout: RolloutSampling carries every vLLM sampling field (top_k, min_p, penalties, min_tokens). vLLM engines return processed logprobs, so importance ratios match the trainer when temperature is not 1. With top_k > 0 and temperature > 0, rollouts record each token's sampling support and GRPO, LLMPPO and REINFORCE normalize policy, old and reference logprobs over it, so top_p, min_p and answer grammars keep the importance ratio exact; Arena manifests that use those filters without that support are rejected. Turns made only of forced tokens keep their reward; PPO/REINFORCE micro-batches with no action tokens give zero loss instead of NaN; REINFORCE turn KL averages over action tokens.
  • Vision: each model's own image placeholder, per-image vision rows for Qwen3.5/3.6 and Gemma 4, a tower lockstep so ranks without images don't hang, a frozen-tower feature cache per learn step, vLLM-served tower LoRA for Qwen3.5 and Gemma 4 (skipped on towers the engine doesn't build), and pixel staging through pinned buffers. Qwen3.5, Qwen3.6, Qwen3.8 and Gemma 4 E4B are marked vision-capable.
  • Envs: RemoteEnvClient.reset and the in-process client raise when the reset reports an error. Hosted reset images are sent as base64 PNG. The harness restarts the context at the model budget when restarts are on. New repeat_system_prompt setting.
  • vLLM compile cache directories are keyed by the engine's GPU ids (apply_vllm_compile_cache_env).
  • agilerl-arena manifests: env specs take modality (text / vision) and read-only host_volumes under /var/cache/agilerl-. Text envs strip vLLM vision towers by default when the model has towers and no tower LoRA; image envs keep them. Every rollout manifest without vllm_config gets the default engine. stop_sequences with async rollout is rejected at validation.
  • agilerl-arena memory estimator: CUDA-visible card sizes, vLLM LoRA stacks, encoder profile and non-torch memory in the KV budget, engine layout from the run's GPUs (--num-gpus), expert-LoRA backward chunks, gated delta net, held FSDP grads, Gemma 4 norms, FSDP slack, and vision host and GPU terms.
  • Breaking: agilerl.arena.models.default_colocated_vllm_config is renamed to default_rollout_vllm_config, with no alias.

• Qwen3.5 reasoning parser, padded GatedDeltaNet forward, GRPO Liger memory sizing

  Qwen3.5 architecture families now set the qwen3 vLLM reasoning parser in their runtime config. vLLM needs a reasoning parser to enforce thinking_token_budget, so rollouts that set a thinking budget on a Qwen3.5 model failed when the engine started.

  packed_layout now starts a document only at each row start and where position_ids is 0, the rule FlashAttention varlen uses. A padded batch passed with position_ids and no attention mask (pads at position 1) used to give every pad token its own document. On Qwen3.5/3.6 that sent thousands of sequences to the FLA varlen scan, whose launch fails with Triton Error [CUDA]: invalid argument once sequences x value heads exceeds 65535. Padded rows now stay one document each; packed rows are unchanged.

  The Arena memory estimator no longer refuses GSPO or trajectory-level GRPO runs with use_liger_loss=True when GRPO runs them on the standard path, which it does whenever the vLLM sampling correction or either off-policy mask is on. It still refuses the fused trajectory-level case, which is not memory-bounded.

• reduce split actor/critic FSDP gradients in parameter dtype

  Non-fused actor/critic training runs a policy backward and a value backward over different parameters. Deferred gradient sync keeps the first pass's gradients in reduce_dtype until the optimizer step. The second pass's gradients stay in param_dtype. FSDP reduce-scatter then rejects the unit when those dtypes differ.

  Split passes that defer gradient sync now reduce in param_dtype, so both passes use one dtype. Fused passes, single-pass runs, and configs that already match are left as configured.

• FSDP DPO and SFT fill unset memory settings from the spec

  Unset FSDP chunk size and optimizer placement for DPO and SFT are filled from the algorithm spec's context length before the algorithm is built. Constructing either on a GPU with those fields unset raises ValueError.

• save router top-k for checkpointed gpt-oss blocks

  Activation checkpointing saves each MoE router's top-k so backward replays the forward routing. Blocks with the transposed packed-expert layout (gpt-oss) were not detected, so their router was recomputed in backward. A token that switches expert then changes the expert loop shapes and raises CheckpointError, or silently uses different routing when shapes happen to match.

  Router saving now covers every packed-expert layout.

• charge saved router top-k in the memory estimator

  FSDP activation checkpointing saves each MoE router's top-k scores and expert indices for backward, so recompute replays the forward routing. The memory estimator charges those tensors as saved_router_topk when FSDP is on: tokens x top_k x MoE layers, at fp32 scores plus int64 indices. They stay on the GPU under activation offload. Flat data parallel recomputes the router, so it is not charged.

Breaking Changes

• train vision-language checkpoints with their own architecture

  Vision-language checkpoints (Qwen3.5/3.6, Gemma 3/4, Nemotron omni) now train with their full multimodal class, with frozen towers, so merged exports keep the original architecture, config.json and vision weights. The trainer class per family lives in the architecture catalog (TrainerRuntimeConfig.model_class, trainer_model_class).

  LoRA target_modules (named or all-linear) always land on the language model of a vision-language model. The trainer memory estimate classifies that list after the language-model regex rewrite, so attention-only adapters are not sized as all-linear. Tower or connector modules are adapted only when named in the new lora_config.tower_target_modules, which requires the towers to be served. The model catalog lists each vision-language checkpoint's language-model prefix and default text-only LoRA modules.

  strip_multimodal_towers: true now means vLLM language_model_only for every family; the Qwen3.5 language-tower vLLM classes are removed.

  Unset target_parameters on a packed-MoE catalog model resolves through the public resolve_lora_defaults in agilerl.arena.models.networks.

  Breaking: lora_target_scope is removed from the LLM algorithms, INIT_HP and the manifest. build_vllm_llm_init_kwargs(lora_target_modules=...) is replaced by tower_lora_target_modules, and apply_multimodal_engine_kwargs no longer takes runtime.

Tests

• speed up multi-rank tests and xdist runs

  Multi-rank tests share one launcher, tests/rank_launch.run_ranks. On Linux
  CPU hosts ranks fork from a forkserver that imports torch, transformers, peft
  and agilerl once per pytest worker instead of re-importing them per rank, and a
  crashed rank fails the test immediately. A new multi_rank marker replaces the
  per-file gloo skips.

  pytest-xdist workers run with one OpenMP thread each and -n logical, so worker
  thread pools stop contending for CPU cores.

  CI collects coverage only on the Python 3.13 jobs that are combined, and macOS
  skips multi_rank tests, which Linux runs. GPU tests run on Python 3.13 for
  pull requests and on every supported version in a nightly scheduled run.

Other

• bump vllm to 0.32.0 and torch to 2.13.0

  Bump the Linux llm extra to vLLM 0.32.0 and torch 2.13, with transformers 5.19.0, triton 3.7.1, huggingface-hub 2.2.0, datasets 5.1.0, liger-kernel 0.8.5, peft 0.21.2, and bitsandbytes 0.50.2. The vllm extra now requires Python >=3.11. Adapt to the new stack: Nemotron-H mamba patches follow the transformers 5.19 single-forward mixer, RoPE buffers (including axial vision rotaries) initialize after a meta load, the EP checkpoint replay assert is SAC-safe, and vLLM-side bitsandbytes quantization is removed (upstream dropped it; trainer-side bnb is unchanged). Hybrid Mamba families keep dense prefix-cache states through a new VllmRuntimeConfig.dense_mamba_prefix_cache and engine_kwargs(). The Nemotron-H Liger RMSNorm matches transformers 5.19 numerics and drops its unused RoPE option, and Qwen3.5 packed rows use the varlen kernels whenever causal-conv1d and FLA are installed. Super-VL loads natively without checkpoint code; tower LoRA targets use the native weight names. Nemotron-H and Nemotron-H Omni no longer load checkpoint code in the trainer or vLLM. Image observations from processors that return image_grid_hw fold into one (h, w, patch_dim) row per image, and the native Super-VL image features unfold those rows when no grid is given.

• start import sync when a labeled pull request changes

  Add a GitHub Actions workflow that runs when a pull request into main is opened, updated, labeled, or reopened. When the pull request has the hub-sync-import label, the job requests an import of that pull request. The workflow does not check out the pull request.

Full Changelog: agilerl-arena/v1.29.0...agilerl-arena/v1.30.0

v2.58.0: default GRPO old log-probs to the rollout engine unless colocated

Released on 2026-10-21 - GitHub - PyPI

Features

• default GRPO old log-probs to the rollout engine unless colocated

  Unset old_logprobs_source uses the inference engine's sampling log-probs (rollout) when generation is not colocated in the trainer process. Colocated vLLM keeps the learn-start policy (trainer) so the vLLM importance-sampling correction still applies. Set the key explicitly to override.

• parse tool calls from token IDs in the rollout harness

  The rollout harness now parses tool calls from generation token IDs and steps MCP-backed environments with structured calls, so environments never parse model output. Parsers cover the Qwen, Hermes, Granite, Nemotron, Gemma 4, and GPT-OSS formats; a new format is a ToolParser subclass. The model family catalog names each family's parser, and RolloutHarness(tool_call_parser=...) picks one explicitly; unset, it is detected from the tokenizer. A turn can hold several calls; they run in order and their rewards sum, and each result is its own tool message. A turn with no call is a prose answer for the environment's answer_tool. Malformed calls return retry feedback and score the new min_reward env setting (default 0). Env clients gain takes_tool_calls and answer_tool; the in-process client supports answer_tool and awaits step_async when an environment overrides it. Each step's info reports the valid tool calls the turn ran (TOOL_CALLS_INFO_KEY) and flags a rejected turn (TOOL_CALL_ERROR_INFO_KEY). Granite 4 hybrid configs remap layer_types full_attention / linear_attention to the names the vLLM engine uses (attention / mamba). See the environments guide Tools section.

Fixes

• merge env info prefix/suffix into rollout prompts

  Game envs in the gem convention return instructions as the observation and the task body as info["suffix"]. The harness now wraps prompt text in the env's info prefix/suffix when present, so the model sees content like the Sudoku board. Previously the board never reached the prompt and the model asked for it.

• choose an omitted vLLM tensor parallel width from the allocation

  vllm_config.tensor_parallel_size now defaults to unset. When the width is omitted, resolve_engine_tensor_parallel picks the width that keeps the most sequences resident on the leftover GPUs. An explicit integer, including 1, is kept.

  to_payload omits an unset width. The memory CLI resolves a width per resource tier and fails a tier that cannot be resolved. On a named GPU, an omitted width is sized as 1. A candidate width that is illegal for a hybrid Mamba checkpoint scores 0. Colocated in-process vLLM still launches on one GPU and does not write 1 back onto the config. generation_settings_from_manifest requires a candidate width when a rollout manifest leaves it unset.

Full Changelog: v2.57.0...v2.58.0

agilerl-arena/v1.29.0: agilerl-arena v1.29.0: default GRPO old log-probs to the rollout engine unless colocated

Released on 2026-10-21 - GitHub - PyPI

Features

• default GRPO old log-probs to the rollout engine unless colocated

  Unset old_logprobs_source uses the inference engine's sampling log-probs (rollout) when generation is not colocated in the trainer process. Colocated vLLM keeps the learn-start policy (trainer) so the vLLM importance-sampling correction still applies. Set the key explicitly to override.

• parse tool calls from token IDs in the rollout harness

  The rollout harness now parses tool calls from generation token IDs and steps MCP-backed environments with structured calls, so environments never parse model output. Parsers cover the Qwen, Hermes, Granite, Nemotron, Gemma 4, and GPT-OSS formats; a new format is a ToolParser subclass. The model family catalog names each family's parser, and RolloutHarness(tool_call_parser=...) picks one explicitly; unset, it is detected from the tokenizer. A turn can hold several calls; they run in order and their rewards sum, and each result is its own tool message. A turn with no call is a prose answer for the environment's answer_tool. Malformed calls return retry feedback and score the new min_reward env setting (default 0). Env clients gain takes_tool_calls and answer_tool; the in-process client supports answer_tool and awaits step_async when an environment overrides it. Each step's info reports the valid tool calls the turn ran (TOOL_CALLS_INFO_KEY) and flags a rejected turn (TOOL_CALL_ERROR_INFO_KEY). Granite 4 hybrid configs remap layer_types full_attention / linear_attention to the names the vLLM engine uses (attention / mamba). See the environments guide Tools section.

Fixes

• merge env info prefix/suffix into rollout prompts

  Game envs in the gem convention return instructions as the observation and the task body as info["suffix"]. The harness now wraps prompt text in the env's info prefix/suffix when present, so the model sees content like the Sudoku board. Previously the board never reached the prompt and the model asked for it.

• choose an omitted vLLM tensor parallel width from the allocation

  vllm_config.tensor_parallel_size now defaults to unset. When the width is omitted, resolve_engine_tensor_parallel picks the width that keeps the most sequences resident on the leftover GPUs. An explicit integer, including 1, is kept.

  to_payload omits an unset width. The memory CLI resolves a width per resource tier and fails a tier that cannot be resolved. On a named GPU, an omitted width is sized as 1. A candidate width that is illegal for a hybrid Mamba checkpoint scores 0. Colocated in-process vLLM still launches on one GPU and does not write 1 back onto the config. generation_settings_from_manifest requires a candidate width when a rollout manifest leaves it unset.

Full Changelog: agilerl-arena/v1.28.0...agilerl-arena/v1.29.0

v2.57.0: drop offload_trainer_during_rollout from GRPO and LLMPPO manifests

Released on 2026-10-21 - GitHub - PyPI

Features

• drop offload_trainer_during_rollout from GRPO and LLMPPO manifests

  Arena generation is async, so this colocated-vLLM trainer-offload field was never applied. Manifests that still set it are rejected as extra input. GSPO and CISPO inherit the GRPO spec, so they lose the field too.

Fixes

• size an fp8 tensor-parallel vLLM engine

  The memory estimator sizes an fp8 engine split across tensor-parallel ranks. KV heads stay replicated when a rank would otherwise get none, and the Mamba state is sharded the way vLLM shards it. Training tensor parallel does not change the activation estimate. Resource-tier placement counts a GPU per tensor-parallel rank.

• stop rewriting env actions into function calls by default

  The rollout harness sends the text after </think> to the env unchanged. Set action_processor to a module:fn when an env needs loose calls rewritten as function calls. That function receives the post-</think> text and returns the string the env executes. Restart prompts list those same actions.

  action_span is the text after the first </think>. env_action_text is removed.

  screenshot_field and goal_images_field name the observation keys for the page image and the task images. They default to screenshot and goal_images. An empty value reads none.

• drop the Qwen3.5 LoRA key prefix in the vLLM LoRA export

  The vLLM LoRA export no longer adds a model.language_model.model. prefix to Qwen3.5 LoRA keys. vLLM's Qwen3.5 weight mapper already maps Hugging Face names onto its modules, and the extra prefix stopped the adapter matching any vLLM layer. LanguageTowerRuntimeConfig.lora_key_prefix and the model_type argument of remap_peft_lora_key_for_vllm and filter_peft_state_dict_for_vllm_lora are removed.

• make the MADDPG and MATD3 PettingZoo tutorials run end to end

• shard Gemma 4 per-layer embeddings under FSDP and init layer-type RoPE

  Shard Gemma 4 packed per-layer embeddings as their own FSDP unit and prefetch them with the language blocks. Fill {layer_type}_inv_freq RoPE buffers and embed_scale after a meta safetensors load.

• default LLMEnvSpec screenshot_field and goal_images_field to unset

  LLMEnvSpec.screenshot_field and LLMEnvSpec.goal_images_field default to unset. Empty or omitted keys read no page or task images. The env JSON Schema no longer publishes string defaults for those fields. Dataset-backed rollouts reject any non-empty image observation keys. Rollout factories omit image observation kwargs when unset so the harness does not read screenshot or goal_images unless the spec names them.

Refactoring

• serve stripped towers with language_model_only

  VLLMConfig.strip_multimodal_towers=True serves Gemma 4 and Nemotron through vLLM language_model_only. Qwen3.5 and Qwen3.6 still load the language-tower class, because vLLM does not register those text configs. A list names modalities to drop (image, video, audio) and sets limit_mm_per_prompt to 0.

  The LoRA export keeps LoRA tensors and undoes trainer-only wrappers. Hugging Face to vLLM renames stay on the serving model's weight mapper. save_peft_adapter_for_vllm_rollout and filter_peft_state_dict_for_vllm_lora no longer take target_modules or strip_multimodal_towers. patch_vllm_strip_multimodal_towers is removed.

Other

• Revert GitHub PR #707

  The pull request merges next so its author is credited.

Full Changelog: v2.56.1...v2.57.0

agilerl-arena/v1.28.0: agilerl-arena v1.28.0: drop offload_trainer_during_rollout from GRPO and LLMPPO manifests

Released on 2026-10-21 - GitHub - PyPI

Features

• drop offload_trainer_during_rollout from GRPO and LLMPPO manifests

  Arena generation is async, so this colocated-vLLM trainer-offload field was never applied. Manifests that still set it are rejected as extra input. GSPO and CISPO inherit the GRPO spec, so they lose the field too.

Fixes

• size an fp8 tensor-parallel vLLM engine

  The memory estimator sizes an fp8 engine split across tensor-parallel ranks. KV heads stay replicated when a rank would otherwise get none, and the Mamba state is sharded the way vLLM shards it. Training tensor parallel does not change the activation estimate. Resource-tier placement counts a GPU per tensor-parallel rank.

• stop rewriting env actions into function calls by default

  The rollout harness sends the text after </think> to the env unchanged. Set action_processor to a module:fn when an env needs loose calls rewritten as function calls. That function receives the post-</think> text and returns the string the env executes. Restart prompts list those same actions.

  action_span is the text after the first </think>. env_action_text is removed.

  screenshot_field and goal_images_field name the observation keys for the page image and the task images. They default to screenshot and goal_images. An empty value reads none.

• drop the Qwen3.5 LoRA key prefix in the vLLM LoRA export

  The vLLM LoRA export no longer adds a model.language_model.model. prefix to Qwen3.5 LoRA keys. vLLM's Qwen3.5 weight mapper already maps Hugging Face names onto its modules, and the extra prefix stopped the adapter matching any vLLM layer. LanguageTowerRuntimeConfig.lora_key_prefix and the model_type argument of remap_peft_lora_key_for_vllm and filter_peft_state_dict_for_vllm_lora are removed.

• make the MADDPG and MATD3 PettingZoo tutorials run end to end

• shard Gemma 4 per-layer embeddings under FSDP and init layer-type RoPE

  Shard Gemma 4 packed per-layer embeddings as their own FSDP unit and prefetch them with the language blocks. Fill {layer_type}_inv_freq RoPE buffers and embed_scale after a meta safetensors load.

• default LLMEnvSpec screenshot_field and goal_images_field to unset

  LLMEnvSpec.screenshot_field and LLMEnvSpec.goal_images_field default to unset. Empty or omitted keys read no page or task images. The env JSON Schema no longer publishes string defaults for those fields. Dataset-backed rollouts reject any non-empty image observation keys. Rollout factories omit image observation kwargs when unset so the harness does not read screenshot or goal_images unless the spec names them.

Refactoring

• serve stripped towers with language_model_only

  VLLMConfig.strip_multimodal_towers=True serves Gemma 4 and Nemotron through vLLM language_model_only. Qwen3.5 and Qwen3.6 still load the language-tower class, because vLLM does not register those text configs. A list names modalities to drop (image, video, audio) and sets limit_mm_per_prompt to 0.

  The LoRA export keeps LoRA tensors and undoes trainer-only wrappers. Hugging Face to vLLM renames stay on the serving model's weight mapper. save_peft_adapter_for_vllm_rollout and filter_peft_state_dict_for_vllm_lora no longer take target_modules or strip_multimodal_towers. patch_vllm_strip_multimodal_towers is removed.

Other

• Revert GitHub PR #707

  The pull request merges next so its author is credited.

Full Changelog: agilerl-arena/v1.27.0...agilerl-arena/v1.28.0

v2.56.1: mark nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16 live

Released on 2026-10-15 - GitHub - PyPI

Features

• mark nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16 live

  Remove status=preview from nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16 in SUPPORTED_MODEL_INFO. The catalog default is live, so Arena lists Super-VL with other live models and the CLI/SDK accept experiment submissions for it. Preview models stay rejected on submit.

• add input modalities to supported models

  ModelInfo now has a modalities field. The default is text-only. Super-VL (nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16) is text and vision. Other catalog ids stay text. arena models supported dumps the field with each entry. Values must be text, or text and vision.

• size multi-row training micro-batches in the memory estimator

  The agilerl-arena memory estimator sizes the trainer micro-batch instead of refusing anything but one row. Unset micro_batch_size_per_gpu follows the same resolution as training (per-rank prompt batch, folded by tensor-parallel degree), and the batch must split into whole mini-batches and micro-batches. Sequence packing is still priced as padded rows, with a warning. Over-budget advice can now suggest dropping micro_batch_size_per_gpu to 1.

Fixes

• check the generation peak against the whole card

  The agilerl-arena memory estimator checks the generation phase against the whole card instead of 95% of it. vLLM claims gpu_memory_utilization of the card at startup, and the CUDA context and runtime are already counted as generation components, so a 0.95 budget on an 80 GB GPU is no longer reported over budget. Training keeps its fragmentation band and recommendation buffer.

• accept agilerl-arena X.Y.Z.devN as a build-time exact pin

  check-extras.py and set-arena-extra-pin.sh now accept PEP 440 X.Y.Z.devN in addition to X.Y.Z and X.Y.Z+local when rewriting the committed agilerl-arena extra to an exact pin for one wheel build.

• size the vLLM profile step the async engine starts with

  The agilerl-arena memory estimator sized vLLM's profile forward with the colocated rule, which reaches 65,536 tokens for 8 sequences at 32k. An async engine profiles the model family's max_num_batched_tokens or vLLM's default of 2,048, capped at max_num_seqs * max_model_len. The oversized forward shrank the predicted KV cache pool.

• parse MoE expert layers and shared experts in the memory estimator

  The agilerl-arena memory estimator raised KeyError on configs without intermediate_size (Qwen3.5/3.6 MoE) and did not count shared experts named only by shared_expert_intermediate_size. Nemotron-H hybrid_override_pattern expert layers (E) were not counted, so routed experts were charged to every layer.

• a generation bar fits only when the KV pool holds one full sequence

  The agilerl-arena memory estimator counted a generation bar as fitting whenever its total was under the card. vLLM refuses to start when its KV pool cannot hold one max_model_len request (KV plus Mamba state), so such a bar no longer fits and carries a warning.

Full Changelog: v2.56.0...v2.56.1

agilerl-arena/v1.27.0: agilerl-arena v1.27.0: add input modalities to supported models

Released on 2026-10-15 - GitHub - PyPI

Features

• add input modalities to supported models

  ModelInfo now has a modalities field. The default is text-only. Super-VL (nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16) is text and vision. Other catalog ids stay text. arena models supported dumps the field with each entry. Values must be text, or text and vision.

• size multi-row training micro-batches in the memory estimator

  The agilerl-arena memory estimator sizes the trainer micro-batch instead of refusing anything but one row. Unset micro_batch_size_per_gpu follows the same resolution as training (per-rank prompt batch, folded by tensor-parallel degree), and the batch must split into whole mini-batches and micro-batches. Sequence packing is still priced as padded rows, with a warning. Over-budget advice can now suggest dropping micro_batch_size_per_gpu to 1.

Fixes

• accept agilerl-arena X.Y.Z.devN as a build-time exact pin

  check-extras.py and set-arena-extra-pin.sh now accept PEP 440 X.Y.Z.devN in addition to X.Y.Z and X.Y.Z+local when rewriting the committed agilerl-arena extra to an exact pin for one wheel build.

• size the vLLM profile step the async engine starts with

  The agilerl-arena memory estimator sized vLLM's profile forward with the colocated rule, which reaches 65,536 tokens for 8 sequences at 32k. An async engine profiles the model family's max_num_batched_tokens or vLLM's default of 2,048, capped at max_num_seqs * max_model_len. The oversized forward shrank the predicted KV cache pool.

• parse MoE expert layers and shared experts in the memory estimator

  The agilerl-arena memory estimator raised KeyError on configs without intermediate_size (Qwen3.5/3.6 MoE) and did not count shared experts named only by shared_expert_intermediate_size. Nemotron-H hybrid_override_pattern expert layers (E) were not counted, so routed experts were charged to every layer.

• a generation bar fits only when the KV pool holds one full sequence

  The agilerl-arena memory estimator counted a generation bar as fitting whenever its total was under the card. vLLM refuses to start when its KV pool cannot hold one max_model_len request (KV plus Mamba state), so such a bar no longer fits and carries a warning.

Full Changelog: agilerl-arena/v1.26.1...agilerl-arena/v1.27.0

agilerl-arena/v1.26.1: agilerl-arena v1.26.1: check the generation peak against the whole card

Released on 2026-10-13 - GitHub - PyPI

Fixes

• check the generation peak against the whole card

  The agilerl-arena memory estimator checks the generation phase against the whole card instead of 95% of it. vLLM claims gpu_memory_utilization of the card at startup, and the CUDA context and runtime are already counted as generation components, so a 0.95 budget on an 80 GB GPU is no longer reported over budget. Training keeps its fragmentation band and recommendation buffer.

Full Changelog: agilerl-arena/v1.26.0...agilerl-arena/v1.26.1

agilerl-arena/v1.26.0: agilerl-arena v1.26.0: mark nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16 live

Released on 2026-10-13 - GitHub - PyPI

Features

• mark nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16 live

  Remove status=preview from nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16 in SUPPORTED_MODEL_INFO. The catalog default is live, so Arena lists Super-VL with other live models and the CLI/SDK accept experiment submissions for it. Preview models stay rejected on submit.

Fixes

• leftover-DP packed expert shards and router topk under FSDP checkpointing

  Packed expert safetensors copies now keep this rank's leftover data-parallel rows when the checkpoint slice is still the expert-parallel block. Indexed expert stacks already did that; 3-D packed slices did not, so leftover-DP loads failed with a destination/source shape mismatch.

  Activation checkpointing on blocks that hold packed experts saves router topk (and device-to-host copies) so recompute cannot change expert assignment or all-to-all splits.

  GRPO learn-phase timing tests advance a fake clock for the no-grad forward delay so the charge does not depend on wall sleep.

Full Changelog: agilerl-arena/v1.25.0...agilerl-arena/v1.26.0

v2.56.0: leftover-DP packed expert shards and router topk under FSDP checkpointing

Released on 2026-10-12 - GitHub - PyPI

Fixes

• leftover-DP packed expert shards and router topk under FSDP checkpointing

  Packed expert safetensors copies now keep this rank's leftover data-parallel rows when the checkpoint slice is still the expert-parallel block. Indexed expert stacks already did that; 3-D packed slices did not, so leftover-DP loads failed with a destination/source shape mismatch.

  Activation checkpointing on blocks that hold packed experts saves router topk (and device-to-host copies) so recompute cannot change expert assignment or all-to-all splits.

  GRPO learn-phase timing tests advance a fake clock for the no-grad forward delay so the charge does not depend on wall sleep.

Full Changelog: v2.55.0...v2.56.0

v2.55.0: support Qwen/Qwen3.6-35B-A3B and Qwen3.5 language towers

Released on 2026-10-12 - GitHub - PyPI

Features

• support Qwen/Qwen3.6-35B-A3B and Qwen3.5 language towers

  Adds Qwen/Qwen3.6-35B-A3B to SUPPORTED_MODEL_INFO with its LoRA targets, allowed ranks, and bundled config.json.

  Registers vLLM language-tower classes for Qwen3.5 VL checkpoints, including text-only M-RoPE positions.

  Windows CPU pytest uses a 10-minute per-test timeout; the GitHub job is capped at 40 minutes.

Full Changelog: v2.54.0...v2.55.0

agilerl-arena/v1.25.0: agilerl-arena v1.25.0: enable vLLM tower/connector LoRA only when the adapter targets those modules

Released on 2026-10-12 - GitHub - PyPI

Features

• support Qwen/Qwen3.6-35B-A3B and Qwen3.5 language towers

  Adds Qwen/Qwen3.6-35B-A3B to SUPPORTED_MODEL_INFO with its LoRA targets, allowed ranks, and bundled config.json.

  Registers vLLM language-tower classes for Qwen3.5 VL checkpoints, including text-only M-RoPE positions.

  Windows CPU pytest uses a 10-minute per-test timeout; the GitHub job is capped at 40 minutes.

Breaking Changes

• enable vLLM tower/connector LoRA only when the adapter targets those modules

  Turn on vLLM tower/connector LoRA only when the LoRA adapter's PEFT targets include the family's tower or connector linears.

  Breaking:

  • ModelRuntimeConfig.enable_tower_connector_lora: bool is now tower_connector_lora_targets: frozenset[str]: the trainer-side PEFT names of those linears (set for nemotron_h_omni).
  • apply_tower_connector_lora_engine_kwargs takes a required lora_target_modules. Tower LoRA turns on when those targets name a tower/connector module, or are a string spec (all-linear / regex) or None. It stays off when the family has no targets or towers are stripped.

  build_vllm_llm_init_kwargs and LLMAlgorithm pass the adapter's target_modules into that gate.

Full Changelog: agilerl-arena/v1.24.0...agilerl-arena/v1.25.0

v2.54.0: enable vLLM tower/connector LoRA only when the adapter targets those modules

Released on 2026-10-11 - GitHub - PyPI

Breaking Changes

• enable vLLM tower/connector LoRA only when the adapter targets those modules

  Turn on vLLM tower/connector LoRA only when the LoRA adapter's PEFT targets include the family's tower or connector linears.

  Breaking:

  • ModelRuntimeConfig.enable_tower_connector_lora: bool is now tower_connector_lora_targets: frozenset[str]: the trainer-side PEFT names of those linears (set for nemotron_h_omni).
  • apply_tower_connector_lora_engine_kwargs takes a required lora_target_modules. Tower LoRA turns on when those targets name a tower/connector module, or are a string spec (all-linear / regex) or None. It stays off when the family has no targets or towers are stripped.

  build_vllm_llm_init_kwargs and LLMAlgorithm pass the adapter's target_modules into that gate.

Full Changelog: v2.53.0...v2.54.0

v2.53.0: default GRPO-family off-policy masks, kl_clamp 10, accumulation_window loss_norm and mean-only advantages

Released on 2026-10-10 - GitHub - PyPI

Breaking Changes

• default GRPO-family off-policy masks, kl_clamp 10, accumulation_window loss_norm and mean-only advantages

  GRPO, GSPO and CISPO now default to loss_norm: accumulation_window, kl_clamp: 10.0 (also LLM PPO), off_policy_token_mask_bounds: [0.5, 5.0], off_policy_sequence_mask_threshold: 0.03, use_bias_correction_kl: true, adv_norm: mean_only, and top_p: 1.0. Agents built or resumed without those keys pick up the new defaults. Opt out with kl_clamp: null, loss_norm: micro_batch, null / false / mean_std / 0.95.

  kl_clamp is a one-sided per-token bound on the K3 KL penalty: tokens further below the reference get no KL gradient. Off-policy token and sequence masks drop the policy and KL gradient of out-of-band tokens and drifted negative-advantage rows. Bias-corrected KL weights K3 by the un-clamped ratio to the old policy.

  The fused Liger GRPO-family loss now uses the same per-token weights as the PyTorch path under every loss_norm. GSPO with use_liger_loss=True runs (Liger sequence level) instead of raising. CISPO + Liger configs that trained at micro_batch now set loss_norm: accumulation_window.

  Arena GRPOSpec / CISPOSpec / GSPOSpec / LLMPPOSpec expose the new fields with those defaults. RolloutLLMSpec.top_p defaults to 1.0.
  Adaptive task sampling can pool rows into families: TaskAssigner(families=..., family_prior_strength=...) starts each row at its family's informative rate. Arena env specs expose task_family_field (off by default) and task_family_prior_strength (1.0).
  Arena TrainingSpec.reuse_prefix_cache_across_syncs (off by default) keeps the served LoRA name, and so vLLM's prefix cache, for max_rollout_version_lag weight versions; it requires an LLM replay_buffer with max_rollout_version_lag >= 1.

Full Changelog: v2.52.0...v2.53.0

agilerl-arena/v1.24.0: agilerl-arena v1.24.0: default GRPO-family off-policy masks, kl_clamp 10, accumulation_window loss_norm and mean-only advantages

Released on 2026-10-10 - GitHub - PyPI

Breaking Changes

• default GRPO-family off-policy masks, kl_clamp 10, accumulation_window loss_norm and mean-only advantages

  GRPO, GSPO and CISPO now default to loss_norm: accumulation_window, kl_clamp: 10.0 (also LLM PPO), off_policy_token_mask_bounds: [0.5, 5.0], off_policy_sequence_mask_threshold: 0.03, use_bias_correction_kl: true, adv_norm: mean_only, and top_p: 1.0. Agents built or resumed without those keys pick up the new defaults. Opt out with kl_clamp: null, loss_norm: micro_batch, null / false / mean_std / 0.95.

  kl_clamp is a one-sided per-token bound on the K3 KL penalty: tokens further below the reference get no KL gradient. Off-policy token and sequence masks drop the policy and KL gradient of out-of-band tokens and drifted negative-advantage rows. Bias-corrected KL weights K3 by the un-clamped ratio to the old policy.

  The fused Liger GRPO-family loss now uses the same per-token weights as the PyTorch path under every loss_norm. GSPO with use_liger_loss=True runs (Liger sequence level) instead of raising. CISPO + Liger configs that trained at micro_batch now set loss_norm: accumulation_window.

  Arena GRPOSpec / CISPOSpec / GSPOSpec / LLMPPOSpec expose the new fields with those defaults. RolloutLLMSpec.top_p defaults to 1.0.
  Adaptive task sampling can pool rows into families: TaskAssigner(families=..., family_prior_strength=...) starts each row at its family's informative rate. Arena env specs expose task_family_field (off by default) and task_family_prior_strength (1.0).
  Arena TrainingSpec.reuse_prefix_cache_across_syncs (off by default) keeps the served LoRA name, and so vLLM's prefix cache, for max_rollout_version_lag weight versions; it requires an LLM replay_buffer with max_rollout_version_lag >= 1.

Full Changelog: agilerl-arena/v1.23.0...agilerl-arena/v1.24.0

v2.52.0: speed up LLM learn with row balance, routed-expert chunks, and memory auto-pick

Released on 2026-10-10 - GitHub - PyPI

Features

• speed up LLM learn with row balance, routed-expert chunks, and memory auto-pick

  LLM algorithms share a faster learn loop: data-parallel ranks exchange segment rows so each rank runs about the mean count; packed-row layout, mixer scan, and micro-batch metrics stay on device until the end of learn; loss finiteness is checked on device and read once per optimizer step.

  Routed-expert LoRA chunks by fixed row ranges with device-side offsets (FSDPConfig.routed_expert_chunk_mib). Unset chunk size and optim_cpu_offload are resolved from the agilerl-arena estimate (largest fitting 64/128/256/512 MiB, then GPU fused AdamW only if the estimate still fits). optim_cpu_offload defaults to None. EP LoRA grads keep their parameter strides so fused AdamW accepts them. Frozen Mamba2 out_proj runs after the fused scan. torch._grouped_mm runs only on sm90/sm100 bf16.

  The arena estimator adds a routed-chunk term, GPU Adam state, packed expert LoRA as stacked tensors, a host breakdown, and shards weights across one shard_group_size group. Segmented rollouts keep prompts within segment_prompt_tokens and end with prompt_limit when a restart does not fit; max_row_tokens sizes the estimate. Optional restart_older_obs_field / restart_older_images shrink restart context (off by default). Frozen vision tower outputs are reused within a GRPO, PPO, or REINFORCE learn step. VLLMConfig.limit_mm_per_prompt is a typed field. TaskAssigner shards rows by stride. Checkpoints can snapshot on the host (snapshot_checkpoint) and store optimizer state (training.checkpoint_optimizer). Hugging Face trust_remote_code loads copy checkpoint code under a cross-process file lock; weight loads run outside that lock.

  Breaking: balanced_row_plan takes ranks_per_group; balance_rows_across_ranks takes shard_group_size and row_values=, and returns (rows, values). pad_row_advantages is pad_row_values. materialize_fsdp2_from_cpu_state needs routed_expert_chunk_mib set; FSDPRuntime.prepare_actor needs optim_cpu_offload set (wrap_models resolves both). fsdp.compile_blocks and compile_backend are removed; a manifest that sets them fails validation. DPO.learn returns cross-rank means of loss and implicit rewards. Distributed LoRA init uses the shared seed on every rank.

Full Changelog: v2.51.0...v2.52.0

agilerl-arena/v1.23.0: agilerl-arena v1.23.0: speed up LLM learn with row balance, routed-expert chunks, and memory auto-pick

Released on 2026-10-10 - GitHub - PyPI

Features

• speed up LLM learn with row balance, routed-expert chunks, and memory auto-pick

  LLM algorithms share a faster learn loop: data-parallel ranks exchange segment rows so each rank runs about the mean count; packed-row layout, mixer scan, and micro-batch metrics stay on device until the end of learn; loss finiteness is checked on device and read once per optimizer step.

  Routed-expert LoRA chunks by fixed row ranges with device-side offsets (FSDPConfig.routed_expert_chunk_mib). Unset chunk size and optim_cpu_offload are resolved from the agilerl-arena estimate (largest fitting 64/128/256/512 MiB, then GPU fused AdamW only if the estimate still fits). optim_cpu_offload defaults to None. EP LoRA grads keep their parameter strides so fused AdamW accepts them. Frozen Mamba2 out_proj runs after the fused scan. torch._grouped_mm runs only on sm90/sm100 bf16.

  The arena estimator adds a routed-chunk term, GPU Adam state, packed expert LoRA as stacked tensors, a host breakdown, and shards weights across one shard_group_size group. Segmented rollouts keep prompts within segment_prompt_tokens and end with prompt_limit when a restart does not fit; max_row_tokens sizes the estimate. Optional restart_older_obs_field / restart_older_images shrink restart context (off by default). Frozen vision tower outputs are reused within a GRPO, PPO, or REINFORCE learn step. VLLMConfig.limit_mm_per_prompt is a typed field. TaskAssigner shards rows by stride. Checkpoints can snapshot on the host (snapshot_checkpoint) and store optimizer state (training.checkpoint_optimizer). Hugging Face trust_remote_code loads copy checkpoint code under a cross-process file lock; weight loads run outside that lock.

  Breaking: balanced_row_plan takes ranks_per_group; balance_rows_across_ranks takes shard_group_size and row_values=, and returns (rows, values). pad_row_advantages is pad_row_values. materialize_fsdp2_from_cpu_state needs routed_expert_chunk_mib set; FSDPRuntime.prepare_actor needs optim_cpu_offload set (wrap_models resolves both). fsdp.compile_blocks and compile_backend are removed; a manifest that sets them fails validation. DPO.learn returns cross-rank means of loss and implicit rewards. Distributed LoRA init uses the shared seed on every rank.

Fixes

• scale MoE expert LoRA on rank-r rows; load LLM checkpoints with extra LoRA target modules

  • low_rank_delta in agilerl.lora.moe.adapters applies the LoRA scaling to the rank-r intermediate, so no multiply runs over the full [rows, out] output in forward or backward. The result is unchanged.
  • LLMAlgorithm checkpoint loading accepts a checkpoint whose LoRA config adapts a strict superset of the live config's target_modules and otherwise matches. It warns and skips the extra modules' adapter weights. Other LoRA config mismatches still raise.

Full Changelog: agilerl-arena/v1.22.0...agilerl-arena/v1.23.0

v2.51.0: invert one memory setting and pick the cheapest fitting GPU tier

Released on 2026-10-09 - GitHub - PyPI

Features

• invert one memory setting and pick the cheapest fitting GPU tier

  arena memory solve FIELD holds every other input fixed and returns the largest value that still fits. Training uses the same underprediction buffer as estimate. Fields: max_model_len, max_num_seqs. --inference sizes a dedicated serving GPU.

  arena memory estimate without --gpu or --device-gb picks the cheapest Arena resource tier (credits per node-hour) whose node fits the manifest. Training fit uses the estimator's underprediction buffer; a tier that only fits on the point estimate is not recommended.

Fixes

• scale MoE expert LoRA on rank-r rows; load LLM checkpoints with extra LoRA target modules

  • low_rank_delta in agilerl.lora.moe.adapters applies the LoRA scaling to the rank-r intermediate, so no multiply runs over the full [rows, out] output in forward or backward. The result is unchanged.
  • LLMAlgorithm checkpoint loading accepts a checkpoint whose LoRA config adapts a strict superset of the live config's target_modules and otherwise matches. It warns and skips the extra modules' adapter weights. Other LoRA config mismatches still raise.

Full Changelog: v2.50.0...v2.51.0

agilerl-arena/v1.22.0: agilerl-arena v1.22.0: invert one memory setting and pick the cheapest fitting GPU tier

Released on 2026-10-09 - GitHub - PyPI

Features

• invert one memory setting and pick the cheapest fitting GPU tier

  arena memory solve FIELD holds every other input fixed and returns the largest value that still fits. Training uses the same underprediction buffer as estimate. Fields: max_model_len, max_num_seqs. --inference sizes a dedicated serving GPU.

  arena memory estimate without --gpu or --device-gb picks the cheapest Arena resource tier (credits per node-hour) whose node fits the manifest. Training fit uses the estimator's underprediction buffer; a tier that only fits on the point estimate is not recommended.

Full Changelog: agilerl-arena/v1.21.0...agilerl-arena/v1.22.0

v2.50.0: add CNN→LSTM encoder for image observations (cnn_lstm)

Released on 2026-10-08 - GitHub - PyPI

Features

• add CNN→LSTM encoder for image observations (cnn_lstm)

  Add EvolvableCnnLstm (CNN trunk + LSTM head) and select it when the observation is a 3D image Box and recurrent=True. Hidden-state keys follow LSTM ({name}_h / {name}_c). Architecture mutation on the composite encoder is disabled. input_shape is channel-first. Sequence forward keeps the time axis like LSTM. The module takes net_config (CnnLstmNetConfig or a dict).

  Arena exposes CnnLstmSpec and arch: cnn_lstm. infer_encoder_arch returns cnn_lstm for image observations with recurrent=True. Sample configs: ppo_cnn_lstm.yaml and ppo_image_recurrent.yaml. Module defaults match CnnLstmNetConfig.

Full Changelog: v2.49.1...v2.50.0

agilerl-arena/v1.21.0: agilerl-arena v1.21.0: add CNN→LSTM encoder for image observations (cnn_lstm)

Released on 2026-10-08 - GitHub - PyPI

Features

• add CNN→LSTM encoder for image observations (cnn_lstm)

  Add EvolvableCnnLstm (CNN trunk + LSTM head) and select it when the observation is a 3D image Box and recurrent=True. Hidden-state keys follow LSTM ({name}_h / {name}_c). Architecture mutation on the composite encoder is disabled. input_shape is channel-first. Sequence forward keeps the time axis like LSTM. The module takes net_config (CnnLstmNetConfig or a dict).

  Arena exposes CnnLstmSpec and arch: cnn_lstm. infer_encoder_arch returns cnn_lstm for image observations with recurrent=True. Sample configs: ppo_cnn_lstm.yaml and ppo_image_recurrent.yaml. Module defaults match CnnLstmNetConfig.

Full Changelog: agilerl-arena/v1.20.0...agilerl-arena/v1.21.0

v2.49.1: pre-submission GPU memory estimate gate

Released on 2026-10-08 - GitHub - PyPI

Features

• pre-submission GPU memory estimate gate

  arena memory estimate sizes a training manifest against a GPU before submit. Exit 0 if both phases fit, 3 if either is over budget. Pass --config to stay offline.

Full Changelog: v2.49.0...v2.49.1

agilerl-arena/v1.20.0: agilerl-arena v1.20.0: pre-submission GPU memory estimate gate

Released on 2026-10-08 - GitHub - PyPI

Features

• pre-submission GPU memory estimate gate

  arena memory estimate sizes a training manifest against a GPU before submit. Exit 0 if both phases fit, 3 if either is over budget. Pass --config to stay offline.

Full Changelog: agilerl-arena/v1.19.0...agilerl-arena/v1.20.0

v2.49.0: Show action errors and instruction images in LLM env prompts, keep recent turns across context restarts, add PPO critic warmup and per-group gradient clipping

Released on 2026-10-07 - GitHub - PyPI

Other

• Show action errors and instruction images in LLM env prompts, keep recent turns across context restarts, add PPO critic warmup and per-group gradient clipping

  • RolloutHarness(action_error_field=...) lists an observation field's error text after its action when a context restart summarizes past actions.
  • RolloutHarness puts the reset observation's goal_images after the first prompt and every restarted prompt, letterboxed to the observation image's size so every image in an episode stacks into one pixel_values tensor.
  • RolloutHarness(restart_keep_turns=k) repeats the last k turns verbatim after a context restart.
  • EnvResponse takes gymnasium terminated and truncated; done is derived from them. TaskAssigner.record_outcome takes a group success outcome, and TaskRowStats reports tied-failure / mixed / tied-success counts.
  • LLMPPO clips actor and critic gradients by their own norms (share_grad_clip=True uses one coefficient from the combined norm). OptimizerStep.clip_coefs replaces clip_coef.
  • LLMPPO critic_warmup_steps trains only the critic for the first N learn steps.
  • FSDP resumes value-head lora_only checkpoints, and the critic LoRA stays trainable after a LoRA load.
  • sync_grads / all_reduce_grads skip params whose grad is None on every rank; ranks that disagree still raise.
  • ModelArch.from_hf_config reads configs that nest the text model under llm_config and give depth only as a per-layer type list.
  • GroupReplayStore keeps per-task trajectories by return so tied rollout groups can be given a different-return member.
  • CosineLRScheduleConfig steps once per learn call: num_epochs is renamed num_steps, with new min_lr_ratio and actor_start_step, and one lr_multiplier(step, start_step) method. The LLMPPO actor's schedule starts after critic_warmup_steps. LLMAlgorithm.current_lr_critic gives the critic rate the next learn trains with.

Full Changelog: v2.48.1...v2.49.0

agilerl-arena/v1.19.0: agilerl-arena v1.19.0: size the fused-pass check from the checkpoint’s config.json

Released on 2026-10-07 - GitHub - PyPI

Fixes

• size the fused-pass check from the checkpoint's config.json

  The auto fused-pass check now builds ModelArch from the checkpoint's raw config.json (PretrainedConfig.get_config_dict) instead of actor.config.to_dict(). transformers 5's NemotronHConfig drops num_hidden_layers, renames hybrid layer types, and can emit MoE defaults for dense checkpoints, which crashed with KeyError: 'num_hidden_layers'.

  CUDA fuse tests write that config.json; a Nemotron Nano 4B config is parsed as 42 layers, 4 attention, 21 Mamba, dense.

Other

• Show action errors and instruction images in LLM env prompts, keep recent turns across context restarts, add PPO critic warmup and per-group gradient clipping

  • RolloutHarness(action_error_field=...) lists an observation field's error text after its action when a context restart summarizes past actions.
  • RolloutHarness puts the reset observation's goal_images after the first prompt and every restarted prompt, letterboxed to the observation image's size so every image in an episode stacks into one pixel_values tensor.
  • RolloutHarness(restart_keep_turns=k) repeats the last k turns verbatim after a context restart.
  • EnvResponse takes gymnasium terminated and truncated; done is derived from them. TaskAssigner.record_outcome takes a group success outcome, and TaskRowStats reports tied-failure / mixed / tied-success counts.
  • LLMPPO clips actor and critic gradients by their own norms (share_grad_clip=True uses one coefficient from the combined norm). OptimizerStep.clip_coefs replaces clip_coef.
  • LLMPPO critic_warmup_steps trains only the critic for the first N learn steps.
  • FSDP resumes value-head lora_only checkpoints, and the critic LoRA stays trainable after a LoRA load.
  • sync_grads / all_reduce_grads skip params whose grad is None on every rank; ranks that disagree still raise.
  • ModelArch.from_hf_config reads configs that nest the text model under llm_config and give depth only as a per-layer type list.
  • GroupReplayStore keeps per-task trajectories by return so tied rollout groups can be given a different-return member.
  • CosineLRScheduleConfig steps once per learn call: num_epochs is renamed num_steps, with new min_lr_ratio and actor_start_step, and one lr_multiplier(step, start_step) method. The LLMPPO actor's schedule starts after critic_warmup_steps. LLMAlgorithm.current_lr_critic gives the critic rate the next learn trains with.

Full Changelog: agilerl-arena/v1.18.0...agilerl-arena/v1.19.0

v2.48.1: size the fused-pass check from the checkpoint’s config.json

Released on 2026-10-07 - GitHub - PyPI

Fixes

• size the fused-pass check from the checkpoint's config.json

  The auto fused-pass check now builds ModelArch from the checkpoint's raw config.json (PretrainedConfig.get_config_dict) instead of actor.config.to_dict(). transformers 5's NemotronHConfig drops num_hidden_layers, renames hybrid layer types, and can emit MoE defaults for dense checkpoints, which crashed with KeyError: 'num_hidden_layers'.

  CUDA fuse tests write that config.json; a Nemotron Nano 4B config is parsed as 42 layers, 4 attention, 21 Mamba, dense.

Full Changelog: v2.48.0...v2.48.1

v2.48.0: run LLMPPO actor and critic as separate or fused passes, picked by memory estimate

Released on 2026-10-07 - GitHub - PyPI

Features

• run LLMPPO actor and critic as separate or fused passes, picked by memory estimate

  • LLMPPO runs the actor (policy and KL loss) and the critic (value loss) as separate forward/backward passes per micro-batch, so peak activation memory is one batch of rows, not two. This covers both the Liger and unfused loss paths.
  • New fuse_actor_critic_pass: bool | None = None on LLMPPO and LLMPPOSpec. True runs one fused pass, False runs two. None fuses when the agilerl.arena.memory estimate of the fused pass fits the GPU, and fuses on non-CUDA devices. The choice is re-resolved on checkpoint load.
  • On the Liger path the critic pass runs under the critic adapter, so the value loss trains the critic LoRA.
  • The critic adapter stays trainable after FSDP2 replaces the model's parameters.
  • With activation_offload=True, the split Liger actor pass also offloads saved activations to CPU.
  • The fused-pass memory estimate uses the packed-expert LoRA path the model actually runs.
  • PPO learn reports actor and critic gradient norms before and after clipping.
  • agilerl-arena memory estimator: TrainingSettings gains fuse_actor_critic_pass (PPO only) and micro_batch_size.
  • agilerl now requires agilerl-arena>=1.18.0.

Full Changelog: v2.47.0...v2.48.0

agilerl-arena/v1.18.0: agilerl-arena v1.18.0: run LLMPPO actor and critic as separate or fused passes, picked by memory estimate

Released on 2026-10-07 - GitHub - PyPI

Features

• run LLMPPO actor and critic as separate or fused passes, picked by memory estimate

  • LLMPPO runs the actor (policy and KL loss) and the critic (value loss) as separate forward/backward passes per micro-batch, so peak activation memory is one batch of rows, not two. This covers both the Liger and unfused loss paths.
  • New fuse_actor_critic_pass: bool | None = None on LLMPPO and LLMPPOSpec. True runs one fused pass, False runs two. None fuses when the agilerl.arena.memory estimate of the fused pass fits the GPU, and fuses on non-CUDA devices. The choice is re-resolved on checkpoint load.
  • On the Liger path the critic pass runs under the critic adapter, so the value loss trains the critic LoRA.
  • The critic adapter stays trainable after FSDP2 replaces the model's parameters.
  • With activation_offload=True, the split Liger actor pass also offloads saved activations to CPU.
  • The fused-pass memory estimate uses the packed-expert LoRA path the model actually runs.
  • PPO learn reports actor and critic gradient norms before and after clipping.
  • agilerl-arena memory estimator: TrainingSettings gains fuse_actor_critic_pass (PPO only) and micro_batch_size.
  • agilerl now requires agilerl-arena>=1.18.0.

Full Changelog: agilerl-arena/v1.17.0...agilerl-arena/v1.18.0

v2.47.0: Nemotron 3.5 Super-VL training with expert and tensor parallel LoRA

Released on 2026-10-06 - GitHub - PyPI

Features

• Nemotron 3.5 Super-VL training with expert and tensor parallel LoRA

  Adds the Nemotron-H / Super-VL architecture, expert-parallel, tensor-parallel and node-local sharded FSDP training, and MoE expert LoRA with grouped GEMM and recompute. Also adds sequence packing that resets Mamba state at document boundaries, old log-probs from rollouts, an episode-balanced loss_norm for GRPO-family learners, and learn profiling. Image episodes record the processor inputs they keep, so a trainer can rebuild pixel_values from the screenshots. Moves LoRA into agilerl.lora.

Full Changelog: v2.46.1...v2.47.0

agilerl-arena/v1.17.0: agilerl-arena v1.17.0: Nemotron 3.5 Super-VL training with expert and tensor parallel LoRA

Released on 2026-10-06 - GitHub - PyPI

Features

• Nemotron 3.5 Super-VL training with expert and tensor parallel LoRA

  Adds the Nemotron-H / Super-VL architecture, expert-parallel, tensor-parallel and node-local sharded FSDP training, and MoE expert LoRA with grouped GEMM and recompute. Also adds sequence packing that resets Mamba state at document boundaries, old log-probs from rollouts, an episode-balanced loss_norm for GRPO-family learners, and learn profiling. Image episodes record the processor inputs they keep, so a trainer can rebuild pixel_values from the screenshots. Moves LoRA into agilerl.lora.

Full Changelog: agilerl-arena/v1.16.0...agilerl-arena/v1.17.0

v2.46.1: closed-form GPU memory estimator

Released on 2026-10-06 - GitHub - PyPI

Features

• closed-form GPU memory estimator

  Peak GPU memory for LLM RL from model geometry, device, and training/generation settings. Two independent phase bars (training and generation never peak at once). No profiling, no fitted correction, no weight download.

  PhaseTimer CPU tests drive a fake clock so they do not depend on Windows sleep resolution.

Full Changelog: v2.46.0...v2.46.1

agilerl-arena/v1.16.0: agilerl-arena v1.16.0: expose MF-PBT evolution planning and hyperparameter reset

Released on 2026-10-06 - GitHub - PyPI

Features

• expose MF-PBT evolution planning and hyperparameter reset

  MultiFrequencySelection.plan_evolution returns one generation as an index plan without cloning agents. apply_hp_reset is public so a migrant can take a destination elite's mutable hyperparameters.

• closed-form GPU memory estimator

  Peak GPU memory for LLM RL from model geometry, device, and training/generation settings. Two independent phase bars (training and generation never peak at once). No profiling, no fitted correction, no weight download.

  PhaseTimer CPU tests drive a fake clock so they do not depend on Windows sleep resolution.

Full Changelog: agilerl-arena/v1.15.0...agilerl-arena/v1.16.0

v2.46.0: add release status to supported models

Released on 2026-10-06 - GitHub - PyPI

Features

• add release status to supported models

  ModelInfo.status marks each supported model live, deprecated, or
  preview, and arena models supported includes it. ArenaClient.submit_experiment
  warns when the manifest uses a deprecated model and rejects a preview one.
  Manifest validation still does not consult status.

  The Super-VL bundled inspected LoRA data includes language-tower dims
  (vision and connector targets stay names-only), so that id has derived LoRA
  ranks and drops latent projections from the allowlist.

• expose MF-PBT evolution planning and hyperparameter reset

  MultiFrequencySelection.plan_evolution returns one generation as an index plan without cloning agents. apply_hp_reset is public so a migrant can take a destination elite's mutable hyperparameters.

Full Changelog: v2.45.0...v2.46.0

agilerl-arena/v1.15.0: agilerl-arena v1.15.0: load every parquet split under a dataset directory

Released on 2026-10-06 - GitHub - PyPI

Features

• load every parquet split under a dataset directory

  A local parquet directory whose files live in immediate child directories now loads every split. When a train split exists alongside another split, training uses every train row and evaluation uses every other split. A file, a flat shard directory, a train-only directory, or split directories with no train directory still use a random holdout. Empty directories still raise.

• add release status to supported models

  ModelInfo.status marks each supported model live, deprecated, or
  preview, and arena models supported includes it. ArenaClient.submit_experiment
  warns when the manifest uses a deprecated model and rejects a preview one.
  Manifest validation still does not consult status.

  The Super-VL bundled inspected LoRA data includes language-tower dims
  (vision and connector targets stay names-only), so that id has derived LoRA
  ranks and drops latent projections from the allowlist.

Full Changelog: agilerl-arena/v1.14.0...agilerl-arena/v1.15.0

v2.45.0: load every parquet split under a dataset directory

Released on 2026-10-06 - GitHub - PyPI

Features

• load every parquet split under a dataset directory

  A local parquet directory whose files live in immediate child directories now loads every split. When a train split exists alongside another split, training uses every train row and evaluation uses every other split. A file, a flat shard directory, a train-only directory, or split directories with no train directory still use a random holdout. Empty directories still raise.

Full Changelog: v2.44.0...v2.45.0

v2.44.0: let LLMEnvSpec declare env sessions and shared services

Released on 2026-10-06 - GitHub - PyPI

Features

• let LLMEnvSpec declare env sessions and shared services

  LLMEnvSpec now accepts env_sessions_per_host, env_session_ports, env_host_memory_limit_bytes, env_host_ready_timeout_s, and env_services so one env_image can run several sessions plus shared backing services. More than one session per host requires env_session_ports. With env_image, env_sessions_per_host defaults to 1 and env_host_ready_timeout_s to 600. EnvServiceSpec and EnvContainerSpec are public models (agilerl.arena.models) and reject unknown keys. Those host settings still require env_image, and dataset environments reject them. The env_image default for cpus_per_env_host is 0.01.

• VisualWebArena env client and multi-turn vision segment learning

  Adds image observations and a vision transcript to the OpenEnv harness, so vision-language models can train on VisualWebArena. Long episodes restart as new segments when the prompt outgrows a token or image budget. GRPO, PPO and REINFORCE for LLMs learn from those episode segments and pass pixel values through the fused log-prob forward and loss. Every LLM algorithm reports learn-phase timings. Adds held-out evaluation fields and adaptive task sampling to the arena env and training models.

Full Changelog: v2.43.0...v2.44.0

agilerl-arena/v1.14.0: agilerl-arena v1.14.0: VisualWebArena env client and multi-turn vision segment learning

Released on 2026-10-06 - GitHub - PyPI

Features

• VisualWebArena env client and multi-turn vision segment learning

  Adds image observations and a vision transcript to the OpenEnv harness, so vision-language models can train on VisualWebArena. Long episodes restart as new segments when the prompt outgrows a token or image budget. GRPO, PPO and REINFORCE for LLMs learn from those episode segments and pass pixel values through the fused log-prob forward and loss. Every LLM algorithm reports learn-phase timings. Adds held-out evaluation fields and adaptive task sampling to the arena env and training models.

Full Changelog: agilerl-arena/v1.13.0...agilerl-arena/v1.14.0

agilerl-arena/v1.13.0: agilerl-arena v1.13.0: let LLMEnvSpec declare env sessions and shared services

Released on 2026-10-06 - GitHub - PyPI

Features

• let LLMEnvSpec declare env sessions and shared services

  LLMEnvSpec now accepts env_sessions_per_host, env_session_ports, env_host_memory_limit_bytes, env_host_ready_timeout_s, and env_services so one env_image can run several sessions plus shared backing services. More than one session per host requires env_session_ports. With env_image, env_sessions_per_host defaults to 1 and env_host_ready_timeout_s to 600. EnvServiceSpec and EnvContainerSpec are public models (agilerl.arena.models) and reject unknown keys. Those host settings still require env_image, and dataset environments reject them. The env_image default for cpus_per_env_host is 0.01.

Full Changelog: agilerl-arena/v1.12.0...agilerl-arena/v1.13.0

v2.43.0: support Qwen/Qwen3.8-27B in agilerl-arena

Released on 2026-10-06 - GitHub - PyPI

Features

• support Qwen/Qwen3.8-27B in agilerl-arena

  Adds Qwen/Qwen3.8-27B to SUPPORTED_MODEL_INFO with its LoRA targets, allowed ranks, and bundled config.json.

Full Changelog: v2.42.5...v2.43.0

agilerl-arena/v1.12.0: agilerl-arena v1.12.0: support Qwen/Qwen3.8-27B in agilerl-arena

Released on 2026-10-06 - GitHub - PyPI

Features

• support Qwen/Qwen3.8-27B in agilerl-arena

  Adds Qwen/Qwen3.8-27B to SUPPORTED_MODEL_INFO with its LoRA targets, allowed ranks, and bundled config.json.

Fixes

• map FSDP checkpoint keys forward through one-to-one renames

  FSDP shard load applies Hugging Face one-to-one weight renames forward over safetensors keys, then looks live parameter names up in that map, matching from_pretrained. Checkpoints that only need part of a family's mapping now load; Nemotron-H stores backbone.embeddings.weight, and undoing both backbone. → model. and embedding.weight → embeddings.weight searched for backbone.embedding.weight. Packed expert shards stacked into one parameter are found under the renamed keys. Split packing converters still resolve through the reverse path. Two checkpoint keys that rename to the same parameter raise an error.

Full Changelog: agilerl-arena/v1.11.1...agilerl-arena/v1.12.0

v2.42.5: point arena CLI login at arena-auth.agilerl.com

Released on 2026-10-05 - GitHub - PyPI

Fixes

• point arena CLI login at arena-auth.agilerl.com

  The default Keycloak URL for arena login was https://auth.arena.agilerl.com, which returns 404 and does not serve the Arena realm. It is now https://arena-auth.agilerl.com. --keycloak-url and ARENA_KEYCLOAK_URL still override the default.

• map FSDP checkpoint keys forward through one-to-one renames

  FSDP shard load applies Hugging Face one-to-one weight renames forward over safetensors keys, then looks live parameter names up in that map, matching from_pretrained. Checkpoints that only need part of a family's mapping now load; Nemotron-H stores backbone.embeddings.weight, and undoing both backbone. → model. and embedding.weight → embeddings.weight searched for backbone.embedding.weight. Packed expert shards stacked into one parameter are found under the renamed keys. Split packing converters still resolve through the reverse path. Two checkpoint keys that rename to the same parameter raise an error.

Full Changelog: v2.42.4...v2.42.5

agilerl-arena/v1.11.1: agilerl-arena v1.11.1: point arena CLI login at arena-auth.agilerl.com

Released on 2026-10-05 - GitHub - PyPI

Fixes

• point arena CLI login at arena-auth.agilerl.com

  The default Keycloak URL for arena login was https://auth.arena.agilerl.com, which returns 404 and does not serve the Arena realm. It is now https://arena-auth.agilerl.com. --keycloak-url and ARENA_KEYCLOAK_URL still override the default.

Full Changelog: agilerl-arena/v1.11.0...agilerl-arena/v1.11.1

agilerl-arena/v1.11.0: agilerl-arena v1.11.0: fold vLLM into agilerl[llm] and lead README with LLM post-training

Released on 2026-10-05 - GitHub - PyPI

Features

• fold vLLM into agilerl[llm] and lead README with LLM post-training

  agilerl[llm] now includes vLLM on Linux. New extra agilerl[cpu-llm] is the Hugging Face, PEFT and datasets stack without vLLM. HAS_LLM_DEPENDENCIES follows [cpu-llm]; vLLM is HAS_VLLM.

  Rewrite the README to lead with LLM post-training: SFT, DPO and RL (GRPO, CISPO, GSPO, REINFORCE, PPO), multi-turn OpenEnv environments, LoRA, FSDP2, memory and model-specific optimizations, and vLLM rollouts. Add sections on training at scale with the Arena CLI (including use from coding agents) and on local LLM training. Shorten the classic RL section, document cpu-llm in the install extras table, and add SFT to the algorithm tables.

• ship supported model info and configs in agilerl-arena

  agilerl.arena.models.SUPPORTED_MODEL_INFO lists supported Hugging Face ids
  with their architecture, LoRA target_modules names, packed-expert
  target_parameters paths, per-target LoRA dims (lora_info), parameter
  counts, and allowed LoRA ranks. Each id bundles one JSON file with its config.json and inspected info;
  everything is read from those. arena models supported
  prints the list as JSON without a server call.

  Manifests for listed ids now reject LoRA ranks above the model's cap and any
  target_modules or target_parameters not in the stored lists, with the valid
  options in the error. target_modules must be all-linear or exact module
  names; regex strings are rejected. On MoE models an unset target_parameters
  adapts every expert and zeros lora_dropout ([] adapts none). Unlisted ids skip these checks. Gemma 4
  example configs use bare projection names. Supported ids resolve their family from the bundled config with no Hub call. agilerl now requires agilerl-arena 1.11.0 or newer.

Fixes

• re-tie output head and recompute RoPE after meta load

  to_empty on the FSDP meta-device load path splits a tied lm_head / embed_out from embed_tokens. Tied checkpoints omit the head tensor, so the head stays empty and LoRA gradients are zero. Call tie_weights() on the module whose registered children include the head (walking through PEFT and value-head wrappers) and raise if a tied key is still a separate parameter. Non-persistent RoPE inv_freq buffers are not stored in safetensors; recompute them the same way Hugging Face _init_weights does.

  Pin COVERAGE_FILE to an absolute path so pytest's session chdir cannot drop coverage hits, and combine parallel coverage DBs from Linux 3.13 shards.

• Gemma 4 vLLM startup, FSDP scalar buffers, L4 flex tiles

  Gemma 4 text_config has no architectures list. Stripping multimodal towers now maps that nested config to Gemma4ForCausalLM so vLLM can load the language engine. Tower-connector LoRA stays off for Gemma 4; only families that implement encoder token counts enable it. FSDP shard load uses get_tensor for full buffer copies so 0-dim buffers (clipped-linear clamp bounds) load. Flex attention on L4-class GPUs uses smaller backward tiles to fit their ~99 KB shared memory.

• load Nemotron-H trainers with the transformers class

  Nemotron-H trainer runtime no longer sets trust_remote_code. Hugging Face transformers then uses its own nemotron_h class instead of the checkpoint auto_map code, which only allows eager attention. nemotron_h_omni still sets trust_remote_code because transformers does not ship that model type. vLLM still sets trust_remote_code for both families.

Full Changelog: agilerl-arena/v1.10.0...agilerl-arena/v1.11.0

v2.42.3: load Nemotron-H trainers with the transformers class

Released on 2026-10-05 - GitHub - PyPI

Fixes

• load Nemotron-H trainers with the transformers class

  Nemotron-H trainer runtime no longer sets trust_remote_code. Hugging Face transformers then uses its own nemotron_h class instead of the checkpoint auto_map code, which only allows eager attention. nemotron_h_omni still sets trust_remote_code because transformers does not ship that model type. vLLM still sets trust_remote_code for both families.

Full Changelog: v2.42.2...v2.42.3

v2.42.2: Gemma 4 vLLM startup, FSDP scalar buffers, L4 flex tiles

Released on 2026-10-05 - GitHub - PyPI

Fixes

• Gemma 4 vLLM startup, FSDP scalar buffers, L4 flex tiles

  Gemma 4 text_config has no architectures list. Stripping multimodal towers now maps that nested config to Gemma4ForCausalLM so vLLM can load the language engine. Tower-connector LoRA stays off for Gemma 4; only families that implement encoder token counts enable it. FSDP shard load uses get_tensor for full buffer copies so 0-dim buffers (clipped-linear clamp bounds) load. Flex attention on L4-class GPUs uses smaller backward tiles to fit their ~99 KB shared memory.

Full Changelog: v2.42.1...v2.42.2

v2.42.1: re-tie output head and recompute RoPE after meta load

Released on 2026-10-02 - GitHub - PyPI

Fixes

• re-tie output head and recompute RoPE after meta load

  to_empty on the FSDP meta-device load path splits a tied lm_head / embed_out from embed_tokens. Tied checkpoints omit the head tensor, so the head stays empty and LoRA gradients are zero. Call tie_weights() on the module whose registered children include the head (walking through PEFT and value-head wrappers) and raise if a tied key is still a separate parameter. Non-persistent RoPE inv_freq buffers are not stored in safetensors; recompute them the same way Hugging Face _init_weights does.

  Pin COVERAGE_FILE to an absolute path so pytest's session chdir cannot drop coverage hits, and combine parallel coverage DBs from Linux 3.13 shards.

Full Changelog: v2.42.0...v2.42.1

v2.42.0: fold vLLM into agilerl[llm] and lead README with LLM post-training

Released on 2026-10-02 - GitHub - PyPI

Features

• fold vLLM into agilerl[llm] and lead README with LLM post-training

  agilerl[llm] now includes vLLM on Linux. New extra agilerl[cpu-llm] is the Hugging Face, PEFT and datasets stack without vLLM. HAS_LLM_DEPENDENCIES follows [cpu-llm]; vLLM is HAS_VLLM.

  Rewrite the README to lead with LLM post-training: SFT, DPO and RL (GRPO, CISPO, GSPO, REINFORCE, PPO), multi-turn OpenEnv environments, LoRA, FSDP2, memory and model-specific optimizations, and vLLM rollouts. Add sections on training at scale with the Arena CLI (including use from coding agents) and on local LLM training. Shorten the classic RL section, document cpu-llm in the install extras table, and add SFT to the algorithm tables.

Fixes

• emit wizard schema defaults and drop unused form fields

  Served /manifest/schema now includes numeric evo_steps defaults (LLM 20, classic 160000, CNN/MultiInput 320000), inlines lora_config Rank/Alpha/Dropout (1/32/0.05), omits learning_delay from on-policy training, strips unused answer_pattern, and gates Ornstein-Uhlenbeck theta/dt with if/then.

Full Changelog: v2.41.1...v2.42.0

agilerl-arena/v1.10.0: agilerl-arena v1.10.0: onboard gpt-oss-20b with flex attention and packed-expert LoRA

Released on 2026-10-01 - GitHub - PyPI

Features

• onboard gpt-oss-20b with flex attention and packed-expert LoRA

  Family trainer attention defaults (Gemma SWA / gpt-oss Flex Attention) beat YAML and spec values; ATTN_IMPLEMENTATION still overrides. Packed-expert LoRA (target_parameters) forces lora_dropout=0 because PEFT cannot factor dropout out of the parameter-level product. GptOssExperts uses a split-LoRA forward for its matmul layout, biases, and interleaved SwiGLU. The grouped-GEMM capability probe no longer adds saved tensors to an activation checkpoint and no longer reports unsupported when first called under no_grad.

• load parquet files or shard directories into one Dataset

  Add load_parquet_dataset for a local .parquet/.pq file or a directory of *.parquet shards. Shards concatenate as Hugging Face datasets in filename order; mismatched schemas raise. make_llm_env uses this for parquet files and directories that contain *.parquet; other paths still load as a Hub id or local Hugging Face dataset.

• log sampled rollout completions during training

  train_llm_rollout takes a new completion_logging=CompletionLoggingConfig(...) argument. When set, each iteration samples prompt groups from the rollout batch and decodes prompt, completion, reward, turn count and completion-token count. Every interval iterations the samples from every population member go in one write to the console (verbose), a W&B completions table (wb) and an optional JSONL file (jsonl_path). Text fields are capped at max_chars, keeping head and tail. The latest samples are also logged at error level if training raises.

  The building blocks (CompletionLogger, CompletionRecord, build_completion_record and the writers) live in agilerl.training.llm.completion_logging for custom training loops.

• sample trajectories per group and mark truncated logs

  CompletionLoggingConfig gains samples_per_group: when set, each sampled prompt group logs that many random trajectories instead of the whole group (unset keeps the previous whole-group behaviour). Over-long prompt/completion text is still head/tail truncated, now with a <LOG TRUNCATED: N chars> seam marker.

Fixes

• release device memory after evolutionary mutation

  Add release_device_memory() and call it from Mutations.mutation and from
  clean_up() so MPS/CUDA caches from cloning and architecture changes do not
  accumulate across generations. Document architecture-mutation RSS on CPU and
  custom encoder get_init_dict requirements in the mutation guide.

• emit wizard schema defaults and drop unused form fields

  Served /manifest/schema now includes numeric evo_steps defaults (LLM 20, classic 160000, CNN/MultiInput 320000), inlines lora_config Rank/Alpha/Dropout (1/32/0.05), omits learning_delay from on-policy training, strips unused answer_pattern, and gates Ornstein-Uhlenbeck theta/dt with if/then.

Other

• save duration maps on the Linux test runners

  CI durations writes the Linux CPU timing map on ubuntu-24.04 and the GPU map on the self-hosted scale-set, matching the jobs that restore them.

• run a single macOS job on Python 3.13

  GitHub-hosted macOS CI runs one job (macos-26, Python 3.13) instead of four Python versions.

• bump GitHub Actions off Node 20 and CodeQL Action v3

  Put the CodeQL job steps in .github/workflows/codeql.yml so that file has
  on.push (CI still calls it as a merge gate; push to main and the weekly
  schedule still feed the Security tab). Use actions/checkout@v5 and
  github/codeql-action v4.

• shard macOS and Windows pytest with duration maps

  macOS runs three duration-balanced shards and Windows two. Timing maps are stored on the default branch and restored on the same runner family that wrote them. Before a map exists, shards are packed with equal-weight LPT rather than contiguous slices.

• refresh Python lockfile

  Regenerates the Python lockfile after dependency updates.

  Applies Liger Kernel patches to the module returned by PEFT's
  get_base_model().

Full Changelog: agilerl-arena/v1.9.4...agilerl-arena/v1.10.0

v2.41.1: refresh Python lockfile

Released on 2026-09-30 - GitHub - PyPI

Other

• refresh Python lockfile

  Regenerates the Python lockfile after dependency updates.

  Applies Liger Kernel patches to the module returned by PEFT's
  get_base_model().

Full Changelog: v2.41.0...v2.41.1

v2.41.0: sample trajectories per group and mark truncated logs

Released on 2026-09-30 - GitHub - PyPI

Features

• sample trajectories per group and mark truncated logs

  CompletionLoggingConfig gains samples_per_group: when set, each sampled prompt group logs that many random trajectories instead of the whole group (unset keeps the previous whole-group behaviour). Over-long prompt/completion text is still head/tail truncated, now with a <LOG TRUNCATED: N chars> seam marker.

Full Changelog: v2.40.1...v2.41.0

v2.40.1: shard macOS and Windows pytest with duration maps

Released on 2026-09-30 - GitHub - PyPI

Other

• shard macOS and Windows pytest with duration maps

  macOS runs three duration-balanced shards and Windows two. Timing maps are stored on the default branch and restored on the same runner family that wrote them. Before a map exists, shards are packed with equal-weight LPT rather than contiguous slices.

Full Changelog: v2.40.0...v2.40.1

v2.40.0: log sampled rollout completions during training

Released on 2026-09-30 - GitHub - PyPI

Features

• log sampled rollout completions during training

  train_llm_rollout takes a new completion_logging=CompletionLoggingConfig(...) argument. When set, each iteration samples prompt groups from the rollout batch and decodes prompt, completion, reward, turn count and completion-token count. Every interval iterations the samples from every population member go in one write to the console (verbose), a W&B completions table (wb) and an optional JSONL file (jsonl_path). Text fields are capped at max_chars, keeping head and tail. The latest samples are also logged at error level if training raises.

  The building blocks (CompletionLogger, CompletionRecord, build_completion_record and the writers) live in agilerl.training.llm.completion_logging for custom training loops.

Other

• bump GitHub Actions off Node 20 and CodeQL Action v3

  Put the CodeQL job steps in .github/workflows/codeql.yml so that file has
  on.push (CI still calls it as a merge gate; push to main and the weekly
  schedule still feed the Security tab). Use actions/checkout@v5 and
  github/codeql-action v4.

Full Changelog: v2.39.0...v2.40.0

v2.39.0: load parquet files or shard directories into one Dataset

Released on 2026-09-30 - GitHub - PyPI

Features

• load parquet files or shard directories into one Dataset

  Add load_parquet_dataset for a local .parquet/.pq file or a directory of *.parquet shards. Shards concatenate as Hugging Face datasets in filename order; mismatched schemas raise. make_llm_env uses this for parquet files and directories that contain *.parquet; other paths still load as a Hub id or local Hugging Face dataset.

Full Changelog: v2.38.1...v2.39.0

v2.38.1: release device memory after evolutionary mutation

Released on 2026-09-30 - GitHub - PyPI

Fixes

• release device memory after evolutionary mutation

  Add release_device_memory() and call it from Mutations.mutation and from
  clean_up() so MPS/CUDA caches from cloning and architecture changes do not
  accumulate across generations. Document architecture-mutation RSS on CPU and
  custom encoder get_init_dict requirements in the mutation guide.

Full Changelog: v2.38.0...v2.38.1

v2.38.0: onboard gpt-oss-20b with flex attention and packed-expert LoRA

Released on 2026-09-29 - GitHub - PyPI

Features

• onboard gpt-oss-20b with flex attention and packed-expert LoRA

  Family trainer attention defaults (Gemma SWA / gpt-oss Flex Attention) beat YAML and spec values; ATTN_IMPLEMENTATION still overrides. Packed-expert LoRA (target_parameters) forces lora_dropout=0 because PEFT cannot factor dropout out of the parameter-level product. GptOssExperts uses a split-LoRA forward for its matmul layout, biases, and interleaved SwiGLU. The grouped-GEMM capability probe no longer adds saved tensors to an activation checkpoint and no longer reports unsupported when first called under no_grad.

Other

• save duration maps on the Linux test runners

  CI durations writes the Linux CPU timing map on ubuntu-24.04 and the GPU map on the self-hosted scale-set, matching the jobs that restore them.

• run a single macOS job on Python 3.13

  GitHub-hosted macOS CI runs one job (macos-26, Python 3.13) instead of four Python versions.

Full Changelog: v2.37.0...v2.38.0

v2.37.0: publish training_gpus_per_agent only on LLM training schemas

Released on 2026-09-29 - GitHub - PyPI

Fixes

• publish training_gpus_per_agent only on LLM training schemas

  The published agilerl-arena training-manifest schema now defines training_gpus_per_agent only on the LLM training spec. Classic RL and multi-agent gym algorithms no longer expose that key.

  to_payload() omits it for non-LLM runs, and for LLM runs that leave the default unset.

Other

• merge per-root junit as well-formed XML

  Two-root pytest shards no longer concatenate <testsuites> wrappers with string slicing. The merged junit report is built with ElementTree so duration promote can parse it.

  Cover stripping non-form algorithm fields from JSON Schema required lists.

• run Linux CPU, ty, and coverage on GitHub-hosted Ubuntu

  Linux CPU pytest, ty, and coverage combine run on ubuntu-24.04 without CUDA torch or vLLM. GPU pytest stays on self-hosted gha-runner-scale-set runners. The llm extra is Hugging Face transformers/PEFT/datasets plus Liger and bitsandbytes on Linux. vLLM lives in the vllm extra. The cpu extra selects CPU-only PyTorch.

Full Changelog: v2.36.2...v2.37.0

agilerl-arena/v1.9.4: agilerl-arena v1.9.4: run Linux CPU, ty, and coverage on GitHub-hosted Ubuntu

Released on 2026-09-29 - GitHub - PyPI

Other

• run Linux CPU, ty, and coverage on GitHub-hosted Ubuntu

  Linux CPU pytest, ty, and coverage combine run on ubuntu-24.04 without CUDA torch or vLLM. GPU pytest stays on self-hosted gha-runner-scale-set runners. The llm extra is Hugging Face transformers/PEFT/datasets plus Liger and bitsandbytes on Linux. vLLM lives in the vllm extra. The cpu extra selects CPU-only PyTorch.

Full Changelog: agilerl-arena/v1.9.3...agilerl-arena/v1.9.4

agilerl-arena/v1.9.3: agilerl-arena v1.9.3: publish training_gpus_per_agent only on LLM training schemas

Released on 2026-09-29 - GitHub - PyPI

Fixes

• publish training_gpus_per_agent only on LLM training schemas

  The published agilerl-arena training-manifest schema now defines training_gpus_per_agent only on the LLM training spec. Classic RL and multi-agent gym algorithms no longer expose that key.

  to_payload() omits it for non-LLM runs, and for LLM runs that leave the default unset.

Full Changelog: agilerl-arena/v1.9.2...agilerl-arena/v1.9.3

agilerl-arena/v1.9.2: agilerl-arena v1.9.2: merge per-root junit as well-formed XML

Released on 2026-09-29 - GitHub - PyPI

Other

• merge per-root junit as well-formed XML

  Two-root pytest shards no longer concatenate <testsuites> wrappers with string slicing. The merged junit report is built with ElementTree so duration promote can parse it.

  Cover stripping non-form algorithm fields from JSON Schema required lists.

Full Changelog: agilerl-arena/v1.9.1...agilerl-arena/v1.9.2

v2.36.2: publish dataset, epsilon, and SFT beta defaults only for runs that use them

Released on 2026-09-28 - GitHub - PyPI

Fixes

• publish dataset, epsilon, and SFT beta defaults only for runs that use them

  LLM environment fields train_test_split, response_column, rubric_name, strict_chat_template_boundary, and num_envs are optional on LLMEnvSpec. Dataset-backed rollouts still get the train/test split and rubric_name. SFT still gets the split and response_column. GEM-style rollouts still get num_envs and strict_chat_template_boundary. make_llm_env requires response_column; dataset loaders require train_test_split; an unset rollout chat-template boundary is treated as strict. Rainbow DQN no longer publishes epsilon-greedy training defaults. The SFT algorithm schema no longer includes beta; DPO still defaults it to 0.1.

Other

• fail closed when duration promote has no junit

  CI durations now fails if a successful CI run has no 3.13 junit XML or an empty duration map, instead of skipping the cache save and still going green. merge-junit finds XML under the download directory itself.

Full Changelog: v2.36.1...v2.36.2

agilerl-arena/v1.9.1: agilerl-arena v1.9.1: publish dataset, epsilon, and SFT beta defaults only for runs that use them

Released on 2026-09-28 - GitHub - PyPI

Fixes

• publish dataset, epsilon, and SFT beta defaults only for runs that use them

  LLM environment fields train_test_split, response_column, rubric_name, strict_chat_template_boundary, and num_envs are optional on LLMEnvSpec. Dataset-backed rollouts still get the train/test split and rubric_name. SFT still gets the split and response_column. GEM-style rollouts still get num_envs and strict_chat_template_boundary. make_llm_env requires response_column; dataset loaders require train_test_split; an unset rollout chat-template boundary is treated as strict. Rainbow DQN no longer publishes epsilon-greedy training defaults. The SFT algorithm schema no longer includes beta; DPO still defaults it to 0.1.

Other

• split Linux pytest into duration-balanced CPU and GPU shards

  Linux GitHub Actions runs CPU tests (not gpu and not vllm, CUDA hidden) and GPU/vLLM tests as two duration-balanced shards each per Python 3.10–3.13 via scripts/pytest_shard.py. A green CI run promotes 3.13 junit timings onto default-branch Actions caches so the next PR or dispatch can pack from measured times. Each shard appends coverage across tests/ and agilerl-arena/tests and merges their junit reports. A root with no matching tests is an empty shard, not a failure. Combine requires both 3.13 CPU shards and both GPU shards before writing framework-coverage.json. Coverage reports store paths relative to the repo root. macOS and Windows run the full suite and upload junit. Latent-encoder function-preserving checks allow float32 GEMM noise (rtol=1e-5).

• fail closed when duration promote has no junit

  CI durations now fails if a successful CI run has no 3.13 junit XML or an empty duration map, instead of skipping the cache save and still going green. merge-junit finds XML under the download directory itself.

Full Changelog: agilerl-arena/v1.9.0...agilerl-arena/v1.9.1

v2.36.1: split Linux pytest into duration-balanced CPU and GPU shards

Released on 2026-09-28 - GitHub - PyPI

Other

• split Linux pytest into duration-balanced CPU and GPU shards

  Linux GitHub Actions runs CPU tests (not gpu and not vllm, CUDA hidden) and GPU/vLLM tests as two duration-balanced shards each per Python 3.10–3.13 via scripts/pytest_shard.py. A green CI run promotes 3.13 junit timings onto default-branch Actions caches so the next PR or dispatch can pack from measured times. Each shard appends coverage across tests/ and agilerl-arena/tests and merges their junit reports. A root with no matching tests is an empty shard, not a failure. Combine requires both 3.13 CPU shards and both GPU shards before writing framework-coverage.json. Coverage reports store paths relative to the repo root. macOS and Windows run the full suite and upload junit. Latent-encoder function-preserving checks allow float32 GEMM noise (rtol=1e-5).

Full Changelog: v2.36.0...v2.36.1

v2.36.0: Train vision-language models with image observations and vision-tower LoRA

Released on 2026-09-27 - GitHub - PyPI

Other

• Train vision-language models with image observations and vision-tower LoRA

  Image turns pass pixel values through rollout and into GRPO. LoRA adapter keys for the vision encoder and projector are remapped onto the module names used for inference. Vision weights load from the checkpoint conversion registered on the vision module.

Full Changelog: v2.35.0...v2.36.0

agilerl-arena/v1.9.0: agilerl-arena v1.9.0: Train vision-language models with image observations and vision-tower LoRA

Released on 2026-09-27 - GitHub - PyPI

Other

• Train vision-language models with image observations and vision-tower LoRA

  Image turns pass pixel values through rollout and into GRPO. LoRA adapter keys for the vision encoder and projector are remapped onto the module names used for inference. Vision weights load from the checkpoint conversion registered on the vision module.

Full Changelog: agilerl-arena/v1.8.0...agilerl-arena/v1.9.0

v2.35.0: train Nemotron Super VL on the FSDP language tower

Released on 2026-09-27 - GitHub - PyPI

Features

• train Nemotron Super VL on the FSDP language tower

  Catalog nemotron_h_omni and serve the language tower as NemotronHOmniLanguageForCausalLM. FSDP finds that tower under language_model, including a backbone body, and uses it for wrap units, gradient checkpointing, and lm_head. trust_remote_code comes from the family catalog. model_type is read from config.json.

Full Changelog: v2.34.1...v2.35.0

agilerl-arena/v1.8.0: agilerl-arena v1.8.0: train Nemotron Super VL on the FSDP language tower

Released on 2026-09-27 - GitHub - PyPI

Features

• train Nemotron Super VL on the FSDP language tower

  Catalog nemotron_h_omni and serve the language tower as NemotronHOmniLanguageForCausalLM. FSDP finds that tower under language_model, including a backbone body, and uses it for wrap units, gradient checkpointing, and lm_head. trust_remote_code comes from the family catalog. model_type is read from config.json.

Full Changelog: agilerl-arena/v1.7.0...agilerl-arena/v1.8.0

v2.34.1: give Rainbow DQN and Recurrent PPO their own schema names and shortlist defaults

Released on 2026-09-27 - GitHub - PyPI

Features

• give Rainbow DQN and Recurrent PPO their own schema names and shortlist defaults

  Rainbow DQN and Recurrent PPO are distinct agilerl-arena algorithm specs. Recurrent PPO keeps recurrent on; a payload that turns it off is rejected.

  Manifest defaults now match the public shortlist: DQN double, PPO and IPPO action_std_init 0.6, Rainbow value support [-10, 10], LLM use_liger_loss, LLMPPO gae_lambda 0.95, LLMPPO and LLMREINFORCE beta 0.001, gym num_envs 32, evolvable latent_dim 128, and episode_steps left unset.

  Algorithm constructors use the same defaults, so an omitted field means the same locally and on Arena.

  The published JSON Schema lists algorithm-conditional training defaults on the manifest root (if algorithm.name, then training.properties): DQN epsilon, LLM reporting_interval 1, LLMPPO/LLMREINFORCE max_steps 200, SFT/DPO num_epochs 1, and off-policy experience_sharing. A PPO document does not pick up the LLMPPO or off-policy defaults.

  Bandit trainer kwargs omit unset episode_steps so the loop keeps its 500 default.

Full Changelog: v2.34.0...v2.34.1

agilerl-arena/v1.7.0: agilerl-arena v1.7.0: give Rainbow DQN and Recurrent PPO their own schema names and shortlist defaults

Released on 2026-09-27 - GitHub - PyPI

Features

• give Rainbow DQN and Recurrent PPO their own schema names and shortlist defaults

  Rainbow DQN and Recurrent PPO are distinct agilerl-arena algorithm specs. Recurrent PPO keeps recurrent on; a payload that turns it off is rejected.

  Manifest defaults now match the public shortlist: DQN double, PPO and IPPO action_std_init 0.6, Rainbow value support [-10, 10], LLM use_liger_loss, LLMPPO gae_lambda 0.95, LLMPPO and LLMREINFORCE beta 0.001, gym num_envs 32, evolvable latent_dim 128, and episode_steps left unset.

  Algorithm constructors use the same defaults, so an omitted field means the same locally and on Arena.

  The published JSON Schema lists algorithm-conditional training defaults on the manifest root (if algorithm.name, then training.properties): DQN epsilon, LLM reporting_interval 1, LLMPPO/LLMREINFORCE max_steps 200, SFT/DPO num_epochs 1, and off-policy experience_sharing. A PPO document does not pick up the LLMPPO or off-policy defaults.

  Bandit trainer kwargs omit unset episode_steps so the loop keeps its 500 default.

Full Changelog: agilerl-arena/v1.6.0...agilerl-arena/v1.7.0

v2.34.0: split env factory from entrypoint in the training manifest

Released on 2026-09-25 - GitHub - PyPI

Features

• split env factory from entrypoint in the training manifest

  environment.entrypoint is the env to build. Optional environment.factory is the callable that receives that entrypoint. env_config is leftover constructor kwargs only; env_id inside it is rejected.

  GymEnvSpec and LLMEnvSpec gain factory. Construction is factory(entrypoint, **env_config) when factory is set, otherwise entrypoint(**env_config). LLM name is a catalog label (use .label for display), not a dataset source. Gym still accepts name: CartPole-v1 with implied gymnasium.make. agilerl.llm_envs.env_specs is now agilerl.llm_envs.env_sources.

Full Changelog: v2.33.0...v2.34.0

agilerl-arena/v1.6.0: agilerl-arena v1.6.0: split env factory from entrypoint in the training manifest

Released on 2026-09-25 - GitHub - PyPI

Features

• split env factory from entrypoint in the training manifest

  environment.entrypoint is the env to build. Optional environment.factory is the callable that receives that entrypoint. env_config is leftover constructor kwargs only; env_id inside it is rejected.

  GymEnvSpec and LLMEnvSpec gain factory. Construction is factory(entrypoint, **env_config) when factory is set, otherwise entrypoint(**env_config). LLM name is a catalog label (use .label for display), not a dataset source. Gym still accepts name: CartPole-v1 with implied gymnasium.make. agilerl.llm_envs.env_specs is now agilerl.llm_envs.env_sources.

Full Changelog: agilerl-arena/v1.5.0...agilerl-arena/v1.6.0

v2.33.0: shard LLM trainers with PyTorch FSDP2

Released on 2026-09-25 - GitHub - PyPI

Features

• shard LLM trainers with PyTorch FSDP2

  LLM trainers shard the actor with torch.distributed and PyTorch FSDP2 (fully_shard). Omit algorithm.fsdp / fsdp_config for flat data-parallel replicas.

  Rank, device, and collective helpers live in agilerl.distributed. Import gather_tensor, aggregate_metrics_across_gpus, and aggregate_metrics_dict from there; they are no longer in agilerl.utils.llm_utils, and safe_aggregate_metrics is removed.

  LLM algorithms use DPRuntime (full replica per rank) without fsdp_config and FSDPRuntime with it. Wrap units are outermost HuggingFace _no_split_modules, then an untied lm_head, then the root. LoRA adapters stay replicated. Optimizer states can live on CPU via optim_cpu_offload or cpu_offload.
  FSDPConfig lives in agilerl.arena.models.fsdp so agilerl-arena installs without torch; agilerl.distributed.FSDPConfig still works.
  agilerl.utils.llm_utils.get_state_dict is removed; use the algorithm's shard_runtime.export_model_state.
  FSDPConfig fields carry descriptions, and the manifest JSON schema for algorithm.fsdp is generated from them.
  agilerl train --use-accelerator is removed; accelerate launch is detected automatically.
  replay_buffer.buffer_occupancy_multiplier is removed from LLM rollout buffer specs; it had no effect.
  use_memory_efficient_params (INIT_HP USE_MEMORY_EFFICIENT_PARAMS) is renamed to offload_trainer_during_rollout (OFFLOAD_TRAINER_DURING_ROLLOUT).
  Fused logprob helpers take head_w / head_b instead of lm_head_weight / lm_head_bias.
  AutoModelForCausalLMWithValueHead.state_dict() now returns the standard module state dict (pretrained_model.* and v_head.*) for PEFT and non-PEFT backbones. save_pretrained output is unchanged.
  BaseRuntime.backward returns an OptimizerStep (pre/post-clip gradient norms and the new learning rate) on step boundaries, else None.
  .population() copies LoRA agents with clone=True so adapters are not re-attached onto a PeftModel.

Full Changelog: v2.32.0...v2.33.0

agilerl-arena/v1.5.0: agilerl-arena v1.5.0: per-learn LLM telemetry

Released on 2026-09-25 - GitHub - PyPI

Features

• per-learn LLM telemetry

  GRPO learn now reports per-learn advantage stats, policy entropy, K3 KL vs reference and vs rollout policy, importance-ratio tails with advantage-signed clip fractions, and grad norm pre/post clip; vLLM sampling-mismatch stats reach the metrics tracker (also fixed in PPO/REINFORCE). The fused-kernel aux is split into fixed kl (NaN at beta=0) and clipfrac keys, replacing aux_metric_name. Two correctness fixes: the K3 helper had its arguments flipped vs Schulman/TRL/Liger (Liger runs were unaffected — the kernel computes its own KL), and standard-path CISPO now clamps from above only like the kernel.

• shard LLM trainers with PyTorch FSDP2

  LLM trainers shard the actor with torch.distributed and PyTorch FSDP2 (fully_shard). Omit algorithm.fsdp / fsdp_config for flat data-parallel replicas.

  Rank, device, and collective helpers live in agilerl.distributed. Import gather_tensor, aggregate_metrics_across_gpus, and aggregate_metrics_dict from there; they are no longer in agilerl.utils.llm_utils, and safe_aggregate_metrics is removed.

  LLM algorithms use DPRuntime (full replica per rank) without fsdp_config and FSDPRuntime with it. Wrap units are outermost HuggingFace _no_split_modules, then an untied lm_head, then the root. LoRA adapters stay replicated. Optimizer states can live on CPU via optim_cpu_offload or cpu_offload.
  FSDPConfig lives in agilerl.arena.models.fsdp so agilerl-arena installs without torch; agilerl.distributed.FSDPConfig still works.
  agilerl.utils.llm_utils.get_state_dict is removed; use the algorithm's shard_runtime.export_model_state.
  FSDPConfig fields carry descriptions, and the manifest JSON schema for algorithm.fsdp is generated from them.
  agilerl train --use-accelerator is removed; accelerate launch is detected automatically.
  replay_buffer.buffer_occupancy_multiplier is removed from LLM rollout buffer specs; it had no effect.
  use_memory_efficient_params (INIT_HP USE_MEMORY_EFFICIENT_PARAMS) is renamed to offload_trainer_during_rollout (OFFLOAD_TRAINER_DURING_ROLLOUT).
  Fused logprob helpers take head_w / head_b instead of lm_head_weight / lm_head_bias.
  AutoModelForCausalLMWithValueHead.state_dict() now returns the standard module state dict (pretrained_model.* and v_head.*) for PEFT and non-PEFT backbones. save_pretrained output is unchanged.
  BaseRuntime.backward returns an OptimizerStep (pre/post-clip gradient norms and the new learning rate) on step boundaries, else None.
  .population() copies LoRA agents with clone=True so adapters are not re-attached onto a PeftModel.

Fixes

• close non-LLM training resources and stop clone memory leaks

  Training loops now finish loggers, progress bars, and environments when training returns. BanditEnv and DatasetEnv implement close().

  Tournament selection and MF-PBT free evicted agents. clone() skips rollout buffers, copies GraMa scores, and restores the parent Accelerate wrap when wrap=True. Bandit clones trim regret history.

  minari_to_agile_dataset writes HDF5 and returns the file path. Offline training closes HDF5 handles after loading transitions; in-memory array mappings are unchanged.

  test() puts networks back in training mode after a successful evaluation.

• pin vllm extra to 0.25.1

  Pin the Linux llm extra to vLLM 0.25.1. vLLM 0.26 and later crash on Nemotron-H hybrid models when CUDA graphs are enabled.

Full Changelog: agilerl-arena/v1.4.0...agilerl-arena/v1.5.0

v2.32.0: per-learn LLM telemetry

Released on 2026-09-24 - GitHub - PyPI

Features

• per-learn LLM telemetry

  GRPO learn now reports per-learn advantage stats, policy entropy, K3 KL vs reference and vs rollout policy, importance-ratio tails with advantage-signed clip fractions, and grad norm pre/post clip; vLLM sampling-mismatch stats reach the metrics tracker (also fixed in PPO/REINFORCE). The fused-kernel aux is split into fixed kl (NaN at beta=0) and clipfrac keys, replacing aux_metric_name. Two correctness fixes: the K3 helper had its arguments flipped vs Schulman/TRL/Liger (Liger runs were unaffected — the kernel computes its own KL), and standard-path CISPO now clamps from above only like the kernel.

Full Changelog: v2.31.0...v2.32.0

v2.31.0: pin vllm extra to 0.25.1

Released on 2026-09-22 - GitHub - PyPI

Fixes

• pin vllm extra to 0.25.1

  Pin the Linux llm extra to vLLM 0.25.1. vLLM 0.26 and later crash on Nemotron-H hybrid models when CUDA graphs are enabled.

Full Changelog: v2.30.0...v2.31.0

v2.30.0: close non-LLM training resources and stop clone memory leaks

Released on 2026-09-18 - GitHub - PyPI

Fixes

• close non-LLM training resources and stop clone memory leaks

  Training loops now finish loggers, progress bars, and environments when training returns. BanditEnv and DatasetEnv implement close().

  Tournament selection and MF-PBT free evicted agents. clone() skips rollout buffers, copies GraMa scores, and restores the parent Accelerate wrap when wrap=True. Bandit clones trim regret history.

  minari_to_agile_dataset writes HDF5 and returns the file path. Offline training closes HDF5 handles after loading transitions; in-memory array mappings are unchanged.

  test() puts networks back in training mode after a successful evaluation.

Full Changelog: v2.29.0...v2.30.0

v2.29.0: reject unknown fields on live pydantic models

Released on 2026-09-18 - GitHub - PyPI

Fixes

• reject unknown fields on live pydantic models

  Unknown keys on framework runtime configs now raise a validation error instead of being dropped (VllmRuntimeConfig, TrainerRuntimeConfig, MambaPatchConfig, PatchRuntimeConfig, ModelRuntimeConfig). Arena inference request and status DTOs (LLMParams, AgentInfo, StatusResponse, LLMResults, SessionMessage) do the same. PredictResult and SessionInfo ignore extra keys because they parse a subset of the inference JSON (results on /predict; title / created_by on /sessions).

Full Changelog: v2.28.0...v2.29.0

agilerl-arena/v1.4.0: agilerl-arena v1.4.0: reject unknown fields on live pydantic models

Released on 2026-09-18 - GitHub - PyPI

Fixes

• reject unknown fields on live pydantic models

  Unknown keys on framework runtime configs now raise a validation error instead of being dropped (VllmRuntimeConfig, TrainerRuntimeConfig, MambaPatchConfig, PatchRuntimeConfig, ModelRuntimeConfig). Arena inference request and status DTOs (LLMParams, AgentInfo, StatusResponse, LLMResults, SessionMessage) do the same. PredictResult and SessionInfo ignore extra keys because they parse a subset of the inference JSON (results on /predict; title / created_by on /sessions).

Full Changelog: agilerl-arena/v1.3.0...agilerl-arena/v1.4.0

v2.28.0: drop use_vllm from the training spec

Released on 2026-09-17 - GitHub - PyPI

Features

• drop use_vllm from the training spec

  Training YAML and the algorithm constructor no longer have use_vllm. Colocated rollout (training.rollout_mode) fills a default vllm_config when it is unset; that config is what starts the in-process engine. Example GRPO / PPO / REINFORCE / GSPO / CISPO configs and the remote env tutorial drop the flag.

Full Changelog: v2.27.1...v2.28.0

agilerl-arena/v1.3.0: agilerl-arena v1.3.0: resolve trainer attention in one helper

Released on 2026-09-17 - GitHub - PyPI

Features

• resolve trainer attention in one helper

  resolve_attn_implementation now picks the trainer attention backend in one place. Order: an explicit value, then ATTN_IMPLEMENTATION / AGILERL_ATTN_IMPLEMENTATION, then the family trainer default (Gemma sliding-window flex_attention, Nemotron-H flash_attention_2), then flash_attention_2 if flash_attn is installed otherwise sdpa.

  LLMBuilder and create_model_from_name_or_path both go through that helper. Nemotron-H's catalog trainer config sets attn_implementation to flash_attention_2.

• attach reference and critic adapters with packed-expert LoRA

  PEFT hosts multiple target_parameters adapters on the same expert weights. GRPO/CISPO can keep a frozen reference adapter for KL, and a value head can attach a trainable critic. Mixed fused routing works on both routed (Nemotron-H) and sorted (JetMoE) packed-expert modules.

• drop use_vllm from the training spec

  Training YAML and the algorithm constructor no longer have use_vllm. Colocated rollout (training.rollout_mode) fills a default vllm_config when it is unset; that config is what starts the in-process engine. Example GRPO / PPO / REINFORCE / GSPO / CISPO configs and the remote env tutorial drop the flag.

Fixes

• pin deepspeed to 0.19.2

  Pin the llm extra to DeepSpeed 0.19.2. 0.19.3+ (#8148) leaves ZeRO-3 frozen params gathered after activation-checkpoint recompute.

• cap vLLM max_num_batched_tokens at seqs times context length

  Family and explicit max_num_batched_tokens values are capped at max_num_seqs * max_model_len. vLLM cannot schedule more tokens than that product.

Full Changelog: agilerl-arena/v1.2.0...agilerl-arena/v1.3.0

v2.27.1: cap vLLM max_num_batched_tokens at seqs times context length

Released on 2026-09-16 - GitHub - PyPI

Fixes

• cap vLLM max_num_batched_tokens at seqs times context length

  Family and explicit max_num_batched_tokens values are capped at max_num_seqs * max_model_len. vLLM cannot schedule more tokens than that product.

Full Changelog: v2.27.0...v2.27.1

v2.26.1: pin deepspeed to 0.19.2

Released on 2026-09-16 - GitHub - PyPI

Fixes

• pin deepspeed to 0.19.2

  Pin the llm extra to DeepSpeed 0.19.2. 0.19.3+ (#8148) leaves ZeRO-3 frozen params gathered after activation-checkpoint recompute.

Full Changelog: v2.26.0...v2.26.1

v2.26.0: resolve trainer attention in one helper

Released on 2026-09-15 - GitHub - PyPI

Features

• resolve trainer attention in one helper

  resolve_attn_implementation now picks the trainer attention backend in one place. Order: an explicit value, then ATTN_IMPLEMENTATION / AGILERL_ATTN_IMPLEMENTATION, then the family trainer default (Gemma sliding-window flex_attention, Nemotron-H flash_attention_2), then flash_attention_2 if flash_attn is installed otherwise sdpa.

  LLMBuilder and create_model_from_name_or_path both go through that helper. Nemotron-H's catalog trainer config sets attn_implementation to flash_attention_2.

Full Changelog: v2.25.0...v2.26.0

v2.24.0: load family trainer and vLLM defaults by Hugging Face model_type

Released on 2026-09-14 - GitHub - PyPI

Features

• load family trainer and vLLM defaults by Hugging Face model_type

  Callers pass a checkpoint id or path to family_runtime, which reads AutoConfig.model_type and applies catalog defaults with setdefault. Missing config.json raises. Family patches use a loaded config.model_type when present, otherwise the Hugging Face id. nemotron_h gets vLLM mamba_cache_mode=align, max_num_batched_tokens=8192, reasoning_parser=nemotron_v3, and enable_prefix_caching=True. Gemma 3 and 4 types get trainer attn_implementation=flex_attention. Explicit yaml / vllm_config values still win.

Full Changelog: v2.23.0...v2.24.0

v2.23.0: rename RLAlgorithm to SingleAgentAlgorithm

Released on 2026-09-14 - GitHub - PyPI

Refactoring

• rename RLAlgorithm to SingleAgentAlgorithm

  Rename RLAlgorithm to SingleAgentAlgorithm and MultiAgentRLAlgorithm to MultiAgentAlgorithm. Spec bases follow: RLAlgorithmSpec / MultiAgentRLAlgorithmSpec become SingleAgentAlgorithmSpec / MultiAgentAlgorithmSpec. The old algorithm class names remain importable from agilerl.algorithms.core.

Full Changelog: v2.22.0...v2.23.0

agilerl-arena/v1.1.0: agilerl-arena v1.1.0: rename RLAlgorithm to SingleAgentAlgorithm

Released on 2026-09-14 - GitHub - PyPI

Refactoring

• rename RLAlgorithm to SingleAgentAlgorithm

  Rename RLAlgorithm to SingleAgentAlgorithm and MultiAgentRLAlgorithm to MultiAgentAlgorithm. Spec bases follow: RLAlgorithmSpec / MultiAgentRLAlgorithmSpec become SingleAgentAlgorithmSpec / MultiAgentAlgorithmSpec. The old algorithm class names remain importable from agilerl.algorithms.core.

Full Changelog: agilerl-arena/v1.0.0...agilerl-arena/v1.1.0

v2.22.0: define training specs only in agilerl-arena

Released on 2026-09-14 - GitHub - PyPI

Features

• define training specs only in agilerl-arena

  The training manifest lives in agilerl.arena.models. The framework imports those classes as the specs; it does not subclass them to add make_env / init_buffer / build. Builders and strategies sit beside the specs. Unknown keys are rejected. Defaults match the algorithm constructors. evo_steps is optional. LocalTrainer takes networks and HPO as arguments. New CLI: arena manifest validate and arena manifest schema. agilerl now depends on agilerl-arena>=1.0.0,<2.0.

Full Changelog: v2.21.1...v2.22.0

agilerl-arena/v1.0.0: agilerl-arena v1.0.0: build algorithms from specs via paradigm builders

Released on 2026-09-11 - GitHub - PyPI

Features

• build algorithms from specs via paradigm builders

  spec.build_algorithm() delegates to agilerl.builders. Specs remain arena field subclasses with construction wrappers. Training loops still come from the spec. Builder build() takes an AlgorithmBuildRuntime for the population slot, device, HPO, and checkpoint.

• dispatch local training through paradigm strategies

  Training loops are selected by agilerl.strategies.select_strategy from the spec's paradigm flags (off_policy, offline, bandit, env_type). LocalTrainer uses that layer. Specs still expose get_training_fn and get_training_kwargs. Multi-agent fitness logs take a per-agent dict.

• define training specs only in agilerl-arena

  The training manifest lives in agilerl.arena.models. The framework imports those classes as the specs; it does not subclass them to add make_env / init_buffer / build. Builders and strategies sit beside the specs. Unknown keys are rejected. Defaults match the algorithm constructors. evo_steps is optional. LocalTrainer takes networks and HPO as arguments. New CLI: arena manifest validate and arena manifest schema. agilerl now depends on agilerl-arena>=1.0.0,<2.0.

Fixes

• isolate dummy algorithm specs from the global registry

  A unit test no longer leaves a dummy spec on the global algorithm registry, which made later tests that walk every registered spec fail depending on collection order. Training strategy types now include LLM and bandit envs and match the fitness values the loops return.

Other

• bump peft to 0.20.0 and liger-kernel to 0.8.2

  PEFT 0.20 rejects LoRA on Mamba mixer out_proj and conv1d. adapt_lora_config_for_model excludes those modules. hydra-core stays on 1.3.x. LLMAlgorithm backward stays under AMP so fp16 checkpoint recompute matches LoRA dtypes.

Full Changelog: agilerl-arena/v0.9.0...agilerl-arena/v1.0.0

v2.21.1: isolate dummy algorithm specs from the global registry

Released on 2026-09-10 - GitHub - PyPI

Fixes

• isolate dummy algorithm specs from the global registry

  A unit test no longer leaves a dummy spec on the global algorithm registry, which made later tests that walk every registered spec fail depending on collection order. Training strategy types now include LLM and bandit envs and match the fitness values the loops return.

Full Changelog: v2.21.0...v2.21.1

v2.21.0: dispatch local training through paradigm strategies

Released on 2026-09-10 - GitHub - PyPI

Features

• dispatch local training through paradigm strategies

  Training loops are selected by agilerl.strategies.select_strategy from the spec's paradigm flags (off_policy, offline, bandit, env_type). LocalTrainer uses that layer. Specs still expose get_training_fn and get_training_kwargs. Multi-agent fitness logs take a per-agent dict.

Full Changelog: v2.20.0...v2.21.0

agilerl-arena/v0.9.0: agilerl-arena v0.9.0: clamp generation to remaining context per turn

Released on 2026-09-08 - GitHub - PyPI

Features

• subclass framework algorithm specs from agilerl-arena field models

  Framework algorithm specs now subclass the agilerl-arena pydantic field models and keep construction (build_algorithm, get_training_fn). Network encoder specs are the arena classes. Installing agilerl requires agilerl-arena>=0.9.0,<1.0. Builders, strategies, and LocalTrainer are unchanged.

Fixes

• publish to PyPI on tag push and wait for agilerl-arena

  Pushing a v* or agilerl-arena/v* tag now runs Publish release for that tag,
  so every released version reaches PyPI without a second manual step. Running
  the workflow by hand from a tag still works, for backfilling a version PyPI is
  missing or retrying a failed run.

  Before uploading agilerl, the publish job reads the agilerl-arena range
  from the wheel's own Requires-Dist and waits for a matching version to appear
  on PyPI. The two tags are pushed together and the index takes a moment to serve
  a new release, so the agilerl upload waits for agilerl-arena instead of
  failing. An agilerl-arena wheel declares no such requirement and never waits.

  This closes a real gap: agilerl 2.16.1 shipped declaring
  agilerl-arena>=0.7.0,<0.8 when no such version was on PyPI, leaving its
  arena extra uninstallable.

• bump the arena extra from the agilerl-arena version kind

  The committed agilerl-arena extra range must cover the version this release will tag. That version now follows the arena package's own bump kind, not a shared stack bump.

Breaking Changes

• clamp generation to remaining context per turn

  Each turn generates min(configured max_output_tokens, remaining context), or remaining context when the cap is unset. Setup no longer rejects a per-turn cap larger than the window. min_new_tokens clamps to that budget so it cannot exceed max_new_tokens.

  RolloutHarness and make_rollout_env_factory no longer take max_output_tokens. The harness only truncates when the prompt fills max_model_len. max_prompt_tokens_for_model_len now takes only max_model_len. validate_llm_context_lengths is removed.

Full Changelog: agilerl-arena/v0.8.1...agilerl-arena/v0.9.0

v2.18.2: clamp generation to remaining context per turn

Released on 2026-09-08 - GitHub - PyPI

Breaking Changes

• clamp generation to remaining context per turn

  Each turn generates min(configured max_output_tokens, remaining context), or remaining context when the cap is unset. Setup no longer rejects a per-turn cap larger than the window. min_new_tokens clamps to that budget so it cannot exceed max_new_tokens.

  RolloutHarness and make_rollout_env_factory no longer take max_output_tokens. The harness only truncates when the prompt fills max_model_len. max_prompt_tokens_for_model_len now takes only max_model_len. validate_llm_context_lengths is removed.

Full Changelog: v2.18.1...v2.18.2

v2.18.1: align ArenaClient parquet config names with prefix stripping

Released on 2026-09-07 - GitHub - PyPI

Fixes

• align ArenaClient parquet config names with prefix stripping

  ArenaClient strips shared directories from parquet folder uploads while every remaining path still has more than one component. Split directories such as train and test are config names. A flat parquet tree uses config default. Tabular dataset uploads stay rejected.

• publish to PyPI on tag push and wait for agilerl-arena

  Pushing a v* or agilerl-arena/v* tag now runs Publish release for that tag,
  so every released version reaches PyPI without a second manual step. Running
  the workflow by hand from a tag still works, for backfilling a version PyPI is
  missing or retrying a failed run.

  Before uploading agilerl, the publish job reads the agilerl-arena range
  from the wheel's own Requires-Dist and waits for a matching version to appear
  on PyPI. The two tags are pushed together and the index takes a moment to serve
  a new release, so the agilerl upload waits for agilerl-arena instead of
  failing. An agilerl-arena wheel declares no such requirement and never waits.

  This closes a real gap: agilerl 2.16.1 shipped declaring
  agilerl-arena>=0.7.0,<0.8 when no such version was on PyPI, leaving its
  arena extra uninstallable.

• bump the arena extra from the agilerl-arena version kind

  The committed agilerl-arena extra range must cover the version this release will tag. That version now follows the arena package's own bump kind, not a shared stack bump.

Full Changelog: v2.18.0...v2.18.1

agilerl-arena/v0.8.0: agilerl-arena v0.8.0: write Arena completeness files after merged LoRA export

Released on 2026-09-07 - GitHub - PyPI

Features

• write Arena completeness files after merged LoRA export

  Successful export_merged_pretrained clears output_dir, then writes
  arena_artifact_manifest.json (format hf_merged, sha256 and byte size
  per file) and .complete. adapter_config.json is omitted from files[].
  ZeRO layer-wise gather is unchanged.

• strip Hugging Face parquet prefixes and add organisation-key auth

  ArenaClient.create_dataset strips shared Hugging Face folder prefixes before
  choosing a parquet config. Partner servers can authenticate with an organisation
  key plus X-External-User-Id. Inference agents opened from an org-key client send
  the same headers. PAT and Keycloak device login are unchanged. The
  agilerl[arena] extra is agilerl-arena>=0.8.0,<0.9.

Other

• gate agilerl PyPI publish on a resolvable agilerl-arena pin

  Refuse to upload an agilerl wheel whose agilerl-arena requirement is not yet on PyPI. Retryable publish skips artifacts already on the index.

Full Changelog: agilerl-arena/v0.7.1...agilerl-arena/v0.8.0

v2.17.0: write Arena completeness files after merged LoRA export

Released on 2026-09-07 - GitHub - PyPI

Features

• write Arena completeness files after merged LoRA export

  Successful export_merged_pretrained clears output_dir, then writes
  arena_artifact_manifest.json (format hf_merged, sha256 and byte size
  per file) and .complete. adapter_config.json is omitted from files[].
  ZeRO layer-wise gather is unchanged.

Tests

• close Rich Live in arena output tests

  Stop a leftover Live renderer after the check-event test, and use dummy Live objects in select_row tests so they do not start a real Rich Live.

Other

• gate agilerl PyPI publish on a resolvable agilerl-arena pin

  Refuse to upload an agilerl wheel whose agilerl-arena requirement is not yet on PyPI. Retryable publish skips artifacts already on the index.

Full Changelog: v2.16.1...v2.17.0

v2.16.1: add ZeRO-aware LoRA merge to Hugging Face format

Released on 2026-09-06 - GitHub - PyPI

Features

• add ZeRO-aware LoRA merge to Hugging Face format

  Add export_merged_pretrained, a public helper that folds a LoRA adapter
  into base weights and writes a Hugging Face directory (config, bf16
  safetensors, optional tokenizer and generation config) without gathering
  the full model onto one rank.

  Works from a live PEFT module (DeepSpeed ZeRO-3 gathers one module at a
  time) or from a stored actor/ adapter plus a base model path. Replay
  loads a meta skeleton and streams base weights from safetensors. The live
  module is not mutated. A failed collective or write leaves every rank
  able to continue training.

Full Changelog: v2.16.0...v2.16.1

agilerl-arena/v0.7.0: agilerl-arena v0.7.0: upload local parquet files and HF shard folders

Released on 2026-09-07 - GitHub - PyPI

Features

• upload local parquet files and HF shard folders

  ArenaClient.create_dataset and arena datasets create --file now upload a .parquet file with the parquet content type, or walk a Hugging Face layout folder and send shards as repeated multipart file parts with relative paths. Pass config= / --config when the folder has more than one parquet config. CSV uploads are unchanged.

  The agilerl[arena] extra is agilerl-arena>=0.7.0,<0.8.

Full Changelog: agilerl-arena/v0.6.0...agilerl-arena/v0.7.0

v2.16.0: upload local parquet files and HF shard folders

Released on 2026-09-06 - GitHub - PyPI

Features

• upload local parquet files and HF shard folders

  ArenaClient.create_dataset and arena datasets create --file now upload a .parquet file with the parquet content type, or walk a Hugging Face layout folder and send shards as repeated multipart file parts with relative paths. Pass config= / --config when the folder has more than one parquet config. CSV uploads are unchanged.

  The agilerl[arena] extra is agilerl-arena>=0.7.0,<0.8.

Full Changelog: v2.15.0...v2.16.0

agilerl-arena/v0.6.0: agilerl-arena v0.6.0: install architecture patches through one family installer

Released on 2026-09-07 - GitHub - PyPI

Features

• install architecture patches through one family installer

  detect_model_family returns a single family key (or None). FAMILY_PATCHES maps each family to one callable. Nemotron-H uses install_nemotron_h_patches, which applies the Mamba fused-path and stream-ordering workarounds on every ZeRO stage. install_family_patches returns that family key or None.

• add model catalog and default RunSpec discovery to ArenaClient

  ArenaClient can list the HuggingFace model catalog, fetch model info (LoRA modules and context length), and load a default RunSpec from the CLI API.

  The arena extra is agilerl-arena>=0.6.0,<0.7.

Fixes

• publish from the tag ref and open a GitHub Release after PyPI

  Run Publish release from the version tag (Use workflow from). After PyPI
  accepts the upload, the workflow opens a GitHub Release with generated notes
  and the dist files attached.

• build GitHub Release notes from spoke commits

  Publish release opens a GitHub Release with a vX.Y.Z: headline title and
  notes grouped from the commits since the last tag (Features, Fixes, and so on),
  not GitHub’s PR-only generate-notes button.

Tests

• cover encoder layer_norm disable path

  Initialize TD3 with encoder_config.layer_norm=True so the warning that disables it is covered.

Other

• publish PyPI from workflow_dispatch only

  Tag push no longer publishes. Create a GitHub Release from an existing v* or agilerl-arena/v* tag with workflow_dispatch. Minting those tags from hub does not upload to PyPI.

• ReGraMa & Amplified-Gaussian / Random-Reset Parameter Mutations Switches

• Function-Preserving Node Addition & Layer Addition Architecture Mutations

• pin Node 24 Actions and skip uv cache on publish

  Publish release uses Node 24 action pins. The publish job does not check
  out the repo, so uv cache is off.

• Chore(deps): Bump pygame-ce from 2.5.7 to 2.5.8

• stop GitHub Dependabot on the public clone

  Dependency updates for Python packages are handled on the internal hub. This clone no longer ships a .github/dependabot.yml config.

Full Changelog: agilerl-arena/v0.5.0...agilerl-arena/v0.6.0

v2.15.0: add model catalog and default RunSpec discovery to ArenaClient

Released on 2026-09-06 - GitHub - PyPI

Features

• add model catalog and default RunSpec discovery to ArenaClient

  ArenaClient can list the HuggingFace model catalog, fetch model info (LoRA modules and context length), and load a default RunSpec from the CLI API.

  The arena extra is agilerl-arena>=0.6.0,<0.7.

Other

• stop GitHub Dependabot on the public clone

  Dependency updates for Python packages are handled on the internal hub. This clone no longer ships a .github/dependabot.yml config.

Full Changelog: v2.14.3...v2.15.0