Contents Menu Expand Light mode Dark mode Auto light/dark, in light mode Auto light/dark, in dark mode Skip to content
AgileRL Documentation
Light Logo Dark Logo
AgileRL Documentation

Introduction

  • Getting Started
  • Releases

Arena

  • Arena Client

Training

  • Evolutionary Hyperparameter Optimization
    • Tournament Selection
    • Mutation
    • Multi-Frequency Selection
  • Off-Policy Training
  • On-Policy Training
  • Partially Observable Markov Decision Processes (POMDPs)
  • Offline Training
  • Multi-Agent Training
  • LLM Fine-Tuning
    • Environments (OpenEnv)
    • Batch sizes and optimizer steps
    • Multi-GPU LLM training
    • GPU memory estimate
    • Memory-efficient log-probabilities
    • Quantization
    • Saving and Loading LLM Checkpoints
    • Colocated vLLM sleep/wake
    • LLM rollout records
  • Contextual Multi-Armed Bandits
  • Distributed Training
  • Trainers
  • Evolvable Neural Networks
  • Creating Custom Algorithms
  • Debugging Reinforcement Learning

Observability

  • Metrics and Logging

Tutorials

  • Gymnasium Single-agent
    • Acrobot with PPO
    • Lunar Lander with TD3
    • Cartpole with Rainbow DQN
    • Partially Observable Pendulum-v1 with Recurrent PPO
  • PettingZoo Multi-agent
    • Self-Play Connect4 with DQN + Curriculum Learning
    • Space Invaders with MADDPG
    • Speaker-Listener with MATD3
  • Multiple-Frequencies Population-Based Training
    • Lunar Lander with PPO & MF-PBT
  • Hierarchical Skills
  • LLM Fine-Tuning
    • LLM Reasoning Tutorial
    • LLM Fine-Tuning with HPO
    • Multi-turn Fine-Tuning with LLMPPO, LLMREINFORCE, and GRPO
    • Serving Environments Remotely (WebSocket Sessions)
    • LLM Fine-Tuning with SFT and DPO
  • Contextual Multi-arm Bandits
    • Iris with NeuralUCB
    • PenDigits with NeuralTS
  • Creating Custom Networks
    • Building a Dueling Distributional Q Network
    • Integrating Architecture Mutations Into SimBa
  • Training on Arena
    • PPO on a Custom Gym Environment
    • GRPO on GSM8K

API

  • Algorithms
    • EvolvableAlgorithm
    • Algorithms Mutations Registry
    • OptimizerWrapper
    • Conservative Q-Learning (CQL)
    • Deep Deterministic Policy Gradient (DDPG)
    • Deep Q-Learning (DQN)
    • Rainbow DQN
    • Implicit Language Q-Learning (ILQL)
    • Proximal Policy Optimization (PPO)
    • Twin Delayed Deep Deterministic Policy Gradient (TD3)
    • Independent Proximal Policy Optimization (IPPO)
    • Multi-Agent Deep Deterministic Policy Gradient (MADDPG)
    • Multi-Agent Twin-Delayed Deep Deterministic Policy Gradient (MATD3)
    • Neural Contextual Bandits with UCB-based Exploration (NeuralUCB)
    • Neural Contextual Bandits with Thompson Sampling (NeuralTS)
    • Group Relative Policy Optimization (GRPO)
    • Clipped Importance Sampling Policy Optimization (CISPO)
    • Group Sequence Policy Optimization (GSPO)
    • Direct Preference Optimization (DPO)
    • Supervised Fine-Tuning (SFT)
    • LLM Proximal Policy Optimization (LLM PPO)
    • LLM REINFORCE
    • Supervised Fine-Tuning (SFT)
  • Arena
    • ArenaClient
    • Arena manifest models
    • Stream
    • Output
    • Inference
    • Authentication
    • Exceptions
  • Components
    • Experience Replay Buffer
    • On-Policy Rollout Buffer
    • Segment Trees
    • Data Structures and Utilities
    • Experience Sampler
  • Hyperparameter Optimization
    • Mutation
    • Tournament Selection
    • Multi-Frequency Selection
  • Logger
  • Metrics
  • Models
    • Training Manifest
    • Algorithm Specifications
    • Environment Specifications
    • Network Specifications
    • Training & Replay Buffer Specs
    • HPO Specifications
  • Modules
    • EvolvableModule
    • EvolvableWrapper
    • ModuleDict
    • Evolvable Multi-layer Perceptron (MLP)
    • Evolvable Convolutional Neural Network (CNN)
    • Evolvable CNN → LSTM encoder
    • Evolvable Long Short-Term Memory (LSTM)
    • Evolvable Multi-Input Neural Network (Dict / Tuple Observations)
    • Evolvable SimBa
    • Evolvable ResNet
    • DummyEvolvable
    • Evolvable GPT
    • Evolvable BERT
    • Custom Activation Functions
  • Networks
    • EvolvableNetwork
    • QNetwork
    • RainbowQNetwork
    • ContinuousQNetwork
    • EvolvableDistribution
    • DeterministicActor
    • StochasticActor
    • ValueNetwork
  • Population
  • Training
  • Rollouts
    • On-Policy Rollout Functions
  • Utils
    • Arena utilities
    • General Utils
    • Algorithm Utils
    • Population Utils
    • Cache Utils
    • Distributed
    • Evolvable Networks Utils
    • ILQL Utils
    • Log Utils
    • Minari Utils
    • Probe Environments
    • LLM Probe Environments
    • Torch Utils
    • LLM Utils
  • Vector
    • Dummy Vectorized Environments
    • Petting Zoo Async Vector Environment
    • Petting Zoo Vector Base Class
  • Wrappers
    • AgentWrapper
    • RSNorm
    • AsyncAgentsWrapper
    • LLM environments
    • Make Evolvable
    • Skill
    • BanditEnv
    • Petting Zoo Wrapper

Development

  • GitHub
  • Discord
  • Contribute to AgileRL
Back to top
View this page
Edit this page

Utils¶

  • Arena utilities
  • General Utils
  • Algorithm Utils
  • Population Utils
  • Cache Utils
  • Distributed
  • Evolvable Networks Utils
  • ILQL Utils
  • Log Utils
  • Minari Utils
  • Probe Environments
  • LLM Probe Environments
  • Torch Utils
  • LLM Utils
Next
Arena utilities
Previous
On-Policy Rollout Functions
Copyright © 2023, AgileRL
Made with Sphinx and @pradyunsg's Furo