This module implements a Hindsight Experience Replay (HER).
- class HERReplayBuffer(replay_k, reward_fn, capacity_in_transitions, env_spec)¶
Replay buffer for HER (Hindsight Experience Replay).
It constructs hindsight examples using future strategy.
replay_k (int) – Number of HER transitions to add for each regular Transition. Setting this to 0 means that no HER replays will be added.
reward_fn (callable) – Function to re-compute the reward with substituted goals.
capacity_in_transitions (int) – total size of transitions in the buffer.
env_spec (EnvSpec) – Environment specification.
- add_path(self, path)¶
Adds a path to the replay buffer.
For each transition in the given path except the last one, replay_k HER transitions will added to the buffer in addition to the one in the path. The last transition is added without sampling additional HER goals.
- add_episode_batch(self, episodes)¶
Add a EpisodeBatch to the buffer.
episodes (EpisodeBatch) – Episodes to add.
Sample a single path from the buffer.
A dict of arrays of shape (path_len, flat_dim).
- Return type
- sample_transitions(self, batch_size)¶
Sample a batch of transitions from the buffer.
- sample_timesteps(self, batch_size)¶
Sample a batch of timesteps from the buffer.