garage.replay_buffer.her_replay_buffer module

This module implements a Hindsight Experience Replay (HER).

See: https://arxiv.org/abs/1707.01495.

class HerReplayBuffer(replay_k, reward_fun, env_spec, size_in_transitions, time_horizon)[source]

Bases: garage.replay_buffer.base.ReplayBuffer

Replay buffer for HER (Hindsight Experience Replay).

It constructs hindsight examples using future strategy.

Parameters:
  • replay_k (float) – Ratio between HER replays and regular replays
  • reward_fun (callable) – Function to re-compute the reward with substituted goals
  • env_spec (garage.envs.EnvSpec) – Environment specification.
  • size_in_transitions (int) – total size of transitions in the buffer
  • time_horizon (int) – time horizon of rollout.
sample(batch_size)[source]

Sample a transition of batch_size.

Parameters:batch_size (int) – Batch size to sample.
Returns:
Transitions which transitions[key] has the
shape of \((N, S^*)\). Keys include [observation, action, goal, achieved_goal, terminal, next_observation, next_achieved_goal and reward].
Return type:dict[numpy.ndarray]
make_her_sample(replay_k, reward_fun)[source]

Generate a transition sampler for HER ReplayBuffer.

Parameters:
  • replay_k (float) – Ratio between HER replays and regular replays
  • reward_fun (callable) – Function to re-compute the reward with substituted goals
Returns:

A function that returns sample transitions for HER.
Return type:

callable