garage.replay_buffer package

This public package contains the replay buffer primitives.

The replay buffer primitives can be used for RL algorithms.

class HerReplayBuffer(replay_k, reward_fun, env_spec, size_in_transitions, time_horizon)[source]

Bases: garage.replay_buffer.base.ReplayBuffer

Replay buffer for HER (Hindsight Experience Replay).

It constructs hindsight examples using future strategy.

Parameters:
  • replay_k (float) – Ratio between HER replays and regular replays
  • reward_fun (callable) – Function to re-compute the reward with substituted goals
  • env_spec (garage.envs.EnvSpec) – Environment specification.
  • size_in_transitions (int) – total size of transitions in the buffer
  • time_horizon (int) – time horizon of rollout.
sample(batch_size)[source]

Sample a transition of batch_size.

Parameters:batch_size (int) – Batch size to sample.
Returns:
Transitions which transitions[key] has the
shape of \((N, S^*)\). Keys include [observation, action, goal, achieved_goal, terminal, next_observation, next_achieved_goal and reward].
Return type:dict[numpy.ndarray]
class PathBuffer(capacity_in_transitions)[source]

Bases: object

A replay buffer that stores and can sample whole paths.

This buffer only stores valid steps, and doesn’t require paths to have a maximum length. :param capacity_in_steps: total memory allocated for the buffer :type capacity_in_steps: int

add_path(path)[source]

Add a path to the buffer.

Parameters:path (dict) – A dict of array of shape (path_len, flat_dim)
sample_path()[source]

Sample a single path from the buffer.

Returns:A dict of arrays of shape (path_len, flat_dim)
sample_transitions(batch_size)[source]

Sample a batch of transitions from the buffer.

Returns:A dict of arrays of shape (batch_size, flat_dim)
class SimpleReplayBuffer(env_spec, size_in_transitions, time_horizon)[source]

Bases: garage.replay_buffer.base.ReplayBuffer

This class implements SimpleReplayBuffer.

It uses random batch sample to minimize correlations between samples.

sample(batch_size)[source]

Sample a transition of batch_size.