garage.experiment.task_sampler
¶
Efficient and general interfaces for sampling tasks for Meta-RL.
- class TaskSampler¶
Bases:
abc.ABC
Class for sampling batches of tasks, represented as `~EnvUpdate`s.
- abstract sample(n_tasks, with_replacement=False)¶
Sample a list of environment updates.
- Parameters
- Returns
- Batch of sampled environment updates, which, when
invoked on environments, will configure them with new tasks. See
EnvUpdate
for more information.
- Return type
- class ConstructEnvsSampler(env_constructors)¶
Bases:
TaskSampler
TaskSampler where each task has its own constructor.
Generally, this is used when the different tasks are completely different environments.
- Parameters
env_constructors (list[Callable[Environment]]) – Callables that produce environments (for example, environment types).
- sample(n_tasks, with_replacement=False)¶
Sample a list of environment updates.
- Parameters
- Returns
- Batch of sampled environment updates, which, when
invoked on environments, will configure them with new tasks. See
EnvUpdate
for more information.
- Return type
- class SetTaskSampler(env_constructor, *, env=None, wrapper=None)¶
Bases:
TaskSampler
TaskSampler where the environment can sample “task objects”.
This is used for environments that implement sample_tasks and set_task. For example,
HalfCheetahVelEnv
, as implemented in Garage.- Parameters
env_constructor (type) – Type of the environment.
env (garage.Environment or None) – Instance of env_constructor to sample from (will be constructed if not provided).
wrapper (Callable[garage.Environment, garage.Environment] or None) – Wrapper function to apply to environment.
- sample(n_tasks, with_replacement=False)¶
Sample a list of environment updates.
- Parameters
- Returns
- Batch of sampled environment updates, which, when
invoked on environments, will configure them with new tasks. See
EnvUpdate
for more information.
- Return type
- class EnvPoolSampler(envs)¶
Bases:
TaskSampler
TaskSampler that samples from a finite pool of environments.
This can be used with any environments, but is generally best when using in-process samplers with environments that are expensive to construct.
- Parameters
envs (list[Environment]) – List of environments to use as a pool.
- sample(n_tasks, with_replacement=False)¶
Sample a list of environment updates.
- Parameters
- Raises
ValueError – If the number of requested tasks is larger than the pool, or with_replacement is set.
- Returns
- Batch of sampled environment updates, which, when
invoked on environments, will configure them with new tasks. See
EnvUpdate
for more information.
- Return type
- MW_TASKS_PER_ENV = 50¶
- class MetaWorldTaskSampler(benchmark, kind, wrapper=None, add_env_onehot=False)¶
Bases:
TaskSampler
TaskSampler that distributes a Meta-World benchmark across workers.
- Parameters
benchmark (metaworld.Benchmark) – Benchmark to sample tasks from.
kind (str) – Must be either ‘test’ or ‘train’. Determines whether to sample training or test tasks from the Benchmark.
wrapper (Callable[garage.Env, garage.Env] or None) – Wrapper to apply to env instances.
add_env_onehot (bool) – If true, a one-hot representing the current environment name will be added to the environments. Should only be used with multi-task benchmarks.
- Raises
ValueError – If kind is not ‘train’ or ‘test’. Also raisd if add_env_onehot is used on a metaworld meta learning (not multi-task) benchmark.
- sample(n_tasks, with_replacement=False)¶
Sample a list of environment updates.
Note that this will always return environments in the same order, to make parallel sampling across workers efficient. If randomizing the environment order is required, shuffle the result of this method.
- Parameters
n_tasks (int) – Number of updates to sample. Must be a multiple of the number of env classes in the benchmark (e.g. 1 for MT/ML1, 10 for MT10, 50 for MT50). Tasks for each environment will be grouped to be adjacent to each other.
with_replacement (bool) – Whether tasks can repeat when sampled. Since this cannot be easily implemented for an object pool, setting this to True results in ValueError.
- Raises
ValueError – If the number of requested tasks is not equal to the number of classes or the number of total tasks.
- Returns
- Batch of sampled environment updates, which, when
invoked on environments, will configure them with new tasks. See
EnvUpdate
for more information.
- Return type