garage.tf.optimizers package¶
-
class
ConjugateGradientOptimizer
(cg_iters=10, reg_coeff=1e-05, subsample_factor=1.0, backtrack_ratio=0.8, max_backtracks=15, accept_violation=False, hvp_approach=None, num_slices=1)[source]¶ Bases:
object
Performs constrained optimization via line search.
The search direction is computed using a conjugate gradient algorithm, which gives x = A^{-1}g, where A is a second order approximation of the constraint and g is the gradient of the loss function.
Parameters: - cg_iters (int) – The number of CG iterations used to calculate A^-1 g
- reg_coeff (float) – A small value so that A -> A + reg*I
- subsample_factor (float) – Subsampling factor to reduce samples when using “conjugate gradient. Since the computation time for the descent direction dominates, this can greatly reduce the overall computation time.
- backtrack_ratio (float) – backtrack ratio for backtracking line search.
- max_backtracks (int) – Max number of iterations for backtrack linesearch.
- accept_violation (bool) – whether to accept the descent step if it violates the line search condition after exhausting all backtracking budgets.
- hvp_approach (HessianVectorProduct) – A class that computes Hessian-vector products.
- num_slices (int) – Hessian-vector product function’s inputs will be divided into num_slices and then averaged together to improve performance.
-
constraint_val
(inputs, extra_inputs=None)[source]¶ Constraint value.
Parameters: Returns: Constraint value.
Return type:
-
loss
(inputs, extra_inputs=None)[source]¶ Compute the loss value.
Parameters: Returns: Loss value.
Return type:
-
optimize
(inputs, extra_inputs=None, subsample_grouped_inputs=None, name=None)[source]¶ Optimize the function.
Parameters: - inputs (list[numpy.ndarray]) – A list inputs, which could be subsampled if needed. It is assumed that the first dimension of these inputs should correspond to the number of data points
- extra_inputs (list[numpy.ndarray]) – A list of extra inputs which should not be subsampled.
- subsample_grouped_inputs (list[numpy.ndarray]) – Subsampled inputs to be used when subsample_factor is less than one.
- name (str) – The name argument for tf.name_scope.
-
update_opt
(loss, target, leq_constraint, inputs, extra_inputs=None, name=None, constraint_name='constraint')[source]¶ Update the optimizer.
Build the functions for computing loss, gradient, and the constraint value.
Parameters: - loss (tf.Tensor) – Symbolic expression for the loss function.
- target (garage.tf.policies.Policy) – A parameterized object to optimize over.
- leq_constraint (tuple[tf.Tensor, float]) – A constraint provided as a tuple (f, epsilon), of the form f(*inputs) <= epsilon.
- inputs (list(tf.Tenosr)) – A list of symbolic variables as inputs, which could be subsampled if needed. It is assumed that the first dimension of these inputs should correspond to the number of data points.
- extra_inputs (list[tf.Tenosr]) – A list of symbolic variables as extra inputs which should not be subsampled.
- name (str) – Name to be passed to tf.name_scope.
- constraint_name (str) – A constraint name for prupose of logging and variable names.
-
class
FiniteDifferenceHvp
(base_eps=1e-08, symmetric=True, num_slices=1)[source]¶ Bases:
garage.tf.optimizers.conjugate_gradient_optimizer.HessianVectorProduct
Computes Hessian-vector product using finite difference method.
Parameters: -
update_hvp
(f, target, inputs, reg_coeff, name=None)[source]¶ Build the symbolic graph to compute the Hessian-vector product.
Parameters: - f (tf.Tensor) – The function whose Hessian needs to be computed.
- target (garage.tf.policies.Policy) – A parameterized object to optimize over.
- inputs (tuple[tf.Tensor]) – The inputs for function f.
- reg_coeff (float) – A small value so that A -> A + reg*I.
- name (str) – Name to be used in tf.name_scope.
-
-
class
FirstOrderOptimizer
(tf_optimizer_cls=None, tf_optimizer_args=None, max_epochs=1000, tolerance=1e-06, batch_size=32, callback=None, verbose=False, name='FirstOrderOptimizer', **kwargs)[source]¶ Bases:
object
Performs (stochastic) gradient descent, possibly using fancier methods like ADAM etc.
-
update_opt
(loss, target, inputs, extra_inputs=None, **kwargs)[source]¶ Parameters: - loss – Symbolic expression for the loss function.
- target – A parameterized object to optimize over. It should implement methods of the
garage.core.paramerized.Parameterized
class. :param leq_constraint: A constraint provided as a tuple (f, epsilon),of the form f(*inputs) <= epsilon.Parameters: inputs – A list of symbolic variables as inputs Returns: No return value.
-
-
class
LbfgsOptimizer
(max_opt_itr=20, callback=None)[source]¶ Bases:
object
Performs unconstrained optimization via L-BFGS.
-
update_opt
(loss, target, inputs, extra_inputs=None, name=None, *args, **kwargs)[source]¶ Parameters: - loss – Symbolic expression for the loss function.
- target – A parameterized object to optimize over. It should implement methods of the
garage.core.paramerized.Parameterized
class. :param leq_constraint: A constraint provided as a tuple (f, epsilon),of the form f(*inputs) <= epsilon.Parameters: inputs – A list of symbolic variables as inputs Returns: No return value.
-
-
class
PenaltyLbfgsOptimizer
(max_opt_itr=20, initial_penalty=1.0, min_penalty=0.01, max_penalty=1000000.0, increase_penalty_factor=2, decrease_penalty_factor=0.5, max_penalty_itr=10, adapt_penalty=True)[source]¶ Bases:
object
Performs constrained optimization via penalized L-BFGS. The penalty term is adaptively adjusted to make sure that the constraint is satisfied.
-
update_opt
(loss, target, leq_constraint, inputs, constraint_name='constraint', name=None, *args, **kwargs)[source]¶ Parameters: - loss – Symbolic expression for the loss function.
- target – A parameterized object to optimize over. It should
implement methods of the
garage.core.paramerized.Parameterized
class. - leq_constraint – A constraint provided as a tuple (f, epsilon), of the form f(*inputs) <= epsilon.
- inputs – A list of symbolic variables as inputs
Returns: No return value.
-