garage.tf.optimizers.penalty_lbfgs_optimizer

Penalized Limited-memory BFGS (L-BFGS) optimizer.

class PenaltyLBFGSOptimizer(max_opt_itr=20, initial_penalty=1.0, min_penalty=0.01, max_penalty=1000000.0, increase_penalty_factor=2, decrease_penalty_factor=0.5, max_penalty_itr=10, adapt_penalty=True)

Penalized Limited-memory BFGS (L-BFGS) optimizer.

Performs constrained optimization via penalized L-BFGS. The penalty term is adaptively adjusted to make sure that the constraint is satisfied.

Parameters
  • max_opt_itr (int) – Maximum iteration for update.

  • initial_penalty (float) – Initial penalty.

  • min_penalty (float) – Minimum penalty allowed. Penalty will be clipped if lower than this value.

  • max_penalty (float) – Maximum penalty allowed. Penalty will be clipped if higher than this value.

  • increase_penalty_factor (float) – Factor to increase penalty in each penalty iteration.

  • decrease_penalty_factor (float) – Factor to decrease penalty in each penalty iteration.

  • max_penalty_itr (int) – Maximum penalty iterations to perform.

  • adapt_penalty (bool) – Whether the penalty is adaptive or not. If false, penalty will not change.

update_opt(self, loss, target, leq_constraint, inputs, constraint_name='constraint', name='PenaltyLBFGSOptimizer', **kwargs)

Construct operation graph for the optimizer.

Parameters
  • loss (tf.Tensor) – Loss objective to minimize.

  • target (object) – Target object to optimize. The object should implemenet get_params() and get_param_values.

  • leq_constraint (tuple) – It contains a tf.Tensor and a float value. The tf.Tensor represents the constraint term, and the float value is the constraint value.

  • inputs (list[tf.Tensor]) – List of input placeholders.

  • constraint_name (str) – Constraint name for logging.

  • name (str) – Name scope.

  • kwargs (dict) – Extra unused keyword arguments. Some optimizers have extra input, e.g. KL constraint.

loss(self, inputs)

The loss.

Parameters

inputs (list[numpy.ndarray]) – List of input values.

Returns

Loss.

Return type

float

Raises

Exception – If loss function is None, i.e. not defined.

constraint_val(self, inputs)

The constraint value.

Parameters

inputs (list[numpy.ndarray]) – List of input values.

Returns

Constraint value.

Return type

float

Raises

Exception – If loss function is None, i.e. not defined.

optimize(self, inputs, name='optimize')

Perform optimization.

Parameters
  • inputs (list[numpy.ndarray]) – List of input values.

  • name (str) – Name scope.

Raises

Exception – If loss function is None, i.e. not defined.