garage.tf.optimizers.penalty_lbfgs_optimizer module

Penalized Limited-memory BFGS (L-BFGS) optimizer.

class PenaltyLbfgsOptimizer(max_opt_itr=20, initial_penalty=1.0, min_penalty=0.01, max_penalty=1000000.0, increase_penalty_factor=2, decrease_penalty_factor=0.5, max_penalty_itr=10, adapt_penalty=True)

Bases: object

Penalized Limited-memory BFGS (L-BFGS) optimizer.

Performs constrained optimization via penalized L-BFGS. The penalty term is adaptively adjusted to make sure that the constraint is satisfied.

Parameters:

• max_opt_itr (int) – Maximum iteration for update.
• initial_penalty (float) – Initial penalty.
• min_penalty (float) – Minimum penalty allowed. Penalty will be clipped if lower than this value.
• max_penalty (float) – Maximum penalty allowed. Penalty will be clipped if higher than this value.
• increase_penalty_factor (float) – Factor to increase penalty in each penalty iteration.
• decrease_penalty_factor (float) – Factor to decrease penalty in each penalty iteration.
• max_penalty_itr (int) – Maximum penalty iterations to perform.
• adapt_penalty (bool) – Whether the penalty is adaptive or not. If false, penalty will not change.

constraint_val(inputs)

The constraint value.

Parameters:inputs (list[numpy.ndarray]) – List of input values. Returns:Constraint value. Return type:float Raises:Exception – If loss function is None, i.e. not defined.

loss(inputs)

The loss.

Parameters:inputs (list[numpy.ndarray]) – List of input values. Returns:Loss. Return type:float Raises:Exception – If loss function is None, i.e. not defined.

optimize(inputs, name='optimize')

Perform optimization.

Parameters:

• inputs (list[numpy.ndarray]) – List of input values.
• name (str) – Name scope.

Raises:

Exception – If loss function is None, i.e. not defined.

update_opt(loss, target, leq_constraint, inputs, constraint_name='constraint', name='PenaltyLbfgsOptimizer', **kwargs)

Construct operation graph for the optimizer.

Parameters:

• loss (tf.Tensor) – Loss objective to minimize.
• target (object) – Target object to optimize. The object should implemenet get_params() and get_param_values.
• leq_constraint (tuple) – It contains a tf.Tensor and a float value. The tf.Tensor represents the constraint term, and the float value is the constraint value.
• inputs (list[tf.Tensor]) – List of input placeholders.
• constraint_name (str) – Constraint name for logging.
• name (str) – Name scope.
• kwargs (dict) – Extra unused keyword arguments. Some optimizers have extra input, e.g. KL constraint.