garage.torch.q_functions.discrete_mlp_q_function

This modules creates a continuous Q-function network.

class DiscreteMLPQFunction(env_spec, hidden_sizes, hidden_nonlinearity=F.relu, hidden_w_init=nn.init.xavier_normal_, hidden_b_init=nn.init.zeros_, output_nonlinearity=None, output_w_init=nn.init.xavier_normal_, output_b_init=nn.init.zeros_, layer_normalization=False)

Bases: garage.torch.modules.MLPModule

Inheritance diagram of garage.torch.q_functions.discrete_mlp_q_function.DiscreteMLPQFunction

Implements a discrete MLP Q-value network.

It predicts the Q-value for all possible actions based on the input state.

Parameters
  • env_spec (EnvSpec) – Environment specification.

  • hidden_sizes (list[int]) – Output dimension of dense layer(s). For example, (32, 32) means this MLP consists of two hidden layers, each with 32 hidden units.

  • hidden_nonlinearity (callable or torch.nn.Module) – Activation function for intermediate dense layer(s). It should return a torch.Tensor. Set it to None to maintain a linear activation.

  • hidden_w_init (callable) – Initializer function for the weight of intermediate dense layer(s). The function should return a torch.Tensor.

  • hidden_b_init (callable) – Initializer function for the bias of intermediate dense layer(s). The function should return a torch.Tensor.

  • output_nonlinearity (callable or torch.nn.Module) – Activation function for output dense layer. It should return a torch.Tensor. Set it to None to maintain a linear activation.

  • output_w_init (callable) – Initializer function for the weight of output dense layer(s). The function should return a torch.Tensor.

  • output_b_init (callable) – Initializer function for the bias of output dense layer(s). The function should return a torch.Tensor.

  • layer_normalization (bool) – Bool for using layer normalization or not.

forward(self, input_value)

Forward method.

Parameters

input_value (torch.Tensor) – Input values with (N, *, input_dim) shape.

Returns

Output value

Return type

torch.Tensor

property output_dim(self)

Return output dimension of network.

Returns

Output dimension of network.

Return type

int