garage.tf.models.cnn_mlp_merge_model
¶
CNN and MLP Merge Model.
-
class
CNNMLPMergeModel
(filters, strides, hidden_sizes=(256), output_dim=1, action_merge_layer=- 2, name=None, padding='SAME', max_pooling=False, pool_strides=(2, 2), pool_shapes=(2, 2), cnn_hidden_nonlinearity=tf.nn.relu, cnn_hidden_w_init=tf.initializers.glorot_uniform(seed=deterministic.get_tf_seed_stream()), cnn_hidden_b_init=tf.zeros_initializer(), hidden_nonlinearity=tf.nn.relu, hidden_w_init=tf.initializers.glorot_uniform(seed=deterministic.get_tf_seed_stream()), hidden_b_init=tf.zeros_initializer(), output_nonlinearity=None, output_w_init=tf.initializers.glorot_uniform(seed=deterministic.get_tf_seed_stream()), output_b_init=tf.zeros_initializer(), layer_normalization=False)¶ Bases:
garage.tf.models.model.Model
Convolutional neural network followed by a Multilayer Perceptron.
Combination of a CNN Model (optionally with max pooling) and an MLP Merge model. The CNN accepts the state as an input, while the MLP accepts the CNN’s output and the action as inputs.
- Parameters
filters (Tuple[Tuple[int, Tuple[int, int]], ..]) – Number and dimension of filters. For example, ((3, (3, 5)), (32, (3, 3))) means there are two convolutional layers. The filter for the first layer have 3 channels and its shape is (3 x 5), while the filter for the second layer have 32 channels and its shape is (3 x 3).
strides (tuple[int]) – The stride of the sliding window. For example, (1, 2) means there are two convolutional layers. The stride of the filter for first layer is 1 and that of the second layer is 2.
hidden_sizes (tuple[int]) – Output dimension of dense layer(s). For example, (32, 32) means the MLP of this q-function consists of two hidden layers, each with 32 hidden units.
output_dim (int) – Dimension of the network output.
action_merge_layer (int) – The index of layers at which to concatenate action inputs with the network. The indexing works like standard python list indexing. Index of 0 refers to the input layer (observation input) while an index of -1 points to the last hidden layer. Default parameter points to second layer from the end.
name (str) – Model name, also the variable scope.
padding (str) – The type of padding algorithm to use, either ‘SAME’ or ‘VALID’.
max_pooling (bool) – Boolean for using max pooling layer or not.
pool_shapes (tuple[int]) – Dimension of the pooling layer(s). For example, (2, 2) means that all the pooling layers have shape (2, 2).
pool_strides (tuple[int]) – The strides of the pooling layer(s). For example, (2, 2) means that all the pooling layers have strides (2, 2).
cnn_hidden_nonlinearity (callable) – Activation function for intermediate dense layer(s) in the CNN. It should return a tf.Tensor. Set it to None to maintain a linear activation.
cnn_hidden_w_init (callable) – Initializer function for the weight of intermediate dense layer(s) in the CNN. Function should return a tf.Tensor.
cnn_hidden_b_init (callable) – Initializer function for the bias of intermediate dense layer(s) in the CNN. Function should return a tf.Tensor.
hidden_nonlinearity (callable) – Activation function for intermediate dense layer(s) in the MLP. It should return a tf.Tensor. Set it to None to maintain a linear activation.
hidden_w_init (callable) – Initializer function for the weight of intermediate dense layer(s) in the MLP. The function should return a tf.Tensor.
hidden_b_init (callable) – Initializer function for the bias of intermediate dense layer(s) in the MLP. The function should return a tf.Tensor.
output_nonlinearity (callable) – Activation function for output dense layer in the MLP. It should return a tf.Tensor. Set it to None to maintain a linear activation.
output_w_init (callable) – Initializer function for the weight of output dense layer(s) in the MLP. The function should return a tf.Tensor.
output_b_init (callable) – Initializer function for the bias of output dense layer(s) in the MLP. The function should return a tf.Tensor.
layer_normalization (bool) – Bool for using layer normalization or not.
-
network_input_spec
(self)¶ Network input spec.
-
build
(self, *inputs, name=None)¶ Build a Network with the given input(s).
* Do not call tf.global_variable_initializers() after building a model as it will reassign random weights to the model. The parameters inside a model will be initialized when calling build(). *
It uses the same, fixed variable scope for all Networks, to ensure parameter sharing. Different Networks must have an unique name.
- Parameters
- Raises
ValueError – When a Network with the same name is already built.
- Returns
- Output tensors of the model with the given
inputs.
- Return type
list[tf.Tensor]
-
network_output_spec
(self)¶ Network output spec.
-
property
parameters
(self)¶ Parameters of the model.
- Returns
Parameters
- Return type
np.ndarray
-
property
name
(self)¶ Name (str) of the model.
This is also the variable scope of the model.
- Returns
Name of the model.
- Return type
-
property
input
(self)¶ Default input of the model.
When the model is built the first time, by default it creates the ‘default’ network. This property creates a reference to the input of the network.
- Returns
Default input of the model.
- Return type
tf.Tensor
-
property
output
(self)¶ Default output of the model.
When the model is built the first time, by default it creates the ‘default’ network. This property creates a reference to the output of the network.
- Returns
Default output of the model.
- Return type
tf.Tensor
-
property
inputs
(self)¶ Default inputs of the model.
When the model is built the first time, by default it creates the ‘default’ network. This property creates a reference to the inputs of the network.
- Returns
Default inputs of the model.
- Return type
list[tf.Tensor]
-
property
outputs
(self)¶ Default outputs of the model.
When the model is built the first time, by default it creates the ‘default’ network. This property creates a reference to the outputs of the network.
- Returns
Default outputs of the model.
- Return type
list[tf.Tensor]
-
reset
(self, do_resets=None)¶ Reset the module.
This is effective only to recurrent modules. do_resets is effective only to vectoried modules.
For a vectorized modules, do_resets is an array of boolean indicating which internal states to be reset. The length of do_resets should be equal to the length of inputs.
- Parameters
do_resets (numpy.ndarray) – Bool array indicating which states to be reset.
-
property
state_info_specs
(self)¶ State info specification.
- Returns
- keys and shapes for the information related to the
module’s state when taking an action.
- Return type
List[str]
-
property
state_info_keys
(self)¶ State info keys.
- Returns
- keys for the information related to the module’s state
when taking an input.
- Return type
List[str]
-
terminate
(self)¶ Clean up operation.
-
get_trainable_vars
(self)¶ Get trainable variables.
- Returns
- A list of trainable variables in the current
variable scope.
- Return type
List[tf.Variable]
-
get_global_vars
(self)¶ Get global variables.
- Returns
- A list of global variables in the current
variable scope.
- Return type
List[tf.Variable]
-
get_regularizable_vars
(self)¶ Get all network weight variables in the current scope.
- Returns
- A list of network weight variables in the
current variable scope.
- Return type
List[tf.Variable]
-
get_params
(self)¶ Get the trainable variables.
- Returns
- A list of trainable variables in the current
variable scope.
- Return type
List[tf.Variable]
-
get_param_shapes
(self)¶ Get parameter shapes.
- Returns
A list of variable shapes.
- Return type
List[tuple]
-
get_param_values
(self)¶ Get param values.
- Returns
- Values of the parameters evaluated in
the current session
- Return type
np.ndarray
-
set_param_values
(self, param_values)¶ Set param values.
- Parameters
param_values (np.ndarray) – A numpy array of parameter values.
-
flat_to_params
(self, flattened_params)¶ Unflatten tensors according to their respective shapes.
- Parameters
flattened_params (np.ndarray) – A numpy array of flattened params.
- Returns
- A list of parameters reshaped to the
shapes specified.
- Return type
List[np.ndarray]