0

I'm trying to concatenate outputs of two concurrent Slim ResNet-v1-101 models with last layer (logits) removed. My goal is to 'grab' the outputs from last blocks of each branch, concatenate it and reduce it to [batch_size x 1 x 1 x num_clases] tensor, so having [batch_size x num_classes] lables tensor I could easily compute sparse_softmax_cross_entropy. Here, the batch_size=2 and num_classes=3.

When I try to calculate loss, I get the following error:

tensorflow.python.framework.errors_impl.InvalidArgumentError: Can not squeeze dim[3], expected a dimension of 1, got 3 for 'sparse_softmax_cross_entropy_loss/remove_squeezable_dimensions/Squeeze' (op: 'Squeeze') with input shapes: [2,?,?,3].

The error makes sense - the last layer is Tensor("conv3_1/Relu:0", shape=(1, 1, 512, 3), dtype=float32) - it is a 1x1 convolution with 512 input and 3 output filters.

My questions are:

1) Am I concatenating the model outputs correctly? Is last conv of the last block a good place to get a handle?

2) Considering I have an output of correct size (batch size of 2 and 3 output filters should give [batch_size x ? x ? x num_classes]) - why does loss op complain?

3) Is there any other problem with my solution that causes it to fail?

Code:

with tf.variable_scope("rgb_branch"):
    with tf.contrib.slim.arg_scope(resnet_v1.resnet_arg_scope()):
        rgb_logits, rgb_end_points = resnet_v1.resnet_v1_101(features[0], self.num_classes, global_pool=True, is_training=is_training)        

        #Exclude logits
        rgb_variables_to_restore = tf.contrib.slim.get_variables_to_restore(exclude=['rgb_branch/resnet_v1_101/logits'])

        # Strip the scope name for assingment map
        rgb_assignment_map = { rgb_variables_to_restore[0].name.split(':')[0] : rgb_variables_to_restore[0]}
        rgb_assignment_map.update({ v.name.split(':')[0].split('/', 1)[1] : v for v in rgb_variables_to_restore[1:] })

        tf.train.init_from_checkpoint(self.pre_trained_model_path, rgb_assignment_map)


with tf.variable_scope("depth_branch"):
    with tf.contrib.slim.arg_scope(resnet_v1.resnet_arg_scope()):     
        depth_logits, depth_end_points = resnet_v1.resnet_v1_101(features[1], self.num_classes, global_pool=True, is_training=is_training)        

        #Exclude rgb branch already existing in the graph and logits
        depth_variables_to_restore = tf.contrib.slim.get_variables_to_restore(exclude=['rgb_branch', 'depth_branch/resnet_v1_101/logits'])

        # Strip the scope name for assingment map
        depth_assignment_map = { depth_variables_to_restore[0].name.split(':')[0] : depth_variables_to_restore[0]}
        depth_assignment_map.update({ v.name.split(':')[0].split('/', 1)[1] : v for v in depth_variables_to_restore[1:] })

        tf.train.init_from_checkpoint(self.pre_trained_model_path, depth_assignment_map)

rgb_branch_last_conv = tf.contrib.framework.get_variables('rgb_branch/resnet_v1_101/block4/unit_3/bottleneck_v1/conv3')[0]
depth_branch_last_conv = tf.contrib.framework.get_variables('depth_branch/resnet_v1_101/block4/unit_3/bottleneck_v1/conv3')[0]

model = tf.concat( [rgb_branch_last_conv, depth_branch_last_conv], axis=0, name='branches_concat')
model = slim.conv2d(model, 512, [1, 1], scope='conv1_1')
model = slim.conv2d(model, 128, [1, 1], scope='conv2_1')
model = slim.conv2d(model, 3, [1, 1], scope='conv3_1')

print(model, tf.shape(model))
print(labels, tf.shape(labels))

loss = tf.losses.sparse_softmax_cross_entropy(logits=model, labels=labels)  

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.