Recurrent Neural Networks for Semantic Instance Segmentation

Amaia Salvador Jordi Torres Xavier Giró-i-Nieto Ferran MarquésManel BaradadMíriam Bellver

Motivation

2

Semantic Instance Segmentation

- Proposal-based solutions: - Hundreds/Thousands of redundant predictions- Post-processing needed (NMS)

- Holistic & class-agnostic methods- Reduced set of predictions- Separate network for semantics

Motivation

3

- Our solution: - Recurrent model that sequentially predicts binary masks and

categorical labels for each object in an image.- Learns to stop once all objects have been found.- Does not need post-processing on its output.

Semantic Instance Segmentation

Model

4

Model

5

Pascal VOC

6

Average Precision at different IoU thresholds

Pascal VOC

7

Average Recall at different IoU thresholds (class-agnostic evaluation)

Pascal VOC

8

CVPPP Leaves Segmentation

9

Cityscapes

10

Object Sorting Patterns

11

Encoder ActivationsCorrelation with convolutional activations in the encoder before and after training

Encoder ActivationsActivations at the end of the encoder before and after training