Normalized Cut Loss for Weakly-supervised CNN Segmentation Meng

Normalized Cut Loss for Weakly-supervised CNN Segmentation Meng

Normalized Cut Loss for Weakly-supervised CNN Segmentation
Meng Tang1,2 Abdelaziz Djelouah1 Federico Perazzi1,3 Yuri Boykov2 Christopher Schroers1
Disney Research Zurich, Switzerland

1

University of Waterloo, Canada

2

Adobe Research

3

Previous work: Proposal Generation

Our methodology: Regularized Loss

E.g. Normalized Cut Regularized Loss

Train CNN from full but fake proposals:

Joint loss for labeled & unlabeled pixels
(pointwise) Empirical risk loss
(pairwise or high-order) regularization loss

Shallow normalized cut segmentation
Pairwise affinity Wij on RGBXY
Balanced color clustering
Deep normalized cut loss

Scribbles (partial masks)

scribbles

Proposals (full masks)

test image

Via shallow segmentation e.g. graph cut:

data term

test image

unknown pixels

[Lin et al. 2016]

w/ full masks

partial Cross Entropy (pCE)

Examples of regularization:
clustering criteria, e.g. normalized cut (NC)
pairwise CRF
Gradient computation only, no inference

w/ proposals

w/ extra NC loss

Gradient of normalized cut
Towards better color clustering

ground truth

Weak Supervison
pCE
pCE + NC

DeepLab-MSc-largeFOV+CRF
DeepLab-VGG16
ResNet101
ResNet101+CRF

w/ pCE loss only

w/ NC loss

mIOU on val set with different networks
Network

shallow NC

Whats wrong with proposals?
a heuristic to mimic full supervision
mislead training to fit errors
require expensive inference

ground truth

empirical risk loss regularization loss
for labeled pixels for unlabeled pixels

regularization term

Experiments

62.0
60.4
69.5
72.8

65.1
62.4
72.8
74.5

Full Sup.
68.7
68.8
75.6
76.8

Train with shorter scribbles
length 1

length 0.5

follow-up
this work

length 0.3

training image

network output

length 0
proposal method

gradient
click

Motivation: Regularized Loss for Semi-supervised Learning
Given M labeled data
unlabeled data , learn

and U

Semi-supervised deep learning [Weston et al. 2012]
input

?

?

empirical risk loss
for labeled data

regularization loss
for unlabeled data

Partial CE as Loss Sampling

Follow-up Work [Tang et al. arXiv 2018]

Simple but overlooked
Sampling in Stochastic Gradient Descent
scribble as sampling
up=1 for scribbles, 0 otherwise

Other regularization as losses:
pairwise CRF regularization
normalized Cut plus CRF
Besides weak-supervision
full-supervision (all fully labeled)
semi-supervision (w/ unlabeled images)

output
labeled data only

labeled & unlabeled data

1. Lin et al. "Scribblesup: Scribble-supervised convolutional networks for semantic segmentation." CVPR 2016.

2. Weston et al. "Deep learning via semi-supervised embedding." Neural Networks: Tricks of the Trade, 2012

3. Tang et al. "On Regularized Losses for Weakly-supervised CNN Segmentation." arXiv 2018.

Recently Viewed Presentations

  • This weeks theme is: ANGER: Strong Feeling of

    This weeks theme is: ANGER: Strong Feeling of

    Resentment- a feeling of indignant displeasure or persistent ill will at a wrong, insult, or injury (resentful) Acrimony- bitterness (bitter) 9/24 MATH WARM-UP Find the equation of a line going through the points: (8,2) & (6, 4) 9/25 Warm-up: Give...
  • Inspection Review - HUD.gov / U.S. Department of Housing and ...

    Inspection Review - HUD.gov / U.S. Department of Housing and ...

    If a more precise location or comment is needed, provide that information in the Contractor box. (Continued on next page). Using the definitions in the UPCS software to determine if the original recording is correct. ... Times New Roman Helv...
  • Negative Reinforcement - University of Idaho

    Negative Reinforcement - University of Idaho

    The Development of Stimulus Control SD Response SR+ Telephone rings Pick up phone and say "hello" Friendly conversation S SO Doorbell rings Pick up phone and say "hello" Friendly conversation withheld Response Concept Formation In ABA, is not a hypothetical...
  • Renewable Energy Garth Ratcliffe Department of Environmental and

    Renewable Energy Garth Ratcliffe Department of Environmental and

    Integrated solar tiles installed by Solar Century on a current development in Milton Keynes by English partnership and Bloor homes Roof mounted solar panels (Solar century) Innovative SunSlates installation by Solar century for Liang Homes Offshore sites Land Installed Marine...
  • Patent Pools - University of Wisconsin-Madison

    Patent Pools - University of Wisconsin-Madison

    Efficient level of fishing is 6.3 hours/day for each person, giving everyone utility of 793.8 fish/day But acting in their own interest, everyone fishes 12 hours/day, giving everyone utility of 144 fish/day The point: since fishing imposes a negative externality,...
  • Transport properties of strongly coupled gauge theories from ...

    Transport properties of strongly coupled gauge theories from ...

    Andrei Starinets Oxford University From Gravity to Thermal Gauge Theories: the AdS/CFT correspondence Fifth Aegean Summer School Island of Milos Greece September 21-26, 2009 What is viscosity? Viscosity of gases and liquids First-order transport (kinetic) coefficients Second-order transport ...
  • Chapter 22 The Cardiovascular System Vessels and Circulation

    Chapter 22 The Cardiovascular System Vessels and Circulation

    And gives rise to the right subclavian artery, which supplies blood to the right side of the head and brain and to the right subclavian artery (supplies blood to the right arm) Left common carotid artery
  • Cooperation BiE3 + Art 12 reporting

    Cooperation BiE3 + Art 12 reporting

    Cooperation with Birds in Europe 3, and Pan-European and EU27 Red List of Birds Ian Burfield European Science & Data Manager, BirdLife International Article 12 reporting kick-off meeting Brussels, 13 October 2011 Talk structure BirdLife's assessments of European birds Impacts...