Coherent Parametric Contours for Interactive Video Segmentation
Yao Lu, Xue Bai, Linda Shapiro, Jue Wang
CVPR 2016 PDF


Dataset and groundtruth: cpc.zip.
Code example for evaulating spatial accuracy and temporal consistency: cpc-eval.zip
(Visual Studio 2015 + OpenCV). Let me know if there's any problem.
Some results for CPC, MaskTracker, and SnapCut.
Folder names are offsets.

Motivation
In this paper, we aim to improve the sub-pixel-level boundary quality for video segmentation.
In previous scribble-based systems, the temporal consistency is often ignored. Hence even for
a simple rigid object shown below, boundary quality cannot be garanteed. The resulting boundary
wiggles around across frames, causing temporal boundary jitter, which is not acceptable in real
production.
SnapCut, Bai et al. 2009.
Temporal boundary jitter in detail:
SnapCut-detail

We aim to provide coherent parametric contours (CPC) that are both spatially accurate and temporally
coherent:
CPC-detail

Below is a comparison with current video segmentation systems:

Method	Input	Boundary quality	Overall efficiency	Boundary manipulation
SnapCut, Bai et al. 2009.	Non-parametric scribbles	Low	High	Difficult
MaskTracker, Adobe After Effects	Parametric curves	Low	High	Easy
Rotoscoping, Mocha	Parametric curves	Very high (Groundtruth)	Very Low	Easy
Our method, CPC	Parametric curves	High	High	Easy

Method	Handling complex topology	Handling occlusions	Handling partial boundaries	Handling non-rigidity
SnapCut, Bai et al. 2009.	Yes	Yes	No	Yes
MaskTracker, Adobe After Effects	No	No	Yes	No
Rotoscoping, Mocha	Alternative*	Yes	Yes	Yes
Our method, CPC	Alternative*	Yes	Yes	Yes

*: due to the limitation of parametric boundary representation, objects with complex topology
are decomposed into parts, where each part can still be deformable. Please see the example below.

Putting parts together for complex objects.
(Mimicking Rotoscoping)
Minion-all, len=30, start=0

CPC	Mask Tracker

Videos from our proposed dataset

Comparison with SnapCut (non-parametric)
Timelapse, len=11, start=0

CPC	SnapCut

Toyhorse, len=11, start=20

CPC SnapCut

Minion-torso, len=11, start=15

CPC	SnapCut

Boy, len=11, start=0

CPC	SnapCut

Freeway, len=11, start=5

CPC	SnapCut

Comparison with MaskTracker (parametric)
Boy, len=11, start=0

CPC	Mask Tracker

Minion-torso, len=11, start=15

CPC	Mask Tracker

Sunset, len=11

CPC	Mask Tracker

Other videos.
Hilicoptor, len=77, illumination change

CPC (without Bezier fitting)

Mask Tracker

Child, len=55, occlusion

CPC (without Bezier fitting)

Mask Tracker