| Home | People | Research | Publications | Demos |
| News | Jobs |
Prospective
Students |
About | Internal |
| Demo of Layered Dynamic Texture | |||||||||||||||||||||||||||
|
These are examples of motion segmentation using the layered dynamic texture (LDT). We also compare to segmentations using mixtures of dynamic textures (DTM) or generalized PCA (GPCA). The video is available in both AVI format (DivX, playable with RealPlayer or Windows Media player) and Quicktime format (H.264).
This is a case where the LDT corrects a
noisy DTM segmentation (imprecise boundaries and spurious segments)
This is an
example where the DTM produces a poor segmentation (e.g. the border between two
textures erroneously marked as a segment), which the LDT corrects.
This is an
example where the DTM produces a poor segmentation (e.g. the border between two
textures erroneously marked as a segment), which the LDT corrects.
This is a difficult case. The initial DTM segmentation is very poor. Albeit a
substantial improvement, the LDT segmentation is still noisy.
This is a difficult case. DTM splits
the two water segments incorrectly (the two textures are very similar). The LDT substantially
improves the segmentation, but the difficulties due to the great similarity of water patterns
prove too difficult to overcome completely.
These are examples of segmenting real video using the layered dynamic texture.
For K=2, both LDT and DTM segment the static background from the moving ferris wheel.
However, for K=3 regions, the plausible segmentation, by LDT, of the foreground into two
regions corresponding to the ferris wheel and a balloon moving in the wind, is not matched by
DTM. Instead, the latter segments the ferris wheel into two regions, according to the dominant
direction of its local motion (either moving up or down), ignoring the balloon motion.
For K = 3 regions, LDT segments the windmill into regions corresponding
to the moving fan blades, parts of the shaking tail piece, and the background. When segmenting
into K = 4 regions, LDT splits the fan blade segment into two regions, which correspond
to the fan blades and the internal support pieces. On the other hand, the DTM segmentations
for K = {3, 4} split the fan blades into different regions based on the orientation (vertical or
horizontal) of the optical flow.
This is an example showing the interesting property of LDT segmentation:
that it tends to produce a sequence of segmentations which captures a hierarchy of scene
dynamics. The whirlpool sequence contains different levels of moving and
turbulent water. For K = 2 layers, the LDT segments the scene into regions containing moving
water and still background (still water and grass). Adding another layer splits the moving
water segment into two regions of different water dynamics: slowly moving ripples (outside of
the whirlpool) and fast turbulent water (inside the whirlpool). Finally for K = 4 layers, LDT
splits the turbulent water region into two regions: the turbulent center of the whirlpool, and
the fast water spiraling into it.
A crowd moves in a circle around a pillar. The left side of the scene is less
congested, and the crowd moves faster than on the right side.
The crowd moves
with three levels of speed, which are stratified into horizontal layers
A crowd
gathers at the entrance of an escalator, with people moving quickly around the edges. These
segmentations show that LDT can distinguish different speeds of crowd motion, regardless of
the direction in which the crowd is traveling.
The LDT segments a highway scene
into still background, the fast moving traffic on the highway, and the slow traffic that merges into
it.
A whirlpool, where the turbulent water component is segmented
from the remaining moving water.
A windmill scene, which
the LDT segments into regions corresponding to the windmill (circular motion), the trees waving
in the wind, and the static background.
Examples of synthetic circular motion. The LDT was learned with n=2, and
is capable of synthesizing the different speeds of the circular rings.
An example with three textures: two types of water and moving jellyfish.
Note that the LDT (n=20) can simultaneously synthesize all three motions, which have distinct speeds.
An example with three textures: water, fire, and jellyfish.
The fire texture is more stochastic than the smoother water texture,
or fast jellyfish texture. The LDT was learned with n=20.
An example with two textures: fire and smoke.
The fire texture is chaotic, while the smoke is smoother.
Some small errors in the segmentation cause small artifacts in the synthesis
(e.g. top-left of the smoke texture). The LDT was learned with n=20.
|
|||||||||||||||||||||||||||
©
SVCL
