The LIP automatic segmentation algorithm test test
It is commonly observed that visual information provides
a precious help to the listener under degraded acoustic conditions. Visual
cues are effectively used by human beings to improve speech intelligibility.
The motivation of the present work is to extract information for automatic
audiovisual lipreading.
Here, an original algorithm is proposed for lip segmentation
under natural conditions, the only requirement being that the micro-camera
is mounted on a light helmet worn by the speaker in order to be fixed w.r.t.
the head. The video sequence should contain the region of the face spanning
from chin to nostrils.
Left: An RVB image sequence of mouth movements (soft
red make-up lips).
Right: Sequence of final lip shape superposed on the
initial sequence.
Automatic Lip Tracking: Bayesian Segmentation
and Active Contours In A Cooperative Scheme
Main Authors: M.Liévin and P.Delmas.
Abstract
An algorithm for speaker's lip contour extraction is presented here. A
color video sequence
of speaker's face is acquired, under natural lighting conditions and
without any particular make-up.
First, a logarithmic color transform is performed from RGB to HI (hue,
intensity) color space. A
statistical approach using Markov random field modelling helps to segment
the mouth area, integrating red hue and motion into a spatiotemporal neighbourhood.
Simultaneously, a Region Of Interest (ROI) and relevant boundaries points
are automatically extracted. Next, an active contour using spatially varying
coefficients is initialised with the results of the preprocessing stage.
Performance of active contours are greatly improved when initialisation
is close to the desired
features. Finally, an accurate lip shape with inner and outer borders
is obtained with good quality
results in this challenging situation.
Results
Top Left: An RVB image sequence of mouth movements.
Top Right: Sequence of final lip shape superposed
on the initial sequence.
Bottom Left: Initial points and mouth corners (see
P.Delmas)
Bottom Right: Final shape of the mouth using active
contours (see P.Delmas)
Top Left: An RVB image sequence of mouth movements.
Top Right: Sequence of final lip shape superposed
on the initial sequence.
Bottom Left: Initial points and mouth corners (see
P.Delmas)
Bottom Right: Final shape of the mouth using active
contours (see P.Delmas)