2.4 Multilayer MRF modelization
2.4.1 Application to motion segmentation and change detection
The proposed algorithm has been tested on real video sequences [27, 29]. We also compare the results to motion only and color only segmentation (basically a monogrid model similar to the one defined for the feature layers but without inter-layer cliques). The mean vectors and covariance matrices were computed over representative regions selected by the user. The number of motion and color classes is known a priori but classes on the combined layer are estimated during the segmentation process. Fig. 2.8 shows some segmentation results. Note that the head of the men on this image can only be separated from the background using mo-tion features. Clearly, the multi-layer model provides significantly better results compared to color only and motion only segmentations. See Fig. 2.9 to compare the performance of the proposed method with the one from [130] on the Mother and Daughter standard sequence.
Some of the contours are lost by [130] (the sofa, for example) while our method successfully identifies region boundaries. In particular, our method is able to separate the hand of the mother from the face of the daughter in spite of their similar color. This demonstrates again that the proposed method is quite powerful in combining motion and color features in order to detect boundaries visible only in one of the features. We can also handle occlusion and more complex motions using a modified multilayer model presented in [29]. The model has also been successfully applied to color-textured image segmentation [30, 31].
Finally, we present the application of multilayer modeling for automatic change detection on airborne images taken with moving cameras [2]. Essentially, we want to extract the accurate silhouettes of moving objects or object-groups in images taken by moving airborne vehicles in consecutive moments. This problem is solved in two steps: first a coarse (but robust) image registration is performed for camera motion compensation, then the aligned input image pair is segmented into moving (foreground) objects and background. Main challanges are camera motion, noise and the parallax artifacts caused by the static objects having considerable height (buildings, trees, walls etc.) from the difference image.
A three-layer MRF model is constructed on a graph G whose structure is shown in Fig. 2.7. The final goal is to perform a binary segmentation of the images into foreground (fg) and background (bg) classes. For the segmentation, two type of features are extracted from the aligned image pair: d(s), the gray level difference of the corresponding pixels in the registered images; andc(s), the maxima in the local correlation function around pixels.
The sites ofGare arranged into three layers:Sd,Sc andS∗, each layer having the same size as the image latticeS. We assign to each pixels ∈ S a unique site in each layer: e.g.sd is the site corresponding to pixelson the layerSd. We denotesc ∈Sc ands∗ ∈ S∗ similarly.
The segmentation is obtained by assigning a label ω(.)to all sites ofG from the label-set:
L ,{fg, bg}. The labeling ofSd/Sc corresponds to the segmentation based on thed(.)/c(.) feature, respectively; while the labels at theS∗ layer present the final change mask.
In Fig. 2.10, we show some results obtained on three pairs of aerial images. For each pair, we show the ground truth change masks obtained by manual segmentation, the multi-layer MRF results and a simple fusion obtained as a logical AND operation on the change masks of two monolayer segmentations based on each features. The increased precision of
the multi-layer model is clearly visible. Another application of multilayer modeling can be found in [62].
2.4. Multilayer MRF modelization 45
(a) First input image (b) First input image (c) First input image
(d) Second input image (e) Second input image (f) Second input image
(g) Fusion of two MRFs (h) Fusion of two MRFs (i) Fusion of two MRFs
(j) Multi-layer MRF (k) Multi-layer MRF (l) Multi-layer MRF
(m) Ground Truth (n) Ground Truth (o) Ground Truth
Figure 2.10: Experimental results [2].
I N T HIS C HAPTER :
3.1 Introduction . . . . 48 3.2 Higher order active contours . . . . 49 3.3 The ‘gas of circles’ HOAC model . . . . 50 3.3.1 Stability analysis . . . . 50 3.3.1.1 Parameter constraints . . . . 52 3.3.2 Geometric experiments . . . . 53 3.4 Phase field model . . . . 55 3.5 Equivalence of the HOAC, phase field, and MRF models . . . . 57 3.6 Discretization . . . . 58 3.6.1 Quantization of the functionφ . . . . 58 3.6.2 Discretization of the domainD. . . . 59 3.6.3 Discretization of the energy functional . . . . 60 3.6.3.1 Relationship between the parameters of the contour and field energies . . . . 62 3.6.3.2 Parameters of the discrete energy functional . . . . 62 3.7 Markovian interpretation . . . . 62 3.7.1 Singleton potential . . . . 64 3.7.2 Doubleton potential . . . . 65 3.7.3 Long range potential . . . . 66 3.8 The ’gas of circles’ MRF model . . . . 66 3.8.1 Experiments . . . . 67 3.9 Application in remote sensing . . . . 70
3.
The ’gas of cicrcles’ MRF model The ’gas of cicrcles’ MRF model The ’gas of cicrcles’ MRF model The ’gas of cicrcles’ MRF model The ’gas of cicrcles’ MRF model The ’gas of cicrcles’ MRF model The ’gas of cicrcles’ MRF model The ’gas of cicrcles’ MRF model The ’gas of cicrcles’ MRF model The ’gas of cicrcles’ MRF model The ’gas of cicrcles’ MRF model The ’gas of cicrcles’ MRF model The ’gas of cicrcles’ MRF model The ’gas of cicrcles’ MRF model The ’gas of cicrcles’ MRF model The ’gas of cicrcles’ MRF model The ’gas of cicrcles’ MRF model The ’gas of cicrcles’ MRF model The ’gas of cicrcles’ MRF model The ’gas of cicrcles’ MRF model The ’gas of cicrcles’ MRF model
O
bject extraction remains one of the key problems of computer vision, which can be stated as finding regions in the image domain occupied by a specified object or objects. The solution often re-quires high-level knowledge about the shape of the objects. HOAC models integrate shape knowledge via the inclusion of explicit long-range dependencies between region bound-ary points. Herein, we will show how to set the parameters of the HOAC model to fa-vor regions consisting of any number of ap-proximately circular connected components, each component having approximately the same, specified radius. This yields the ’gas of circles’ HOACs.A subsequent reformulation of HOAC mod-els as phase fields can be interpreted as
real-valued continuum Markov random fields.
Discretizing the phase field GOC model, we will develop an equivalent ‘gas of circles’
Markov random field model that assigns high probability to regions in the image domain consisting of an unknown number of circles of a given radius. The MRF model is con-structed in a principled way, thereby creat-ing an equivalent MRF. The model can be used as a prior for object extraction when-ever the objects conform to the ‘gas of cir-cles’ geometry, e.g. tree crowns in aerial images or cells in biological images.
Here we present a theoretical and ex-perimental analysis of these models. The performance is demonstrated on various syn-thetic images as well as on the problem of tree crown detection in aerial images.