Extraction of Focus Maps

The goal of the image classication technique is to extract the focus maps, or so called regions of relevance from images, to automatically determine the areas of images where the attention focuses.

To achieve this we use the presented localized blind deconvolution.

During the classication step, the locally obtained relative error values (eq. 4.7) are used to separate the areas which are more in focus (crepresents the number of classes), with the following linear formula:

F(r) = c·(Er(g, gk)−min{Er(·,·)})

max{Er(·,·)} −min{Er(·,·)} (4.8) We compared our extraction method to other methods usable for focus area estimation, e.g. gradient- or autocorrelation-based methods. In the present comparisons we consider only the basic image information content and do not exploit any specic a priori knowledge about texture

4.1.2. Extraction of Focus Maps 53

or shapes. The use of higher order optimizations like wavelet coecients, Markov random elds, texture features, etc. could also be added, but we are considering the basic capabilities of the proposed method herein.

Edge content and/or gradient based sharpness measures [17] exploit detections in edge changes for local and global sharpness measures, while autocorrelation methods also can provide local sharpness measure by checking how well neighboring pixels correlate. Practically, in-focus im-ages contain many small correlated areas, having high central peaks. For a quick visual comparison see Figure 4.7, 4.7(b) being the deconvolution-based map.

(a) (b) (c) (d)

Figure 4.7: Visual comparison for a) input image, for b) deconvolution, c) autocorrelation and d) edge content based extractions.

The proposed blind deconvolution based extraction and classication does not require any a priori information about the image contents, giving rened and well scaled relative focus esti-mations. Depending on edge measurements can give false results e.g. when there is a low blur dierence between image areas, and autocorrelation usually cannot provide enough area discrimi-nation for images with textured content.

Figure 4.8 contains samples of focus extraction with our method and the ones mentioned above for visual comparison. The rst (Fig. 4.8(a)) is an example for using the same texture with areas progressively more blurred (numbers show increasing Gaussian blur). The deconvolution-based method can provide good segmentation and visually distinguishable relative scales of the blurred areas. The second gure (4.8(b)) shows an example where the image is constructed from four dierent textures and the same blur is applied through dierent areas. Our method can both segment the blurred areas from the rest of the image and also provide a relative scaling between the dierent textures, because of the contrast weighting.

Figure 4.9 also shows another example for comparison, where texture areas were blurred and the focus maps were extracted. This example also shows the consistency of the deconvolution-based approach (second column).

4.1.2. Extraction of Focus Maps 54

(a) (b)

Figure 4.8: Example images showing blurred texture discrimination properties of our (top right), autocorrelation (lower left) and edge content based (lower right) methods. a) Top left is the input, numbers show increasingly blurred sub-areas. b) Top left is the input containing blurred areas shown with black borders.

Figure 4.9: Columns from left to right for each row: input image with a blurred area, deconvolution based map, autocorrelation based map, edge content based map. Higher intensity means higher detected focusness.

In Figure 4.10 numerical evaluation is presented for comparing our deconvolution-based and the other approaches. We used texture-sets [103] of histogram-equalized Brodatz-tiles [8] for the rst two examples. Central areas (with black rectangles) are blurred with changing strength, and the segmentation power of the methods is checked through the masking error. A ground truth mask is generated of the hand-blurred regions, calculating the ratio between the blurred area and the whole image. Then the methods are compared by taking their results and generating a mask containing the most blurred area with the same ratio as in the ground truth. The error metric used

4.1.2. Extraction of Focus Maps 55

Figure 4.10: Evaluation (lower is better) of the performance of deconvolution/ autocorrelation/

edge content extraction methods. For deconvolution both the new error measure (deconv) and the mean square error approach (deconv-mse) are investigated. Horizontal axis shows increasing blur, vertical axis shows extraction error relative to the real size of the blurred area. Blurred areas are shown with black borders. Errors larger than100%mean that areas outside the ground truth area were also falsely detected as being blurred, thus increasing the error.

was the ratio between the extracted blurred areas and the ground truth (i.e. the real hand-selected and blurred areas)

error(%) = 100· kAextracted−Arealk

Areal (4.9)

where lower values mean better extraction. The horizontal axis represent the increasing radius of the applied blur. As the gures show, deconvolution based focus extraction with the ADE measure can give good results even from low blur to high blur and the others can only achieve similar good extraction for high levels of blur, when probably every technique would be able to dierentiate the blurred areas. Also, method can achieve this consistently, proportionally dierentiating blurred areas and identifying areas with the same blur.

In Figure 4.9 we also have seen examples of consistent focus map extraction of the deconvolu-tion method, and Figure 4.15 (at the end of this secdeconvolu-tion) adds another set of comparison examples, in this case real life images, to show that the extraction is more consistent than correlation and edge based approaches not just on synthetic images.

Figure 4.11 shows examples for extracted areas on images with varying content and texture.

Figure 4.12 shows examples of using the focus extraction method to select focused targets in video.

It also can track focus changes across a video scene (Figure 4.13).

Of course, the question can arise, whether the above presented method could be used on high depth of eld images, where there is no, or just small dierence in blur among the image

4.1.2. Extraction of Focus Maps 56

Figure 4.11: Examples for focus extraction on images with various textures (top row: input, bottom row: respective focus maps).

Figure 4.12: Two examples of focus extraction on video frames: top row shows the video frames, bottom row shows their respective focus maps (higher intensity means higher focus).

areas. The answer to that is that is there is no calculable dierence in blur, the classication will depend on the eventual dierences in contrast of the image areas, since contrast also is part of the error measure used in the classication. If there is no dierence in either blur or contrast, the extracted focus map will of naturally not give any usable information, besides that the image has a at overall focusness of course. Figure 4.14 contains examples for images where there is no really percieavable focus dierence, still the classication can extract some image areas.

Figure 4.13: Following the move of focus across a scene (focus shifted from the left of the image to the stairs in the background). Upper row are the input frames, bottom row are their respective focus maps with higher intensity meaning higher focus.

4.1.2. Extraction of Focus Maps 57

Figure 4.14: Map extraction results on some high depth of eld images.

Figure 4.15: Map extraction results on images with a focused area on blurred background. Columns in each row from left to right: input, deconvolution based map, autocorrelation based map, edge content based map.