ressearch project afm

parlab Swiss National Center of Competence in Research (NCCR) on Interactive Multimodal Information Management (IM2)

research> activities

Research projects

Visual Attention (VA) on the sphere

Iva Bogdanova, Alexandre Bur

Project program

Partners: ITS/EPFL, LASA/EPFL, IDIAP, FKI/UniBE, BIWI/ETHZ

Keywords

Computer vision, visual attention, omnidirectional image processing, saliency map, conspicuity map, dynamic visual attention, non-euclidean geometry, spherical wavelets, spherical gaussian pyramid, parabolic catadioptric sensors, omnidirectional camera, dome camera, video processing, surveillance, teleconference

Purpose of the project

This project proposal aims at integrating two recent trends in computer vision: visual attention computing and omnidirectional vision.

1. Visual attention

The visual attention is the ability of vision system, biological or artificial, to rapidly select most salient and thus relevant information about the visual environment in which the system is operating. Computer visual attention models and algorithms thus allow to increasing the performance of computer vision tasks by providing generic spots of attention to be used for universal detection and selection purposes.

2. Omnidirectional vision

The omnidirectional images are obtained from catadioptric sensors. These sensors are able to view the world in all directions from the centre of projection. The main advantage of these sensors against the conventional ones consists in capturing a dynamic scene. Also omnidirectional images obtained by stitching together images from multicamera systems have similar properties.

The concrete objective is to provide a solution for computing VA in spherical geometry in order to generate saliency maps and spots of attention which behave homogenously in any direction of vision.

Methodology

VA is computed in spherical non-Euclidian geometry and provides homogeneous omnidirectional behaviour, as shown in figure 1 below.

Figure 1: With spherical computation the spots are found in all directions (3 images left) but with conventional computation (image right), no spots on the poles will be recognized. Green spot #2 located at the pole in the images left is not detected in the image right.

Results

A new paradigm for computing visual attention in omnidirectional scenes was proposed and developed for the first time in order to get rid of distortions found in conventional approaches, We have worked out the computation of visual attention in the spherical geometry. The reasoning for associating the omnidirectional images and the sphere is mathematically grounded. Based on it, together with performing a multi-scale analysis on the sphere, we have developed:

1. A visual attention algorithm that operates in spherical geometry [1], [3].

2. A dynamic visual attention model based on multi-resolution motion estimation on the sphere [2].

3. Application of the developed visual attention to different types of omnidirectional images (spherical, hyperbolic and parabolic)[3].

Both theoretical and implementation results are delivered.

References

[1] I. Bogdanova, A. Bur, H. Hügli, Visual Attention on the Sphere, IEEE Transactions on Image Processing, [submitted]

[2] A. Bur, P. Wurtz, R. Müri, H. Hügli, Dynamic visual attention: motion direction versus motion magnitude, Proc. SPIE Vol 6806, Jan., 2008

[3] I. Bogdanova, A. Bur, H. Hügli, The spherical approach to omnidirectional visual attention, EUSIPCO Lausanne, Switzerland, [submitted]