Object and Concept Recognition for Content-Based Image Retrieval

This reseach is funded by the National Science Foundation under grant nr. IIS-0097329,

Project Summary

With the advent of powerful but inexpensive computers and storage devices and with the availability of the World Wide Web, image databases have moved from research to reality. Search engines for finding images are available from commercial concerns and from research institutes. These search engines can retrieve images by keywords or by image content such as color, texture, and simple shape properties. Content-based image retrieval is not yet a commercial success, because most real users searching for images want to specify the semantic class of the scene or the object(s) it should contain. The large commercial image providers are still using human indexers to select keywords for their images, even though their databases contain thousands or, in some cases, millions of images. Automatic object recognition is needed, but most successful computer vision object recognition systems can only handle particular objects, such as industrial parts, that can be represented by precise geometric models. Content-based retrieval requires the recognition of generic classes of objects and concepts. A limited amount of work has been done in this respect, but no general methodology has yet emerged.

The goal of this research is to develop the necessary methodology for automated recognition of generic object and concept classes in digital images. The work will build on existing object-recognition techniques in computer vision for low-level feature extraction and will design higher-level relationship and cluster features and a new unified recognition methodology to handle the difficult problem of recognizing classes of objects, instead of particular instances. Local feature representations and global summaries that can be used by general-purpose classifiers will be developed. A powerful new hierarchical multiple classifier methodology will provide the learning mechanism for automating the development of recognizers for additional objects and concepts. The resulting techniques will be evaluated on several different large image databases, including commercial databases whose images are grouped into broad classes and a ground-truth database that provides a list of the objects in each image. The results of this work will be a new generic object recognition paradigm that can immediately be applied to automated or semi-automated indexing of large image databases and will be a step forward in object recognition.