Archive for the 'Multimedia Search' Category

A Probabilistic Ranking Framework using Unobservable Binary Events for Video Search

Monday, May 19th, 2008, posted by Djoerd Hiemstra

by Robin Aly, Djoerd Hiemstra, Arjen de Vries, and Franciska de Jong

CIVR 2008, Niagara Falls This paper concerns the problem of search using the output of concept detectors (also known as high-level features) for video retrieval. Unlike term occurrence in text documents, the event of the occurrence of an audiovisual concept is only indirectly observable. We develop a probabilistic ranking framework for unobservable binary events to search in videos, called PR-FUBE. The framework explicitly models the probability of relevance of a video shot through the presence and absence of concepts. From our framework, we derive a ranking formula and show its relationship to previously proposed formulas. We evaluate our framework against two other retrieval approaches using the TRECVID 2005 and 2007 datasets. Especially using large numbers of concepts for retrieval results in good performance. We attribute the observed robustness against the noise introduced by less related concepts to the effective combination of concept presence and absence in our method. The experiments show that an accurate estimate for the probability of occurrence of a particular concept in relevant shots is crucial to obtain effective retrieval results.

The paper will be presented at the ACM International Conference on Image and Video Retrieval CIVR 2008 in Niagara Falls, Canada

[download pdf]

The Effectiveness of Concept Based Search for Video Retrieval

Monday, September 24th, 2007, posted by Djoerd Hiemstra

by Claudia Hauff and Robin Aly and Djoerd Hiemstra

In this paper we investigate how a small number of high-level concepts derived for video shots, such as Sports, Face, Indoor, etc., can be used effectively for ad hoc search in video material. We will answer the following questions: 1) Can we automatically construct concept queries from ordinary text queries? 2) What is the best way to combine evidence from single concept detectors into final search results? We evaluated algorithms for automatic concept query formulation usingWordNet based concept extraction, and we evaluated algorithms for fast, on-line combination of concepts. Experimental results on data from the TREC Video 2005 workshop and 25 test users show the following. 1) Automatic query formulation through WordNet based concept extraction can achieve comparable results to user created query concepts and 2) Combination methods that take neighboring shots into account

[download pdf]

Building Detectors to Support Searches on Combined Semantic Concepts

Friday, July 27th, 2007, posted by Djoerd Hiemstra

by Robin Aly, Djoerd Hiemstra, and Roeland Ordelman

Bridging the semantic gap is one of the big challenges in multimedia information retrieval. It exists between the extraction of low-level features of a video and its conceptual contents. In order to understand the conceptual content of a video a common approach is building concept detectors. A problem of this approach is that the number of detectors is impossible to determine. This paper presents a set of 8 methods on how to combine two existing concepts into a new one, using a logical AND operator. The scores for each shot of a video for the combined concept are computed from the output of the underlying detectors. The findings are evaluated on basis of the output of the 101 detectors including a comparison to the theoretical possibility to train a classifier on each combined concept. The precision gains are significant, specially for methods which also consider the chronological surrounding of a shot promising.

[download pdf]

StreetTivo: Multimedia analysis in your living room

Monday, June 18th, 2007, posted by Djoerd Hiemstra

Great performance of Peter Boncz…

Evaluation of Multimedia Retrieval Systems

Friday, February 16th, 2007, posted by Djoerd Hiemstra

by Djoerd Hiemstra and Wessel Kraaij

In this chapter, we provide the tools and methodology for comparing the effectiveness of two or more multimedia retrieval systems in a meaningful way. Several aspects of multimedia retrieval systems can be evaluated without consulting the potential users or customers of the system, such as the query processing time (measured for instance in milliseconds per query) or the query throughput (measured for instance as the number of queries per second). In this chapter, however, we will focus on aspects of the system that influence the effectiveness of the retrieved results. In order to measure the effectiveness of search results, one must at some point consult the potential user of the system. For, what are the correct results for the query “black jaguar”? Cars, or cats? Ultimately, the user has to decide.

Download author version.

[Information Retrieval]: Guest lecture by Arjen de Vries

Tuesday, October 24th, 2006, posted by Djoerd Hiemstra

Arjen de Vries from CWI Amsterdam will give a guest lecture on Multimedia Retrieval. Where: Langezijds 1812.
When: 25 October 2006, 13.45 h. - 15.30 h.
What: Multimedia Retrieval

New project member: Robin Aly

Thursday, July 6th, 2006, posted by Djoerd Hiemstra
Robin Aly has joined our group to work on audio and video retrieval. His project is sponsored by the CTIT SRO NICE (Strategic Research Orientation: Natural Interaction in Computer-mediated Environments). In this project we will closely cooperate with people from the Human Media Interaction group.