Homepage IMIX project VIDIAM, dialogue management and the visual channel

On this page you will find the newest information about VIDIAM.

IMIX is a multimodal dialogue system for question answering. IMIX answers factual questions in a medical domain, generating its answers in multimedia (speech, text, pictures, videos). The user may use speech, text, and pen strokes on a tablet. It is a collaboration of 7 research projects. VIDIAM concerns the dialogue manager.

VIDIAM collaborates with PARADIME and ROLAQUAD to form a dialogue manager. VIDIAM emphasises multimodality and wishes to combine state of the art techniques from multimodal dialogue systems and qa dialogue systems. PARADIME emphasises the possibilities of direct implementation of Dynamic Interpretation Theory using a production rule type system. ROLAQUAD wishes to deliver classifier modules for the dialogue manager, such as dialogue act classification and problematic communication indication.

People

• Rieks op den Akker, coordinator
• Boris van Schooten, postdoc
• Hilco Kats has done a D assignment on dialogue managers for clarification dialogues. (homepage)

Publications

• Trung H. Bui, Boris van Schooten, and Dennis Hofs. Practical dialogue manager development using POMDPs. In SIGDIAL 2007, pages 215-218, 2007.

• Boris van Schooten, Sophie Rosset, Olivier Galibert, Aurélien Max, Rieks op den Akker, and Gabriel Illouz, Handling speech input in the Ritel QA dialogue system. In: Interspeech 2007.

• Boris van Schooten and Rieks op den Akker, Follow-up utterances in QA dialogue. In: Traitement Automatique des Langues, 46(3), 2005 (Appeared in January 2007). Pages 181-206.

• Boris van Schooten and Rieks op den Akker, Multimodal follow-up questions to multimodal answers in a QA system In: Tenth international symposium on social communication, Cuba. January 2007. Pages 469-474.

• Mariet Theune, Emiel Krahmer, Boris van Schooten, Rieks op den Akker, Charlotte van Hooijdonk, Erwin Marsi, Wauter Bosma, Dennis Hofs and Anton Nijholt. Questions, Pictures, Answers: Introducing Pictures in Question-Answering Systems In: Tenth international symposium on social communication, Cuba, January 2007. Pages 450-463.

• Rieks op den Akker and Boris van Schooten, Vervolguitingen in vraag-antwoorddialoogsystemen. In DIXIT, tijdschrift over toegepaste taal- en spraaktechnologie, 4(1), pages 10-11, June 2006.

• Akker, R. o. d., Bunt, H., Keizer, S., and Schooten, B. v. (2005). From question answering to spoken dialogue: towards an information search assistant for interactive multimodal information extraction. In Interspeech 2005, pages 2793-2796.

Other documents

• Invited talk at the LCT group at the university of Bolzano on 21 february 2008: powerpoint | PDF.

• Talks given at the 13 september IMIX meeting: Vidiam (ppt), Demonstrator (ppt).

• Talk given at a local meeting in Twente: PDF

• IMIX talk, 7 september 2006, slides: Powerpoint.

• Limsi talk 28 nov 2006 about a dialogue manager implementation in Ritel, slides: large, pdf | small, ps | large, pdf

• IMIX talk 3 march 2006, slides: Powerpoint | Postscript

• Demo 2 dialogue functionality, 16 dec 2005 HTML | Postscript | PDF

• Functional specification VIDIAM, 6 oct 2005 Postscript | PDF

• Evaluation plan of Paradime and Vidiam

• IMIX talk 17 nov 2005, slides: Powerpoint | Postscript

• Original project proposal for VIDIAM | Update VIDIAM proposal, 14 september 2004

• More stuff can be found on the IMIX site.
  IMIX main page | IMIX Demo-2 page | The new Demo-3 page | IMIX public page

Corpora

• The multimodal follow-up question corpus (you can still do the experiment here)
• The second utterance corpus
• The corpus of example dialogues