We wish the dialogue manager to present the system in a "search engine" type of way, making explicit that the question answering mechanics are imperfect, search engine like. For example, a prompt presenting an answer may be stated like: "I found the following answer: ...", instead of just presenting the answer. See also the functional specification.
It is also likely we want the user to know that the answers are excerpts from documents. It may be very useful if the user is able to view more of the document the answer came from.
The information state will contain some (epistemic) information about the following things:
An overview of possible dialogue acts follows. All dialogue acts are candidates for being supported in the first demonstrator version. This classification is a pragmatic one, in which:
(1) the term "dialogue act" is interpreted as IMIX-specific operations on the system's information state, while
(2) the user is sufficiently able to identify the distinct intentional meaning of each act.
A system Question is a request for a specific piece of information in the information state.
An utterance that fulfills a previous specific information request of the other participant.
The "initial user question", in the case of IMIX, is a question, requesting factual, general technical information.
The question is likely to be self contained. While a question describing a specific situation (a diagnostic question) need not be supported by IMIX, the question may contain references to the asker (like: "Can I get RSI?"). A QA Question may by a request rather than a question, for example, "Give me a list of ...". However, since this is likely to bite the QA technology, we consider this an exception. Possibly, such non-question requests, if common enough, may be handled in future versions of the QA engines or dialogue manager.
These are questions asked as a reaction to the system's answer. The questions concern the application domain. These questions need not be self contained, but are reactions to the system's answer, and implicitly or explicitly refer to previous utterances. Many multimodal and visual utterances are likely to fall in this category.
We may further subdivide these questions.
Follow Up Question: pose a new, related, question. The user wants more information because either the answer presented information that changed the user's information need, or the user proceeds to a next step in his search agenda. Either way, the goal here is to formulate a new query by selecting and augmenting bits of information from previous utterances.
Domain Clarification Question: clarify a detail in the answer. Here, the answer is satisfactory but incomplete, and the user wants more information related to the answer. In terms of topic shifts, the user is more likely to return to the previous topic after the clarification question has been answered, because the clarification is a subtask. In terms of information source, the user is more likely to want to know something that is found in the same document that the answer came from.
Examples:
Clarification of a term: "what does [word] mean?"
Clarification of content of images: "what does this red thing on the right mean?"
Clarification of source: "from what document was this answer taken?"
Requests by the system to provide more information for the QA request, in order to arrive at a useable QA query.
Example:
Ik begrijp het woord [woord] niet.
Example:
U:kan ik een ziekte krijgen?
S: Welke ziekte bedoel je?
The system's presentation of a QA query will be called QA Answer. It is more complex and specific than answer. It contains some meta information, such as confidence level, and a formulation of the question submitted to the QA.
Request for repetition and re-representation. It is a signal that something was not understood, and that exactly the same information has to be presented again, possibly using another style or information channel. The request may contain information about the desired representation.
This is not only the standard speech (and pen) recognition problem solving, but also: request of the user to replay a video, to zoom in on a picture, to present information as text, or as speech, or a list, etc., and perhaps to present the same information in a summarised form. And: request of the system to spell a word out, to use a synonym, to explain an anaphoric reference.
Note: it may be possible that non-understanding is signalled without meaning a request to repeat the information.
Acts that are meant to verify some item in the common ground. This may be seen as similar to the representation request. There is a gliding scale, going from: "what did you say?" to "did you say X or Y?" to "did you say X?".
Generally, any acts that are meant to update the dialogue control status other than the "normal" dialogue flow through the other acts. This may mean explicitly talking about dialogue acts, or hinting at them.
Examples:
"I asked you a question"
"Did you ask me a question?"
"I don't want to answer this question"
"Let's start over again"
The dialogue manager will have some operational model of the search task. For example: first obtain a proper query using a dialogue, then present the query and enabling tweaking and feedback, then formulate a new query. The user must be aware of (part of) the task being done.
Some of these activities should be negotiable. These may be seen as specific cases of Dialogue Control.
This is still something of a "miscellaneous" category. Some examples.
In the QA Question formulation phase, the user may signal a re-start.
If the answer is dead wrong, a reformulation of the query is in order. System and user may signal the start of a re-formulation task.
Instead on relying on dialogue structure cues, dialogue structure and topic shifts may be negotiated. This is analogous to but more general than "back" and "forward" buttons in a web browser.
The system may give feedback on search failures, using confidence information or (possibly) conditional results and presupposition problems.
Requests for broadening or narrowing the search.
Ex. "I need more search terms".
Ex. "Could you tell me more about ...".
Ex. "But I didn't ask about ...".
The user gives an opinion about the QA answer. While we want to include a small "evalation form" at the end of each dialogue, the user is likely to give (implicit) evaluative feedback during the dialogue, which may be used to refine the query.
Ex. "ja maar, ik bedoelde ..."
Ex. "maar wat heeft dat te maken met ...?"
Ex. "gaat dit wel over ...?"
Dialogue openings and closings, thank yous, etc. While we probably won't use these acts for analysis, we should account for them by responding reactively.
Acts related to concepts in which the system or user is subject. While we probably won't answer meta questions, we should account for some of them and tell the user that they cannot be answered.
Examples:
What does this system do?
What does this button mean?
Do you know something about <general-non-domain-subject>?
Where can i find your maker?
Any utterances addressing turn taking issues.