This is the html version of the file http://www.ipsi.fraunhofer.de/~stein/publications/fullpapers/SteinGullaMuellerThiel98.pdf.
G o o g l e automatically generates html versions of documents as we crawl the web.
To link to or bookmark this page, use the following url: http://www.google.com/search?q=cache:Rr2-y2-52o0J:www.ipsi.fraunhofer.de/~stein/publications/fullpapers/SteinGullaMuellerThiel98.pdf+%22Abductive+Dialogue+Planning+for+Concept-Based+Multimedia+Information+Retrieval'&hl=en

Google is not affiliated with the authors of this page nor responsible for its content.
These search terms have been highlighted: abductive dialogue planning for concept based multimedia information retrieval 
Page 1
129
In: Fankhauser, P. & Ockenfeld, M. (Eds.), Integrated Publication and Information Systems. 10 Years of
Research and Development at IPSI. Samkt Augustin: GMD – Forschungszentrum Informationstechnik,
1998, pp. 129-148.
Abductive Dialogue Planning for Concept-Based Multimedia
Information Retrieval
Adelheit Stein, Jon Atle Gulla, Adrian Müller, and Ulrich Thiel
Abstract. In this article, we describe methods and techniques for context-dependent dialogue
planninginanintelligentmultimediainformationretrievalsystem.Supportingcontent-oriented
retrieval interaction requires both semantic representations and interpretations of searches,
under consideration of the dialogue context. A conceptual query, that is, a specification of some
information need in terms of abstract concepts, may be interpreted in many alternative ways.
This, in turn, affects the determination of what items are relevant for retrieval. To deal with
these problems, the MIRACLE prototype combines knowledge-based retrieval and dialogue
components which employ abductive inference in order to interpret user inputs. Vague or
ambiguous queries and other (unexpected) dialogue control acts of users are interpreted with
respect to the internal knowledge bases, i.e., the semantic domain model, the dialogue model,
and the dynamically created dialogue history. The generated interpretations are then negotiated
interactively to determine what users actually intended and which dialogue continuations they
would prefer. The user’s choices are stored as constraints in the dialogue history and are used
to dynamically adapt the system’s future behavior to individual user preferences.
1 Introduction
Ineverydaylife,mostpeopletrytosolveinformationproblemsbyaskingotherswhomight
be better informed or can help in finding information. Usually, both parties engage in an inter-
action in order to achieve a mutual understanding of the information problem. Often the infor-
mation provider then can offer a solution by conveying some factual data the information seeker
asked for. However, there are situations where this is not possible: the information need might
be more complex or “open-ended” in that it is underspecified at the beginning and there is no
immediate, well-defined problem-solving procedure at hand. Here, it is crucial for the partners
tosuccessivelyestablish–inadditiontoasharednotionoftheproblem–amutualunderstanding
of the relevance criteria that should be used to decide whether information items constitute a
solutiontotheproblemwithinagivensituationorcontext.Hence,incaseswheretheinformation
problem is non-trivial, retrieval interaction is far more complex than the simple provision of
facts and resembles a cooperative problem-solving dialogue.
Skilled librarians and human intermediaries to online information systems have a well-
tried repertoire of tactics and strategies for assisting end-users in such complex information-
seeking situations (cf., e.g., Saracevic et al., 1997). In fully computerized information retrieval
(IR) systems, some of these functions will be fulfilled by “intelligent information agents” which
aim to provide effective interaction support including the capability to engage in a cooperative
dialogue, especially with inexperienced and casual users.
Informationsystemsofthepastsuchasonlinelibrarycatalogueswererestrictedtoalimited
range of possible usages, mostly due to their brittle user interfaces. These systems showed very
little interaction capabilities and simply processed user queries. Soon the shortcomings of the
“matching” paradigm (cf. Bates, 1986) and the need for more complex interaction facilities
becameobvious.Subsequently,informationretrievalwasregardedasaninteractiveprocessthat
Page 2
130
involved cycles of query refinements, result inspections, etc. In early experimental systems (cf.
Salton & McGill, 1983) users could react to individual retrieval results, accepting or rejecting
the documents one by one. The decision to implement this relevance feedback mechanism on
the object level, rather than on the concept level, was mostly due to the prevailing representation
method for information items, i.e., “documents” or their surrogates, which was based on vector
spaces.The user’s reaction then led to a recompilation of the coefficients of the query vector.
Oddy’s (1977) proposal for an IR technique solely based on this kind of interaction showed
that, in principle, IR resembled an information-seeking dialogue rather than a programming
activity as suggested by the matching paradigm.
A deeper insight into the retrieval process shows that even conceptual retrieval systems
that pertain to the matching paradigm are not the ultimate solution. Even if we can determine
the proper meaning of a user query in terms of a semantic theory, how can we be sure that this
is what the user is looking for? Providing representations is only the prerequisite for conceptual
search. Many ways of representing the contents and structure of a multimedia document appear
plausible, and there are many more ways of determining whether an information object is
relevant. Given a conceptually defined information need, users might use different strategies
(e.g., employ different domain rules for querying the database) to obtain (partially overlapping)
alternative result sets. If the task of maintaining these access paths to the database can be left
to the system, users can concentrate on their primary goal, i.e., selecting relevant items. The
relevance assessment process can be supported by the system, for instance, by making explicit
the retrieval rules applied, showing alternative ways of handling a query or negotiating a mod-
ification of an unsatisfactory query. These capabilities require a far more complex dialogue and
user interface than those currently in use.
Interfaces that act as intelligent mediators between users and other components of an
informationsystemaimtointeractivelyprovidetask-orientedassistance,exploitingthedialogue
context. Cooperative dialogue systems must be able to not only respond to specific user requests
butalsototaketheinitiativesuggestingsuitabledialogueoptionsandproblem-solvingstrategies
to the user (cf. Stein et al., 1999). Hence, representation and evaluation of the semantics and
pragmatics of the dialogue are important issues for these kinds of systems.
From our perspective, an integrated solution to the problem of intelligent multimedia
information retrieval must include the following features:
• multimedia indexing combining feature extraction with semantic annotation,
• methods for bridging the gap between the users’ conceptualization of information needs
and the system’s way of representing items and interpreting information requests, and
• dialogue structures and strategies that are appropriate in a given context and situation.
This article addresses the last two issues, focusing on the handling of ambiguous user
inputs and dialogue planning aspects. Section 2 provides an overview of related work on intel-
ligent IR. In Section 3 we describe the theoretical framework and baseline components of the
MIRACLEretrievalsystem.Asabductivereasoningplaysakeyroleforboththeretrievalengine
and the dialogue manager, this inference technique is discussed in Section 4. The dialogue
modelandbasicfeaturesofthedialoguecomponentarepresentedinSection5,analyzingtypical
interaction examples. In Section 6 we illustrate specific interface features of the first application
of MIRACLE, which provides access to a database in the domain of art history.
Page 3
131
2 Related work on intelligent IR and dialogue modeling
Theprocessoffindinginformationinlargedatarepositoriesinvolvesavarietyofreasoning
tasks, ranging from problem definition to relevance assessment. Intelligent IR systems are
expected to support users in these tasks, and over the last decades a growing number of ap-
proaches to employing automatic reasoning techniques have been proposed.
The first approaches were influenced by experimental question-answering systems in the
sixties and seventies. Their main concern was to replace the document representation, which
atthattimewasbasedoninvertedfilesortermvectors,bymoreexpressivealternatives:semantic
networks and frames. The representation allowed semantic analysis and provided a basis for
knowledge-based matching procedures. Since then, various projects demonstrated the feasibil-
ity of extracting factual information from texts for the purpose of semantic indexing and auto-
matic creation of hyperlinks. Here, knowledge-based methods were combined with linguistic
tools such as “partial semantic parsers” to overcome the limits of statistical text processing.
Whereas knowledge-based matching extends the potential for semantic retrieval, other AI
techniques have been employed to achieve even more inferential power. The applicability of
rule-based reasoning, which was demonstrated by the first working expert systems, stimulated
a lot of experiments in IR. Most of these efforts took the expert system approach literally and
devised “automatic search intermediaries”. Exploiting the semantic and to some limited extent
pragmatic knowledge about a specific problem domain, these systems were able to assist inex-
periencedusersinqueryformulationandsearching.Theprototypeswereintentionallyrestricted
tofunctionsthatsimulateahumanintermediary(thedataareassumedtobestoredinatraditional
IR system), whereas other expert system designs aimed at increasing the responsiveness and
flexibility of the retrieval system. For instance, in the I3R system (Croft & Thompson 1987)
the user can directly explore parts of the term–concept–document network, which is the basis
of the retrieval process. The integrated probabilistic retrieval engine was later replaced by a
spreading activation mechanism (Croft et al., 1989) to apply the model of retrieval as plausible
inference suggested by van Rijsbergen (1989): in order to estimate the relevance of a document
D
with respect to a query
Q
, the probability
p (D
→
Q)
must be estimated.
The idea of logic-based information retrieval is more general than previous IR models,
since it abstracts away from the inference mechanism, which can be probabilistic reasoning or
some other logical procedure, as well as from the data representation format (Nie, 1992). As a
consequence, it encompasses a variety of approaches which go beyond the heavily domain-
dependent rule-based reasoning of traditional expert systems. Some approaches employ a prob-
abilistic inferential model based on Bayesian dependence nets. For instance, the INQUERY
retrieval system (Callan et al., 1992) estimates the probability that the user’s information need
I
(as expressed in the query
Q
) is satisfied by a document
D
, that is,
p (I
|
D)
, by combining
evidence along the one or more paths between query and document nodes.
We can conclude that most existing logic-based IR models mainly rely on more or less
restricted forms of deductive inference in a first-order or probabilistic logic. However, since the
consequence (the query) is known and we want to know the set of potential premises (the
documents), inferential processes might be appropriate which allow us to find those premises.
Both (probabilistic) abductive reasoning (Müller & Thiel, 1994, Thiel et al., in this volume) and
Bayesian networks can accomplish this. In fact, for propositional logic and discrete networks
they have been proven to be equivalent (cf., for example, Poole, 1993).
Page 4
132
Mainstream research in IR has concentrated on the task of processing a given query while
neglecting the fact that the retrieval effectiveness is also heavily dependent on the human-
computer interaction. Some researchers did not assume a single shot retrieval process: this is
reflected by concepts like “relevance feedback” and the notion of “retrieval as interaction” (cf.,
e.g., Oddy, 1977, Belkin & Vickery, 1985, Bates, 1986, Croft & Thompson, 1987, Thiel 1990,
Ingwersen, 1992). Whereas these approaches tried to explore the nature of retrieval dialogues
and proposed interface designs to support the interaction, little work has been done on making
explicit, or explaining, the system’s decisions. To be able to also capture such aspects it has
been proposed to model the entire interaction as a cooperative human-computer dialogue where
the contributions of the two participants are represented as a complex web of conversational
acts, tactics, and strategies (see Stein & Thiel, 1993, and Belkin et al., 1995).
Inthiscontext,relatedresearchcanbefoundinthefieldsofhuman-computercollaboration
and intelligent user interfaces (cf., e.g., Terveen, 1995, Maybury, 1993, Maybury & Wahlster,
1998). Particularly relevant are AI-based approaches to collaborative planning, discourse mod-
eling, and adaptive user modeling. Computational dialogue and user models have mostly been
developed for natural language applications, such as text planning systems employing user
modelingcomponents(cf.,e.g.,Jamesonetal.,1997)andtask-orientedspokendialoguesystems
(cf., e.g., Maier et al., 1997). Focusing on the agents’ beliefs and goals, most of these dialogue
modelsandsystemspresupposewell-definedtasks,e.g.,travelplanningorschedulingmeetings,
while the tasks in information retrieval contexts are quite different in nature. On the other hand,
most state-of-the-art retrieval systems do not employ elaborate dialogue models and dialogue
planning techniques. Although there has been little crossover between the above mentioned
research on dialogue and IR, we believe that methods from dialogue modeling/planning and
intelligent information retrieval can feasibly be combined in a conversational retrieval system.
3 A framework for conversational multimedia IR
Semantic access to multimedia information, logic-based retrieval methods, and context-
dependent dialogue support form the central components of the framework proposed here. As
pointedoutintheprevioussection,noneofthesethreeissuesisentirelynewtotheIRcommunity
and related areas. We only rarely find, however, approaches that focus on a combination or
integration of these issues and research areas, in particular, in terms of application in complex
systems. For example, elaborate computational models of discourse and human-computer col-
laboration have been developed in AI and HCI respectively, but they often neglect the specific
problems related to information retrieval.
Many users of IR systems have no well-defined information need and problem-solving
plan at the beginning of a session, and their information needs and strategies gradually change
as the dialogue develops. This often results in ambiguous user queries an intelligent retrieval
system should be able to deal with. However, this is only part of the whole picture. Ambiguous
and changing information needs/strategies over longer phases of the interaction demand an
elaborate account of the dialogue structure and context. We assume that users will benefit from
a flexible user guidance that provides useful information-seeking strategies while allowing for
deviations in a natural way, for example, if an explanation is needed or the strategy has to be
negotiated between system and user. This type of interaction resembles a conversation to a large
extent, and hence we refer to it as conversational information retrieval. Resolving naturally
Page 5
133
occurring ambiguities, both with respect to query/retrieval aspects and to dialogue aspects, is
the principle concern of the research presented here.
Before turning to the discussion of our approach, we briefly describe the system architec-
ture of MIRACLE (Multimedia Information RetrievAl of Concepts in a Logical Environment)
as shown in Figure 1. The first application of MIRACLE provides access to a large database in
the domain of art history, which consists of SGML documents (artist’s biographies, reference
articles, etc.), factual knowledge, and descriptions of thousands of works of arts. The prototype
is implemented in C, Smalltalk and Prolog and runs on System V and BSD UNIX platforms.
Theindexingcomponentcombinesprobabilistictextindexingwithrepresentationmethods
for pictures derived from content-based multimedia retrieval approaches (cf. Müller & Kut-
schekmanesch, 1996). The abductive retrieval engine (AIR) works on a knowledge base com-
prising a semantic domain model, a model of the document/object structure, and the semantic
counterparts(conceptindex)tothesyntacticindextermsassignedtothemultimediadocuments.
The abductive dialogue component (ADC) mediates between the user and the retrieval system.
To achieve an appropriate amount of user guidance, this component uses the internal dialogue
model and a repository of dialogue control rules together with a dynamically constructed dia-
logue history.
A user’s query is a description of what the user is looking for and is unlikely to match
database entries directly. To deal with these descriptions, we distinguish between an intensional
Abductive
Retrieval
Engine
(AIR)
document/object structure model
concept index
syntactic index
Indexing Component
Query
Expanded
Graphical User Interface
Abductive
Dialogue
Component
(ADC)
semantic domain model
dialogue model & dialogue rules
dialogue history
Stratified Knowledge Base:
Input
Response
Figure 1: System architecture of MIRACLE
DB
Page 6
134
(or:conceptual)representationofthedomain,andtheextensionalmodel,i.e.,instancesretrieved
from the database (cf. Thiel et al., in this volume). A query is formulated at the intensional level,
and the abductive inference mechanism generates query reformulations with respect to the
available information structures, i.e., the inference process is set up to map from conceptual
query statements to arbitrary information elements.
This retrieval model requires distinguishing between at least three global phases of the
interaction, i.e., query formulation, inspection of the generated query interpretations, and in-
spection of instances retrieved from the database. Real retrieval dialogues, however, are often
far more complex, and additional interaction options are to be provided. For example, users
should be enabled to compare and evaluate the generated query interpretations before selecting
the appropriate one to be executed, to ask for further explanations, and to correct some previous
decision, etc. To keep track of such complex interaction structures and to assist the user in
findingan orientation, the dialoguemanager must relyon an elaboratemodelof dialogue. Based
on this model the dialogue manager dynamically builds up a structured dialogue history which
is exploited to plan the subsequent dialogue steps.
4 Abductive inference for information retrieval
The term abduction was coined by Charles S. Peirce. He defined abduction to be the
explanation of the “surprising observation of a certain fact” and, referring to human reasoning,
said in a lecture in 1903: “The abductive inference comes to us like a flash. [...] it is the idea
of putting together what we had never before dreamed of putting together which flashes the new
suggestion before our contemplation.” (cited after Buchler, 1955, p. 184).
Combining partial knowledge to form a more general concept of the world is usually
regarded as the most promising property of abductive inference. One of the crucial problems
of using logics in IR is the need to model vague and often inconsistent properties of information
by the formal and precise means of logical formalisms. A number of logic mechanisms have
been developed which cope with uncertainty, default knowledge and the like. Unfortunately,
the use of a calculus which allows the treatment of vague facts and rules does not automatically
imply a higher degree of robustness. In fact, a logical theory will hardly cover all aspects of a
real-world domain. Thus, one needs an inference process which is able to recover from (or
maintain) a partially inconsistent or insufficient model of the domain.
As opposed to deduction, abduction infers explanations for a given observation (see also
Figure 2). The basic inference step can roughly be described as a kind of inversion of the Modus
Ponens. Whereas in deductive systems the Modus Ponens is based on material implication,
abduction usually tries to find a causal relationship with respect to an observation and a given
theory. But abduction need not be restricted to this kind of inference. Levesque (1989) suggests
not to demand a direct causal relationship between both formulae but to view the resulting
formula as one of the possible reasonable explanations for the observation. When we use ab-
duction within an IR system, the relations are implicational and not necessarily causal. Since
not all of the explanations generated need to be valid, we refer to each explanation (proof) as
a feasible hypothesis.
An abductive system generates all explanations for an observation with respect to a theory
and a suitable form of logical implication. A logical theory
T
is defined over a language
L
of
Page 7
135
well-formed formulae, built from variables, constants, and predicates. Given a theory
T
and a
sentence
a
which needs to be explained in terms of
T
the abductive reasoning process will yield
a set of explanations (or hypotheses)
H
so that
T
∪
H
|—
a
.
In accordance with van Rijsbergen’s proposal, we assume that a retrieval method attempts
to prove that a document
D
entails (a part of) the query
Q
, that is,
D
→
Q
(van Rijsbergen,
1989). Now, let
T
be a logical theory providing a model of the semantic domain, e.g., an
appropriate ontology. Additionally, the theory may capture knowledge about the document
structure as well. When a user’s query statement
Q
is the sentence to be “explained”, we can
use the process of abductive reasoning to infer an intensional specification
H
(in terms of the
underlying database) of the relevant documents
D
. In other words, the inference process pro-
poses a way of interpreting a query statement by providing a number of query reformulations.
These query interpretations/reformulations can be presented to the user, who then may choose
the one to be executed accessing the database. As it is beyond the scope of this paper to explain
the details of abductive retrieval, we refer to Müller & Thiel (1994), Müller (1997), and Thiel
et al., in this volume, for detailed discussions.
Inthefollowingsectionsweconcentrateontheinteractionaspects,showingawayinwhich
the complex results provided by the retrieval engine can be communicated to and negotiated
with the user. The system must take into account already established knowledge about the goals
and plans of the user acquired during interaction. For instance, an inferred query interpretation
accepted by the user should not be disregarded in subsequent interaction steps, but can be
maintainedasasetofconstraints
C
whichprunethespaceofvalidhypotheses.Theseconstraints
can be used as filters according to the following definition: an explanation
H
, which yields
T
∪
H
|—
a
, is valid if
T
∪
H
∪
C
is consistent, where
C
(a set of constraints) is constructed
from elements of
T
. Note that this definition might conflict with multiple hypotheses
H
. Since
hypotheses can be mutually inconsistent, the inference process might find no consistent expla-
nation if unstructured hypotheses are accumulated during a retrieval session. Thus, the dialogue
component needs to maintain a collection of constraints which model the goals of each indi-
vidual step in the retrieval dialogue dynamically building up a structured dialogue history.
from: a → b (rule)
a
(fact)
-------------------------------
infer: b
(conclusion)
Deduction
Abduction
from: a → b All paintings have a creation date.
(rule)
b
“Mona Lisa” has a creation date.
(observation)
----------------------------------------------------------------------------------
infer: a
Probably, “Mona Lisa” is a painting. (hypothesis)
Figure 2: Deductive vs. abductive inference
Page 8
136
5 Dialogue modeling and abductive dialogue planning
Effectiveinformationretrievalsupportmustbebasedonanelaborateaccountofthehuman-
computer interaction that allows monitoring the ongoing dialogue and evaluating the dialogue
history. This requires a comprehensive model of information-seeking dialogue as well as an
active mechanism which allows the dialogue component to exploit the internally represented
knowledge about the user’s individual dialogue behavior and interests. Further, since users’
information needs and dialogue goals often change in the course of a retrieval session, they
might need assistance from the dialogue system to find a good strategy for proceeding with the
retrieval session.
5.1 Dialogue model: COR and Scripts
The dialogue model we have developed and partly applied in previous retrieval systems
(forinstance,MERIT,cf.Stein&Thiel,1993)hasbeenenhancedandformalizedforintegration
into the MIRACLE prototype
1
. The model comprises two tiers that mutually constrain each
other. The Conversational Roles model (COR) specifies the general local interaction options
(dialogue acts, embedded clarification dialogues) for all possible dialogue states. Scripts are
used as global task-oriented plans to guide the user through the retrieval session. This form of
user guidance, recommending appropriate problem-solving steps, is helpful to users unfamiliar
with the system. However, it is impossible to anticipate all of the problematic situations that
can occur during interaction. To also cover unexpected situations, COR models additional
dialogue options not included in any script specification.
COR is intended to hold for all kinds of mixed initiative information-seeking dialogue,
modeling the pragmatic function of dialogue contributions and the role changes between infor-
mationseekerandprovider(see,e.g.,Sitter&Stein,1992/96,andSteinetal.,1999forextensive
descriptions). The basic units of dialogue are atomic dialogue acts, i.e., the actual graphical or
linguisticactionsofthetwoparticipants(AandB).Genericdialogueacts/movesarecategorized
according to the main purpose (“illocutionary point”) expressed. These include, for example,
requests, offers, promises, etc. Dialogue acts are elements of superordinated complex moves
which are assigned the same type and label as the respective atomic acts. The COR model of
the entire dialogue is represented as a recursive transition network (Figure 3) consisting of
dialogue states and transitions between the states, i.e., the moves.
A dialogue is either initiated by a
request
for information from the information seeker (A)
or the information provider’s (B)
offer
to search for information and afterwards to present the
retrieved items (
inform
). Sequences of moves starting in dialogue state 1 and returning to that
state are called dialogue cycles (for example,
request
→
reject_request
builds a short but
complete cycle). Bold arcs represent expected moves that comply with the role expectations;
the sequences 1–2/2’–3–4–1 are cycles that represent “ideal” courses of action, where no with-
drawals or rejections occur. Thus, for any of the dialogue states the possible follow-up moves
and possible action sequences can be described by the COR model.
1. Using different specifications, the dialogue model has also been used in the SPEAK! system as a part
of its dialogue manager which built one component in elaborate natural language generation frame-
work (cf., for example, Bateman et al., in this volume, for a discussion of the SPEAK! approach).
Page 9
137
Moves are also represented as recursive transition networks (not displayed here). They can
consist of atomic/primitive acts, other (complex) moves, and embedded sub-dialogues (e.g.,
clarification dialogues). Since these subordinated moves and embedded dialogues are optional,
a dialogue move can consist of a single atomic dialogue act, and even the entire move may be
omitted in certain situations, i.e., when the respective intention can be inferred from the context
(for instance,
promise
is often skipped in case the requested information can be given imme-
diately). The whole COR model is specified as a set of related state transition networks for
dialogues and several types of moves.
The example dialogue given in Figure 4 (see also screenshots of the MIRACLE interface
in Section 6), allows us to illustrate how a hierarchical dialogue history is built up when tra-
versing the recursive COR networks. When the user asks for information about abstract art in
Spain, the system finds the query ambiguous and suggests two different query interpretations.
The query corresponds to a COR
request
act, and the system initiates a clarification dialogue
asking (
request
) the user to choose one of the generated interpretations for accessing the data-
base.Theuserchoosesinterpretation2butimmediatelywithdrawsthischoiceand(afteranother
short dialogue cycle) asks to use query interpretation 1 instead.
For each dialogue act of user and system, the dialogue manager creates an entry in a
hierarchical dialogue history, as illustrated in Figure 5. The history stores all acts performed
during the interaction (as leaves of the tree, e.g.,
s: request
) as well as the user queries and
constraints associated with generated query interpretations. Acts following each other like
offer
evaluate
(A,B,T)
evaluate (A,B,T)
reject_request (B,A,T)
withdraw_request/accept (A,B,T)
withdraw_offer/promise
inform (B,A,T)
dialogue
(_,_,m)
withdraw_request (A,B,T)
withdraw_offer (B,A,T)
reject_offer (A,B,T)
withdraw_request
reject_request (B,A,T)
withdraw
(_,_,T)
withdraw_
offer (B,A,T)
reject_offer
(A,B,T)
2
1
3
4
5
6
7
7’
8
2’
withdraw_
offer/promise
withdraw_
request/accept
Dialogue (A,B,T)
dialogue states
dialogue moves
A information seeker
B information provider
T type of dialogue or move: m = meta-level; r = retrieval
–
parameter A or B
withdraw_
inform (B,A,T)
request
(A,B,T)
accept
(A,B,T)
offer
(B,A,T)
promise
(B,A,T)
Figure 3: COR network for dialogues and subdialogues
Page 10
138
and
accept_offer
are inserted as siblings in the tree structure. Tacit transitions (e.g., a
promise
tofulfillarequest)areinsertedasemptyacts(e.g.,betweenthe
request
andthefollowing
inform
act), since this is required by the COR model. In our example dialogue the first request move
is decomposed into a primitive
request
(the user’s first query) and a
subdialogue
(initiated by
the system asking the user to choose one of the generated query interpretations). When a sub-
dialogue is initiated like that, it means that the interaction in this subordinate part is founded
on what has happened at the higher level. Also, if the COR model is traversed so that the same
kind of move is triggered several times at the same level of the dialogue, like the two request
moves for query 1 and query 1a, we divide the dialogue into different dialogue cycles.
An important aspect of the dialogue history is the storing of constraints. When the user
has chosen a particular query interpretation, the corresponding proof tree is sent to the dialogue
manager and stored in the history together with the act itself.
C1
and
C2
in Figure 5 represent
the two query interpretations generated in our example (the proof trees of the query interpreta-
tions are given in Figure 10). These proof trees are referred to as constraints, since they may
be used to constrain the interpretation of later queries. Note that in our example
C1
is stored in
the history but is later not considered relevant any more, since the user withdrew this choice
right away. By contrast,
C2
in the next dialogue cycle is considered highly relevant: after the
user’s modified query (1a) the system does not bother the user any more by asking again which
of the query interpretations should be used – but keeps to the last relevant one.
. . .
U: enters query terms: about: “abstract art”, country: “Spain”
request (query 1)
S: Which query interpretation do you want? Please choose:
request
β1
: artist should be born in Spain
β2
: works of art exhibited in Spain linked to artist by . . .
U: checks both and selects
β2
for searching the database
inform
U: interrupts search by clicking on the “withdraw” button
withdraw_inform
S: Now, the following options are available, you may
offer
α1
: choose another query interpretation
α2
: modify your previous query
α3
: enter a completely new query
U: chooses
α1
accept_offer
S: displays the last query interpretations again
inform
U: selects
β1
and clicks on the “search database” button
request
S: Here are the retrieved hits: . . . (shows list or table)
inform
U: decides to modify query and adds profession: “painter”
request (query 1a)
S: I’m searching the database, using query interpretation
β1
again.
promise
S: Here are the retrieved hits: . . .
inform
. . .
Figure 4: Fragment of an example dialogue in MIRACLE
Page 11
139
The complete dialogue history, thus, is a hierarchically structured analysis of what has
happened at the dialogue act level and what decisions the user has made in the course of the
interaction. As we will come back to in the next section, this is the only contextual information
we need to deal with unexpected user actions and sequences of related user queries.
As opposed to the COR model, dialogue scripts give us structured guidelines for the
information retrieval session. Based on a multi-dimensional classification of information-seek-
ing strategies, the scripts proposed by Belkin et al. (1995) describe (or prescribe) prototypical
sequences of dialogue acts/steps that are useful to fulfill a particular task and strategy. A script
includes all possible system actions and all recommended user actions at the various stages of
aretrievaldialogue,i.e.,asubsetoftheCORdialogueacts.Weusearecursivetransitionnetwork
formalismwithpreconditionsandpostconditionstorepresentscripts,andthetransitionscontain
references to COR acts (see Figure 6). The preconditions decide when an act is available,
whereas the postconditions ensure that the necessary actions are executed by the system.
S1 is the standard script used to govern the retrieval interaction in MIRACLE. Script S2
is used to structure special types of meta-dialogues and is triggered when the user has entered
dialogue(u,s,r)
request
u:request
. . .
query 1
dialogue(u,s,r)
request
s:request
Which interpretation?
promise
ε
inform
u:inform
β2
withdraw_inform
offer
s:offer
Whichdialogueoption?
dialogue(u,s,m)
accept
u: accept
inform
s:inform
evaluate
α1
showsqueryint.again
ε
clicks on “withdraw”
displays hits
I’m searching, using
β1
displays hits
. . .
u:withdraw
withdraw_promise
ε
dialogue(u,s,m)
. . .
u = user; s = system;
ε
= empty
r = retrieval dialogue; m = meta-dialogue
dialogue(u,s,r)
dialogue
cycles
dialogue
moves
C1
C2
C
constraints
β1
request
u: request
inform
s:inform
evaluate
u: evaluate
promise
ε
wants to modify query
request
u:request
query 1a
inform
s:inform
promise
s: promise
. . .
Figure 5: COR analysis (history tree) of the example dialogue
Page 12
140
an ambiguous dialogue act (query or dialogue control act) that is not contained in script S1. In
our example dialogue these scripts were instantiated one after the other: S1 for the two top-
level retrieval dialogue cycles (starting with query 1 and 1a, respectively), and S2 right after
the user’s first query (negotiating the query interpretations) as well as in the next subdialogue
(for negotiating the interpretation of the user’s ambiguous
withdraw
). Naturally, dialogue con-
trol acts like rejections, withdrawals, and unexpected help requests of the user cannot be pre-
defined in the retrieval script but are an important means for the user to take control of the
dialogue. All these unexpected acts and subdialogues interrupt the dialogue course proposed
or recommended by the retrieval script and call for some exceptional treatment. As they tend
to express a desire to change the current direction of the dialogue, special mechanisms for
dealing with these cases are needed, as discussed in detail in the next section.
5.2 Abductive Dialogue Component (ADC)
ThedialoguemanagerformsaseparatecomponentinMIRACLEandprovidesaninterface
to the retrieval engine. ADC administers the script being used and the COR analysis of the
dialogue. Implemented as recursive state transition networks, the COR model and the scripts
12
S11
S12
S13
S14
S15
S16
S17
S18
1
2
3
4
5
6
7
8
9
10
11
1
2
3
4
5
25 request(u,s,r,query(_)):
u posts query
26 request(s,u,r,query_int(one,_)):
s presents one query interpretation
27 request(s,u,r,query_int(many,_)):
s presents query interpretations
28 inform(u,s,r,choice):
u selects query interpretation
29 inform(s,u,r,result(_)):
s shows list of data
30 offer(s,u,r,menu):
s offers list of things to do
31 accept(u,s,r,new_query):
u selects to formulate new query
32 request(u,s,r,modify_query):
u wants to modify query
33 accept(u,s,r,modify_query):
u wants to modify query
34 reject_request(s,u,r,query_int(no,_)):
s found no query interpretation
35 inform(s,u,r,form(_)):
s shows new query form
36 inform(s,u,r,old_form):
s shows old query form
S23
S21
S22
S24
25 offer(s,u,m,act_int):
s presents act interpretations
26 accept(u,s,m,act_int):
u selects an interpretation
27 inform(s,u,m,act_exe):
s informs that act is executed
28 reject_offer(u,s,m,act_int):
u rejects all interpretations
29 inform(s,u,m,act_exe):
s informs that unambiguous
act is being executed
S1: Retrieval Dialogue
S2: Meta-Dialogue
u
user
s
system
r
retieval dialogue
m
meta-dialogue
state of script
transition (dialogue act)
Figure 6: Structure of two scripts
Page 13
141
offer available dialogue acts at the various steps of the interaction. As the dialogue develops,
transitions in the script are fired, and their associated COR acts are used to change the state of
the COR model. By collecting all possible transitions leading out from the active state of the
script, the dialogue manager can present a list of recommended dialogue acts to the user. Users
can suspend the current retrieval script by dialogue control acts not included in the script, such
as
withdraw
and
reject
. These acts are referred to as unexpected acts and are analyzed by the
dialogue component using abduction and the dialogue history.
As described in Section 4, the task of an abductive system is to find potential facts (forming
a hypothesis) that would explain a given observation. The observation to be explained by the
ADC is the unexpected act of the user. If the hypothesis is true, it logically implies the obser-
vation. A domain theory consisting of dialogue control rules (see Figure 7) defines the dialogue
concepts found in the observation (i.e., generic properties of the respective dialogue state) and
establishes logical relationships with other concepts. Concepts not defined by the theory are
referredtoasabducibles,andthesecanbeincludedinthehypothesisexplainingtheobservation.
In order to offer the user a simple but general set of dialogue options of how to continue, these
options are to be interpreted in a particular context. Here, the context is the dialogue history,
and the dialogue control rules map from concrete system actions and properties of the dialogue
history to generic COR acts. When the user performs an unexpected dialogue control act, the
ADC generates a hypothesis that together with the history and the control rules imply the
unexpected act: History ∪ Dialogue_control_rules ∪ Hypothesis |— Unexpected_act.
Often, there are several hypotheses that imply the unexpected control act. Each hypothesis
forms an interpretation of the act, and the user has to choose the “correct” one. Just like in the
query interpretation case, abduction is used to interpret the user’s input and to find a more
precise reformulation of it. Similar techniques of evaluating the dialogue history to generate
hypotheses about the user’s intentions have been used, e.g., by Hobbs et al. (1993) and McRoy
& Hirst (1995). Their approaches focus, however, on linguistic ambiguities in natural language
discourse, whereas we deal here with non-linguistic contextual ambiguity.
1. resume(X) → change_act(X)
If the user intends to reformulate the content of an act, she changes the original act.
2. modify(request(X,Y,Z,query(Q))) → change_act(request(X,Y,Z,query(Q)))
If the user wants to modify a previous query, she changes the original query.
3. request(X,Y,Z,W) ∧ change_act(request(X,Y,Z,W)) → change_input(X,Y,Z)
If the user intends to change her previous request, she changes her inputs.
4. inform(X,Y,Z,W) ∧ change_act(inform(X,Y,Z,W)) → change_input(X,Y,Z)
If the user intends to change her previous inform act, she changes her inputs
.
5. inform(X,Y,Z,W) ∧ change_input(X,Y,Z) → withdraw(inform(X,Y,Z,W))
If the user has given information and wants to change this input, she withdraws her inform act
.
Figure 7: Some dialogue control rules
Page 14
142
Unexpected acts are detected by deviations from the current retrieval script (e.g., S1). Even
though such acts do not comply with the suggestions in the script, they are still assumed to
follow the general interaction patterns modeled in COR, which provides more interaction op-
tionsandhencemoreflexibility.Thescriptisthenterminatedorsuspended,andasetofdialogue
control rules
M
(see Figure 7) are used to reformulate the unexpected acts together with the
dialogue history in terms of concrete system actions.
Consider the
withdraw
act in our example dialogue, where the user is interrupting the use
of query interpretation β2 in the search. This act is unexpected, since it is not contained in the
retrieval script but is taken from the general interaction options offered by the COR model. As
such the user’s withdrawal interrupts the current dialogue course, although it is not obvious
what the user wants to do in this situation, for example, choosing the other query interpretation
for search or posting a new query. When an unexpected act like this is detected, the system
assembles all relevant acts from the dialogue history into a set
D
. It then tries to find hypotheses
H
about the user’s concrete interaction wish, where
H
is given by the formula
H
∪
M
|—
D
.
The user’s withdrawal of the chosen query interpretation is internally represented as
withdraw_inform(u,s,r,choice(1,2)), and the ADC uses the dialogue control rules in the reverse
direction to find possible proofs for the observation.
A proof tree is valid when all non-proven sentences are either abducibles or come from
the dialogue history. In our example dialogue three interpretations of the withdraw act are
inferred: (1) choose another query interpretation, (2) modify the previous query, or (3) restart
the retrieval session with a completely new query. Figure 8 shows the proof trees of the first
two interpretations of the withdraw act. The starting point of each proof tree is the dialogue
α
2:
Modify
previous
query
α
1:
Choose
another
query inter-
pretation
modify(request(u,s,r,query(1)))
withdraw(inform(u,s,r,choice(1,2)))
resume(inform(u,s,r,choice(1,2)))
change_act(inform(u,s,r,choice(1,2)))
change_input(u,s,r)
inform(u,s,r,choice(1,2))
withdraw(inform(u,s,r,choice(1,2)))
request(u,s,r,query(1))
change_input(u,s,r)
inform(u,s,r,choice(1,2))
change_act(request(u,s,r,query(1))
Figure 8: Proof trees of two interpretations of the user’s withdraw act
Page 15
143
control act to be explained. In the first proof rule 5 is used to explain this act. Using rules 3 and
1 to infer one of the premises of rule 5, we get a complete proof tree with one reference to the
dialogue history and one abducible that serves as the interpretation of the control act (e.g.,
resume(inform(u,s,r,choice(1,2))) in Figure 8).
Another important utilization of the dialogue history is the interpretation of new queries
in the light of previously interpreted queries and the use of constraints. When a query interpre-
tation is chosen by the user, the constraint set corresponding to the interpretation is sent to the
dialogue manager. The set is inserted together with the dialogue act into the history (like
C1
and
C2
in Figure 5) so that it can later be accessed by the query interpretation module. However,
not all the constraint sets recorded are considered relevant for later query interpretation. When
a new query is to be interpreted, all relevant constraints have to be accumulated and sent to the
retrievalmodule.Thisisdonebyincludingallconstraintsfoundinthecycleofthelatestinserted
dialogue act, as well as all constraints found in higher level cycles.
When interpreting the user’s extended query in our example (query 1a), only
C2
is added
as constraint set to the abductive reasoning process (
C1
belongs to a cycle not superordinated
to the latest inserted act but to a different completed dialogue cycle). Hence, the last chosen
query interpretation (β1) is still considered relevant, and the system informs the user that it is
used again for the current search. If this, for some reason, does not comply with the user’s
intention, the user may interrupt the search process again, using some control button from the
graphical interface like reject or withdraw. From the retrieval module’s point of view, the dia-
logue manager is accessible through an abstract data type that adapts the reasoning process to
the state of the dialogue and stores the whole interaction history.
6 Working with MIRACLE
In the following we illustrate how our example dialogue is realized in MIRACLE and how
the interaction looks from the user’s point of view. In the introductory sequence of our example
the user selects a domain of interest (here: art history). Then the retrieval dialogue proper
continues with the first query. The user enters “abstract art” in the
about
field (a free-text search
field), “Spain” in the
country
and “?” in the
artist
fields (see also Figure 9).
The retrieval engine finds two interpretations of this query and presents the first of them
to the user as displayed in the front window in Figure 9. This is done by simply paraphrasing
therelevantinternalrulesintheformofalist.Rulesthatapplytoallqueryinterpretationsappear
in the upper part, and the lower area includes active rules for the currently shown interpretation
(black bullets) and non-active rules (light-colored bullets) that apply to the other interpreta-
tion(s).The proof trees of the two query interpretations are shown in Figure 10. The formulae
are presented as directed graphs, directions indicating the inference sequence. For example, the
query interpretation graph β1 represents the following inference steps: Assuming some artist
A
is connected to “abstract art”, if
A
is born in Spain, then he or she is a qualified artist. The
left hand side of the proof tree shows that this interpretation will evaluate the “aboutness”
statementbyexaminingthetextualcomponentsofthedocumentcollection,i.e.,thebiographies
and other full text articles.
A
, the missing link for the two parts of this query reformulation, is
restricted to be a concept of type artist() and, thus, is a key for the document collection.
Page 16
144
Using the slider shown on the MIRACLE screen, the user can inspect the second interpre-
tation (β2). Within this query interpretation, it is not the artist’s place of birth, but the place of
exhibition of a work of art, which is restricted to “Spain”. Here, the textual retrieval part
(about(abstract_art), etc.) and the pictures of work of arts in the database are finally linked by
the artist
A
via the computable relation picture_description().
After choosing interpretation β2, the user asks the system to search the database. Techni-
cally speaking, MIRACLE is asked to find all models of the inferred formulae, which has been
presentedasinterpretationβ2.Thistruthassignmentiscomputedbottom-up,startingfrombasic
predicates and recursively propagating potential instantiations to the root of the proof graph.
Each unique model will be returned as a hit to the user. In our example, however, the user
interrupts the search and chooses interpretation β1 for a new search in the database. The system
displays a large number of retrieved hits, and the user decides to narrow the query (restrict the
set of models) by inserting
profession
“painter”. As mentioned before, this time the system
Figure 9: Query and query interpretation β1
Page 17
145
only considers the first interpretation of “Spain”, and it constrains the inference process so that
the narrowed query is to be interpreted in the same manner as in β1. The additional constraint
is to filter the set of qualifying artists
A
by the condition artist_profession(
A
,painter). After
computing the models, the system presents a list of artists with links to biographies and other
referencearticles.Afterhavinginspectedthislist,theuserfinallyadds
artist
“Miro”and
subject
of works of art
“?” to the query and gets the biography of the painter Joan Miró and a number
of links to pictures of works of art associated with Miró in the database.
The example discussed shows how the functions of the retrieval engine and dialogue
manager are intertwined in the MIRACLE system. The dialogue planning is based on formal
constraints which are defined on the semantic properties of the objects involved. For example,
the association of a given retrieval result with one of the interpretations of the user’s original
query allows the explanation of the relevance decision of the system by referring to the assump-
tions underlying the interpretation.
More generally, the example dialogue illustrates an efficient way of transforming a rather
vagueinformationneed,whichinthebeginningisstatedasatopicalrequirement(“aboutness”),
into a complex conceptual statement about entities represented in the database and their rela-
tionships. Hence, a very precise search can be performed, which is a prerequisite for many
digital library applications (cf. Thiel et al., in this volume).
Query
interpretation
β2
Query
interpretation
β1
picture_description(P,A)
artist(A)
country(Spain)
document(D,A,abstract_art)
about(abstract_art)
place_birth(A,Spain)
country(Spain)
document(D,A,abstract_art)
about(abstract_art)
place_exhibition(A,Spain)
artist(A)
Figure 10: Proof trees of the two generated query interpretations
Page 18
146
7 Conclusions
We have introduced a theoretical framework for intelligent conversational information
retrieval and its application in a multimedia information retrieval system, the MIRACLE pro-
totype. Combining concept-based information retrieval with a comprehensive dialogue model,
the system is capable of assisting the user actively in her information-seeking dialogue through
all phases of the interaction. Both the retrieval engine and the dialogue component employ
abductive reasoning as the basic inference mechanism to resolve ambiguous user inputs and to
generate cooperative system responses. The abductive retrieval engine of MIRACLE generates
interpretations (or plausible reformulations) of ambiguous user queries and offers these inter-
pretations to the user for further negotiation. Employing a two-layered conversational dialogue
model (COR and scripts) to dynamically build up a structured dialogue history, the dialogue
component analyzes unexpected control acts of the user in light of this history and offers the
usersuitablecontinuationsofthedialogue.Thus,thesystemmonitorsandguidestheinteraction
as the dialogue develops.
Using examples of the user-system interaction in MIRACLE, we discussed how the re-
trieval engine and the dialogue manager interact with each other to construct, depending on the
user’s input, a semantically and pragmatically coherent dialogue course. The retrieval engine
and the dialogue manager interact through an abstract data type, which constrains the results
of the inference process with respect to the current state of the dialogue and provides means
for expressing and maintaining choice points of the dialogue history. As a side effect of raising
a set of constraints, the search space of the retrieval engine shrinks to a reasonable size and the
response times improve.
The experiments with the MIRACLE prototype showed the feasibility of combining a
powerful conceptual retrieval method with flexible dialogue management. Thus, two main
strands of IR research were united in one approach, i.e., retrieval as inference and retrieval as
interaction. Aside from this synthesis, which may have its own merits from the standpoint of
IR theory, the MIRACLE experiment provided a sound foundation for the design and develop-
ment of future information systems. These will not always require the full inferential power of
abductive reasoning, e.g., when the mapping of users’ query terms to database entities leaves
less space for ambiguities and the dialogue options are more confined. Nevertheless, the lessons
learnedinacomplexsystemdesignareguidelinesforthedesignofpractical,robustinformation
retrieval systems.
References
Bateman,J. A.,Teich,E.,andStein,A.(1998).Speechgenerationinamultimodalinterfaceforinformation
retrieval: The SPEAK! system. in this volume.
Bates, M. (1986). An exploratory paradigm for online information retrieval. In Brookes, B., ed., Intelligent
Information Systems for the Information Society. Amsterdam: North-Holland, pp. 91–99.
Belkin, N. J., Cool, C., Stein, A., and Thiel, U. (1995). Cases, scripts, and information seeking strategies:
On the design of interactive information retrieval systems. Expert Systems and Applications
9(3):379–395.
Belkin, N. J., and Vickery, A. (1985). Interaction in Information Systems: A Review of Research from
Document Retrieval to Knowledge-Based Systems. London: The British Library.
Page 19
147
Buchler, J., ed. (1955). Philosophical Writings of Peirce. New York: Dover.
Callan, J. P., Croft, W. B., and Harding, S. M. (1992). The INQUERY retrieval system. In Proceedings of
the 3rd International Conference on Database and Expert Systems Application. Berlin and New
York: Springer, pp. 78–83.
Croft, W. B., Lucia, T., Cringean, J., and Willett, P. (1989). Retrieving documents by plausible inference:
An experimental study. Information Processing and Management 25(6):599–614.
Croft, W. B., and Thompson, R. (1987). I3R: A new approach to the design of document retrieval systems.
Journal of the American Society for Information Science 38(6):389–404.
Hobbs, J. R., Stickel, M. E., Appelt, D. E., and Martin, P. (1993). Interpretation as abduction. Artificial
Intelligence 63(1-2):69–142.
Ingwersen, P. (1992). Information Retrieval Interaction. London: Taylor Graham.
Jameson, A., Paris, C., and Tasso, C., eds. (1997). User Modeling: Proceedings of the Sixth International
Conference, UM ’97. Vienna and New York: Springer Wien New York.
Levesque, H. (1989). A knowledge-level account on abduction. In Proceedings of the 11th International
Joint Conference on Artificial Intelligence (IJCAI ’89). New York: ACM Press, pp. 1061–1067.
Maier, E., Mast, M., and LuperFoy, S., eds. (1997). Dialogue Processing in Spoken Language Systems.
ECAI’96 Workshop, Budapest, Hungary. Berlin and New York: Springer.
Maybury, M. T., ed. (1993). Intelligent Multimedia Interfaces. Menlo Park, CA: AAAI Press/MIT Press.
Maybury, M. T., and Wahlster, W., eds. (1998). Readings in Intelligent User Interfaces. San Mateo, CA:
Morgan Kaufman.
McRoy, S. W., and Hirst, G. (1995). The repair of speech act misunderstandings by abductive inference.
Computational Linguistics 21(4):435–478.
Müller, A. (1997). Abductive retrieval of structured documents. PODP Special Issue on Computer and
Mathematical Modelling 26(1):15–28.
Müller, A., and Kutschekmanesch, S. (1996). Using abductive inference and dynamic indexing to retrieve
multimedia SGML documents. In Ruthven, I., ed., MIRO 95. Proceedings of the Final Workshop on
Multimedia Information Retrieval. Berlin and New York: Springer (eWiC, electronic Workshops in
Computing series).
Müller, A., and Thiel, U. (1994). Query expansion in an abductive information retrieval system. In Pro-
ceedings of the Conference on Intelligent Multimedia Information Retrieval Systems and Manage-
ment (RIAO ’94), Vol. 1, pp. 461–480.
Nie, J. (1992). Towards a probabilistic modal logic for semantic-based information retrieval. In Belkin,
N., Ingwersen, P., and Pejtersen, A., eds., Proceedings of the SIGIR ’92. New York: ACM Press, pp.
140–151.
Oddy, R. (1977). Information retrieval through man-machine-dialogue. Journal of Documentation
33(1):1–14.
Poole, D. (1993). Probabilistic horn abduction and Bayesian networks. Artificial Intelligence 64:81–129.
van Rijsbergen, C. J. (1989). Towards an information logic. In Belkin, N., and van Rijsbergen, C., eds.,
Proceedings of the SIGIR ’89. New York: ACM Press, pp. 77–86.
Salton, G., and McGill, M. J. (1983). Introduction to Modern Information Retrieval. New York: McGraw-
Hill.
Saracevic, T., Spink, A., and Wu, M.-M. (1997). Users and intermediaries in information retrieval: What
are they talking about? In Jameson, A., Paris, C., and Tasso, C., eds., User Modeling: Proceedings
of the Sixth International Conference, UM ’97. Vienna and New York: Springer Wien New York,
pp. 43–54.
Page 20
148
Sitter, S., and Stein, A. (1992). Modeling the illocutionary aspects of information-seeking dialogues.
Information Processing & Management 28(2):165–180. See also: Modeling Information-Seeking
Dialogues: The Conversational Roles (COR) Model. RIS: Review of Information Science (online
journal), 1996, 1(1), available from http://www.inf-wiss.uni-konstanz.de/RIS/. . .
Stein, A., Gulla, J. A., Müller, A., and Thiel, U. (1997). Conversational interaction for semantic access to
multimedia information. In Maybury, M. T., ed., Intelligent Multimedia Information Retrieval. Men-
lo Park, CA: AAAI/MIT Press, pp. 399–421.
Stein, A., and Thiel, U. (1993). A conversational model of multimodal interaction in information systems.
In Proceedings of the 11th National Conference on Artificial Intelligence (AAAI ’93), Washington
D. C. Menlo Park, CA: AAAI Press/MIT Press, pp. 283–288.
Stein,A.,Gulla,J. A.,andThiel,U.(1999).User-tailoredplanningofmixedinitiativeinformation-seeking
dialogues. User Modeling and User-Adapted Interaction. Special Issue on Computational Models
for Mixed Initiative Interaction 8(1-2). To appear.
Terveen, L. G. (1995). Overview of human-computer collaboration. Knowledge-Based Systems. Special
Issue on Human-Computer Collaboration 8(2-3):67–81.
Thiel, U. (1990). Konversationale graphische Interaktion mit Informationssystemen: Ein sprechakttheo-
retischer Ansatz. Doctoral dissertation, Universität Konstanz, Germany.
Thiel, U., Everts, A., Lutes, B., and Tzeras, K. (1998). A logic-based approach to search in digital libraries.
in this volume.