Hierarchical structure

Ravenclaw (a DM for goal-oriented dialogs, based on the CMU communicator) allows the author an advanced, multi-level dialog structure description, such as:

• Room reservation task:
  • Login
    • Ask user name
    • Ask user password
  • Room selection
    • Building selection
    • Room number selection
  • Time selection
  • Finish

The Ravenclaw DM keeps a stack of dialog modules, and uses it to process the human input.

This structure encourages code reuse, for example, the login module can be used in other dialogs.

They also claim to allow dynamic dialog-task construction, where the structure is not fixed in advance but constructed on the fly, based on information selected from a backend. For instance, in a system that helps aircraft maintenance personnel throughout the execution of maintenance tasks, the structure of the dialog depends on the structure of the maintenance task and is constructed dynamically.

Topic tracking

Frameworks for chatter-bots, such as ChatScript, allow to control the conversation structure with topics. The author can create rules that capture the topic that

• topic: CHILDHOOD (child boy girl young)
• t: I had a happy childhood.
• t: But it ended too early.
• ...

If the human says one of the words in parentheses, the DM remembers that the topic is "CHILDHOOD". The chat-bot now starts telling the story under the "CHILDHOOD" title, as long as the bot is in control of the conversation (the user passively responds by saying thinks like "OK" or "right"). Yet if the user asks questions, the system can either respond directly, or use up a line of the story it was going to say anyway.

This, too, allows authors to reuse topics, and combine several independent topics to create a smarter chatter-bot.

Form filling

A common use of dialog systems is as a replacement to forms. For example, a flight-reservation agent should ask the human about his origin time and place, and destination time and place - just as if the human is filling a form with these 4 slots.

A simple solution is to use system-initiative, where the dialog system asks the user about each piece of information in turn, and the user must fill them in that exact order, like in this dialog (from a presentation by David Traum):

• Welcome to the flight confirmation system. What is your flight number?
  • United 123 on August 8 from Los Angeles
• What is your departure city?
  • I told you, Los Angeles, on August 8
• I'm sorry, I didn't understand. What is your departure city?
  • Los Angeles leaving August 8th.
• What is the day of departure?
  • You don't listen! August 8!
• Please say the day of departure?
  • August 8
• Flight United 123 confirmed to depart Los Angeles for London at 2pm on August 8.

The opposite of system-initiative is user-initiative, where the user takes lead, and the system respond to whatever the user directs.

A common compromise between the two methods is mixed-initiative, where the system starts with asking questions, but users can barge in and change the dialog direction. The system understands the user even when he speaks about details he was not asked about yet.

However, describing such a system manually, as a state-chart, is very tedious, since the human may first say the origin and then the destination, or vice versa. In each of them, the human may first say the time and then the place, or vice versa.

So, there are DMs that allow the dialog author to just say what information is required, without specifying the exact order. For example, the author may write:

• TRAVEL = {ORIGIN-PLACE, ORIGIN-TIME, DESTINATION-PLACE, DESTINATION-TIME}

The DM keeps track of which slots are already filled and which slots are still empty, and navigates the conversation in order to collect the missing information. For example, the DM may ask the human about the origin place first, but if the human adds the destination place, the DM will keep the information and not ask about it again.

Such DSs were developed in MIT, for example, Wheels (for searching used car ads), Jupiter (for retrieving weather forecasts), and more.

Simple DMs handle slot-filling binarily: either a slot is "filled", or it is "empty". More advanced DMs also keep track of the degree of grounding - how sure are we, that we really understood what the user said: whether it was "Just recently introduced", "Introduced again", "acknowledged", "repeated", etc. We can also allow the author to specify, for each piece of information, the degree to which we NEED it to be understood, e.g. sensitive information need higher degree. The DM uses this information to control the course of dialog, e.g., if the human said something about a sensitive subject, and we are not sure we understood, then the DM will issue a confirmation question. See Roque and Traum (2008).

Information state

The TrindiKit DS, developed during the Trindi project, allows authors to define a complex information state, and write general rules that process this state. Here is a sample rule:

integrateAnswer:
  preconditions: ("If the human gave a relevant answer to a question currently under discussion...")
    in(SHARED.LM, answer (usr, A))
    fst(SHARED.QUD, Q)
    relevant_answer(Q, A)
  effects: ("... then remove it from the Question Under Discussion, and add it to the shared ground")
    pop(SHARED.QUD)
    reduce(Q, A, P)
    add(SHARED.COM, P)

The DM decides, according to the input and the state, which rules are applicable, and applies them to get the new state.

This may help authors re-use general rules for dialog management rules, based on dialog theories. DSs developed with TrindiKit include: GoDiS, MIDAS, EDIS and SRI Autorate.

The information state approach was developed later in projects such as Siridus and the Dipper toolkit.

Another example of an information state based dialog manager is FLoReS. It uses a propositional information state to encode the current state and selects the next action using a Markov decision process. This dialog manager is implemented in the jmNL software.

General planning

A generalization of this approach is to let the author define the goals of the agent, and let the DM construct a plan to achieve that goal. The plan is made of operations. Each speech act is an operation. Each operation has preconditions and postconditions (effects), for example:

Inform(Speaker,Hearer,Predicate):
  Precondition: Knows(Speaker,Predicate) AND Wants(Speaker,Inform(Speaker,Hearer,Predicate))
  Effect: Knows(Hearer,Predicate)
  Body: Believes(Hearer,Wants(Speaker,Knows(Hearer,Predicate)))

The conversation can be navigated using a general planner, such as SOAR (Strengths, Opportunities, Aspirations & Results). The planner maintains the current state, and tries to construct a plan to achieve the goal, using the given operations.

A similar approach is taken in SASO-ST^[1] (a DS for multi-agent negotiation training). Using SOAR allows the incorporation of complex emotional and social models, for example: the agent can decide, based on the human actions, whether he wants to cooperate with him, avoid him, or even attack him.

A similar approach is taken in TRIPS^[2] (a DS for multi-agent collaborative problem solving). They split the dialog management into several modules:

• Reference manager - Given a word (e.g. "the woman"), decide what object in the world it refers to (e.g. "WOM1234").
• Task manager - Identify the problem-solving acts that the user tries to achieve (create new objective, extend an existing objective, etc.).
• Interpretation manager - in addition to calling the first two, also identify discourse obligations, for example: "respond to the latest question".
• Behavioral agent - decides how to accomplish the goal that the user wants. The agent employs several task-specific agents that do the actual planning.

A different kind of planning is theorem proving. A dialog can be described as an attempt to prove a theorem. The system interacts with user to supply "missing axioms" to help complete the proof (this is called "backward chaining"). This approach was implemented by:

• Grammatical Framework.^[3]
• IPSIM (Interruptible Prolog SIMulator), in the Circuit Fixit system.^[4]

The dialog manager can be connected with an expert system, to give the ability to respond with specific expertise.

Error handling

The ASR and NLU modules are usually not 100% sure they understood the user; they usually return a confidence score reflecting the quality of understanding. In such cases, the DM should decide whether to:

• Just assume that the most probable interpretation is correct, and continue the conversation (no-confirmation);
• Continue the conversation, but add some words that show understanding, such as "OK, you want to go to a restaurant. Where exactly?" (implicit-confirmation).
• Ask the user what exactly he intended to say (explicit-confirmation): "Do you mean X?" "Did you say X or Y?", etc.
• Tell the user "I didn't understand, please say this again".

Choosing "no-confirmation" may make the dialog proceed quicker, but may also introduce mistakes which will take longer to correct later.

Error handling was researched extensively by Ravenclaw, which allows the author to manually control the error handling strategy in each part of the dialog.

Initiative control

Some DSs have several modes of operation: the default mode is user-initiative, where the system just asks "what can I do for you?" and lets the user navigate the conversation. This is good for experienced users. However, if there are many misunderstandings between the user and the system, the DM may decide to switch to mixed-initiative or system-initiative - ask the user explicit questions, and accept one answer at a time.

Pedagogical decisions

Tactical decisions of a different type are done by Cordillera (a tutorial DS for teaching physics, built using TuTalk). In many points during the lesson, the DM should decide:

• Whether to Tell the pupil some fact, or try to Elicit this fact from him by asking guiding questions.
• Whether to ask the pupil to Justify his answer, or just Skip the justification and continue.

These decisions affect the overall quality of learning, which can be measured by comparing pre- and post-learning exams.

Learned tactics

Instead of letting a human expert write a complex set of decision rules, it is more common to use reinforcement learning. The dialog is represented as a Markov Decision Process (MDP) - a process where, in each state, the DM has to select an action, based on the state and the possible rewards from each action. In this setting, the dialog author should only define the reward function, for example: in tutorial dialogs, the reward is the increase in the student grade; in information seeking dialogs, the reward is positive if the human receives the information, but there is also a negative reward for each dialog step.

RL techniques are then used to learn a policy, for example, what kind of confirmation should we use in each state? etc. This policy is later used by the DM in real dialogs.

A tutorial on this subject were written by Lemon and Rieser (2009).

A different way to learn dialog policies is to try to imitate humans, using Wizard of Oz experiments, in which a human sits in a hidden room and tells the computer what to say; see for example Passonneau et al (2011).