An end-to-end statistical architecture for human-machine dialogue systems: using computational learning to enhance adaptivity and robustness

Oliver Lemon ( Edinburgh University, School of Informatics )

Current spoken dialogue systems do not adequately treat the uncertainty inherent in human language use, ranging through multiple speech recognition hypotheses, the many possible semantic/pragmatic interpretations of each utterance, the resulting multiple possible dialogue contexts, and multiple natural language generation possibilities (though see the recent work of Jason Williams and Steve Young). To address these problems we present a processing architecture for the end-to-end statistical treatment of uncertainty in spoken dialogue systems, using reinforcement learning in Markov Decision Processes (MDPs) to optimize dialogue planning, and show how this leads to better task completion and user satisfaction. This system has been implemented, currently using a "Q-MDP" model for dialogue management decisions with probability distributions over the possible dialogue states. I'll present this model and compare it to recent work on Partially Observable MDPs (POMDPs) by Williams and Young. I'll also present evaluations of several of the adaptive dialogue policies, trained for problems such as interactive search and troubleshooting, that can be deployed using this architecture. Joint work with James Henderson, Xingkun Liu, Kallirroi Georgila, Paul Crook, Verena Rieser, Ivan Meza-Ruiz, and Srini Janarthanam. This work is funded by the EPSRC (grant EP/E019501/1).