Gastvorträge 2009

User judgments of Spoken Dialog Systems provide evaluators of such systems with a valid measure of the system’s overall quality. Models for the automatic prediction of user judgments have been built, after the introduction of the PARADISE framework (Walker 1997). These take as input interaction parameters describing task success and dialog costs (efficiency and quality related parameters). Main applications are the comparison of systems, the analysis of parameters affecting quality, and the adoption of dialog strategy. However, a common model which applies to different systems and users has not been found so far. In the talk, a new approach to such models is presented, using Hidden Markov Models. In this approach, the user’s opinion is regarded as a continuous process evolving over time. For data collection, an experiment was conducted where user judgments were collected after each dialog turn. In the talk, the experimental method and the modelling method are described, and results are presented.

Felix Hartard studies computer science (Informatik) and presents results from his diploma thesis supervised at T-Labs.

Georg Essl, georg.essl(AT)telekom.de [2]