Title: User Query Behaviour in Different Task Types in a Spoken Language vs. Textual Interface: A Wizard of Oz experiment
Authors: Xiaojun (Jenny) Yuan and Ning Sa
College of Engineering and Applied Sciences, University at Albany, State University of New York, Albany, NY 12222
Introduction. This paper examined how task type affected user query behaviour in a spoken language interface in comparison to a textual input interface.
Method. Pre-search questionnaire, post-search questionnaire, post-system questionnaire and interviews, and a user-centred wizard of Oz experiment were conducted in the University at Albany, State University of New York to compare the user query behaviour difference between these two system interfaces. Forty-eight participants joined the study. Each participant performed 12 tasks of three types, including factual, interpretive, and exploratory tasks.
Analysis. Quantitative and qualitative analyses were used to analyse the collected data. Specifically, both Chi-square test and t test were employed in the quantitative data analysis.
Results. Results indicated that participants initiated significantly fewer queries for both factual and interpretive tasks, and input significantly longer queries for both interpretive and exploratory tasks in the spoken language interface than in the textual interface. Participants employed significantly more stop words for both interpretive and exploratory tasks and used significantly more indicative words in factual tasks in spoken language interface than that in the textual input interface.
Conclusion. We conclude that task type has a significant effect on user query behaviour. System design implications of the spoken language interfaces were discussed and recommendations were provided.
Keywords. User behaviour, spoken language interface, evaluation