Paper presentation Ready, set…? End-user testing of an AR treasure hunt learning game more

Sat, Jun 18, 10:45-11:15 Asia/Tokyo

This presentation describes testing of “KSU Treasure Hunter”, a campus guide smartphone app designed and developed by the researchers. The app presents practical information about campus services in a bilingual, pirate treasure hunt-themed Augmented Reality learning game. Testing was designed to fill a gap in research that balances concerns with both learning, including game design, in-game experience and behaviors, and pedagogy, specifically in-game pedagogical features. It involved the collection of anecdotal evidence from surveys and interviews, and descriptive evidence from observations of the behavior of testers from three distinct groups: students, instructors, and stakeholders. The survey asked testers to rate learning, engagement, and usability. Survey data showed that testers (N=21) rated learning significantly higher than both engagement (p<0.01) and usability (p<0.01), and that engagement was also rated significantly higher than usability (p<0.05). There was only significant difference between the student, instructor, and stakeholder groups in ratings of engagement. Stakeholders (n=6) rated engagement significantly higher than either students (n=10, p<0.05) or instructors (n=5, p<0.05). Interviews with testers raised issues concerning usability and engagement. Observations of testers noted indicators of engagement, frustration, and unanticipated patterns of usage. Some of the major improvements to the app as a result of testing concerned learning by modifying app design, and in-app experience and behaviors. Other changes were made with the classroom implementation of the app in a pedagogical sequence in mind. The presentation concludes with plans to investigate the pedagogical soundness of the learning game.