This presentation reports on the piloting of a smartphone application designed to measure learner engagement in the language classroom. Students report their engagement via this app throughout each lesson, while teachers view live analytics of the students’ social, affective, and cognitive engagement in real-time. We attempted to measure the engagement levels of 124 learners over 15 weeks of online synchronous lessons, while also gathering their perceptions and the experiences of the participating teachers. This presentation demonstrates how engagement measures such as this can be used to innovate classroom teaching and learning, while also highlighting limitations in the application. This work was supported by a JSPS KAKENHI Grant as part of a Japanese MEXT Grants-in-Aid for Scientific Research.