Language learning apps have gained increasing popularity recently for their effectiveness in supporting second language (L2) development, but the issue of low learner engagement persists in both formal classroom (e.g., Hanson & Brown, 2020) and informal self-study (e.g., Loewen et al., 2020) contexts. Additionally, despite over two decades of progress in mobile-assisted language learning (MALL), the integration of MALL into L2 classes is still lacking (Burston, 2014; Chwo et al., 2018). This study aims to provide an example of integrating app-based learning as a course assignment, and to explore how goal setting with feedback as an updated teaching strategy (Mercer & Dörnyei, 2020) affects learner engagement in app-based L2 vocabulary self-study. Sixty-three Japanese learners in college-level English as a Foreign Language (EFL) classes were randomly assigned to treatment (n = 32) or control (n = 31) groups, and studied TOEIC words with the Memrise app as an after-class assignment for 8 weeks. Whereas the control group had their weekly goals of studying 35 words set by the teacher, the treatment group set and checked their weekly goals of studying at least 30 words and provided reasons if not meeting their goals. The number of words studied weekly was recorded for each participant to index engagement. Apart from goal-setting-and-checking activities, we also explored leaderboards and Unfinished Lists (lists of students who did not finish the Memrise assignment) as feedback tools by collecting learner ratings and comments. EFL listening and reading proficiency before and after using Memrise was assessed with TOEIC tests. Results showed the treatment group studied significantly more words than the control group, although TOEIC performance of both groups did not change significantly. Learner perception data supported the value of adopting these three pedagogical interventions in class. Practical pedagogical guidance on adapting these activities into L2 classrooms will be discussed.