We present the results of 15 student case studies, pairing student conversation data with their AP performance. These cases explore how patterns in student questioning relate to performance on the practice assessments, and also how the conversations reveal deeper diagnostic information about a student’s knowledge state and affect.

‍‍

Purpose‍

We conducted an in-depth qualitative analysis of student-AI conversations paired with quantitative learning outcome data to understand how conversation patterns can reveal diagnostic information about students' knowledge states, learning trajectories, and affect.

‍

‍Methodology‍

From a sample of 5,000 students who engaged with both Flexi and adaptive practice (AP) during the Aug 2024-June 2025 school year, we selected 15 cases for intensive analysis based on interaction depth and data completeness. Additional criteria for selection are detailed in Appendix B. We include students who showed significant academic growth on their practice assessments over the school year and engaged in thoughtful, back-and-forth dialogue with Flexi that include the types of questions associated with positive outcomes (e.g., Hakkarainen, 2003; Zhang et al., 2007), including strategic questioning, trust-building, asking follow-up questions (topic coherence), self-directed learning, and productive struggle. We also include students who showed flat or declining practice scores and used the AI tutor superficially, showing patterns such as surface-level engagement, learned helplessness, and academic avoidance. Finally, we include students who showed more surprising or complex patterns that reveal less intuitive watch points and opportunities for accelerating learning.