Optimizing Student Scores on the Medical College Admissions Test
Introduction to ITS
Intelligent Tutoring Systems seek to solve the crisis in personalized education by merging cognitive science and computer science. Traditionally, the architecture of an ITS is four key modules:
Student Model tracks what the student knows and doesn't know.
Knowledge Model has all the subject matter that the ITS is designed to teach.
Pedagogical Model determines how the system teaches the student.
Interface Module determines how the student interacts with the system.
While traditional ITS platforms remain confined to academic labs, MyMCAT breaks free. Our student model uses machine learning to illuminate learning paths. Our knowledge model is the MCAT itself. Our pedagogical model is personalized multimedia — videos, readings, practice problems, flashcards, tests — that evolves with each student. Our interface is engaging and gamified with a coin system that turns study into strategy. Through MyMCAT, every dedicated student can reach their peak—no wasted time, no wasted effort.
Traditional Methods vs Our Method
The MCAT preparation landscape is fragmented, dominated by contradictory advice from online forums and tutors that often conflicts with evidence-based learning principles. Despite being a field that privileges science and learning, studying for the MCAT for many is a culmination of oral tradition and folklore. The conventional approach follows a linear path:
- Initial Content Review (2 months)
- Comprehensive review sets (Kaplan/TPR)
- Practice Application
- Question banks (UWorld)
- Ongoing Review
- Spaced repetition systems (Anki)
- Final Preparation (1 month)
- AAMC Full-Length practice tests
Students invest 40 weekly hours for months, often with minimal improvement. This methodology, though widespread, promotes passive learning and fails to adapt to individual knowledge gaps. Research by Roediger & Butler (2011) demonstrates that retrieval practice enhances long-term retention, suggesting that practice should be integrated throughout rather than concentrated at the end.
The traditional sequence's fatal flaw is its rigidity: students with varying baseline knowledge follow identical paths, working blindly on perceived weaknesses. Content review without immediate application leads to rapid forgetting. Repetitive flashcard use promotes recognition over recall. Practice problems, disconnected from foundational understanding, become exercises in guesswork rather than learning.
Our approach interweaves content mastery with intelligent weakness diagnosis:
- Diagnostic assessment
- KC/CC weakness identification
- Targeted content assignment
- Knowledge profile updates through quizzing
- Continuous reassessment
We map the AAMC's content outline (1A-10A) into Knowledge Categories (KCs) and more granular Concept Categories (CCs). For instance, category 1A translates to the KC "Amino Acids" with CCs including "Amino Acids," "Proteins," and "Enzymes." This granular categorization spans seven subjects, encompassing nearly 100 concepts that form our knowledge profiles. Furthermore, we integrate with major canonized resources on the MCAT. We incorporate UWorld practice problems and AAMC, if the student has it, though we aren't affiliated with either of those organizations. Every single question you solve, on MyMCAT or UWorld/AAMC, is categorized granularly to build student knowledge profiles.
Every interaction—whether through MyMCAT, UWorld, or AAMC materials—refines our understanding of student competency. Our system delivers content through diverse modalities: video lectures (Khan Academy, AK Lectures), readings (LibreText, OpenStax), and our gamified Anki Clinic, which transforms traditional flashcards into an engaging mix of multiple-choice and patient-based scenarios.
Gamification
Gamification—the use of game design elements in non-game contexts—has been successfully employed across various sectors, including education, healthcare, marketing, workplace productivity, and environmental initiatives. The approach has shown a lot of promise in enhancing user engagement and motivation. However, our take on gamifying educational software pushes these boundaries further than what's typically seen
We've developed a coin-based reward system based on a fixed ratio schedule, a concept borrowed from behavioral psychology and popularized in game design (Hamari et al., 2014). Students earn coins for completing daily CARS practice or sticking to their assigned tasks, providing a consistent positive reinforcement loop. The flip side of the system introduces a mild penalty for inconsistency: students lose coins if they skip days, creating a response cost that subtly discourages undesirable behaviors.
The coin system also supports our monetization strategy. Students can earn coins through studying or opt to purchase additional coins, tapping into the same microtransaction mechanics common in the gaming industry. While this may seem unconventional for an educational product, the underlying idea is to increase user investment and make the study experience more dynamic and interactive.
One of our most "ballsy" innovations is the comprehensive transformation of traditional spaced repetition tools like Anki. While retaining the core principles of spaced repetition, we've reimagined the experience by introducing a clinic-themed environment where students engage with flashcard questions to earn or lose coins based on their performance. This approach not only gamifies the learning process but also leverages the testing effect, enhancing long-term retention through active recall. By integrating these elements, we aim to create a more immersive and motivating study experience that encourages consistent engagement and reinforces knowledge acquisition.
Preliminary Results
Over the past two years, we have implemented this system with 25 students from various institutions, utilizing a Google Sheet for tracking which you can access and use for free here. The results have been promising, with an average MCAT score increase of 14.3 points, an average MCAT score of 513, and some students achieved improvements exceeding 30 points. Notably, every participant experienced a score enhancement using this methodology. Despite these encouraging outcomes, certain challenges have emerged:
- Underperformance on the Actual MCAT: Some students did not perform as well on the official exam as anticipated.
- Declining Motivation: A subset of students exhibited reduced engagement over time.
To address these issues, we have enhanced our weakness identification algorithms to provide more precise assessments. Additionally, we have developed an integrated email and coin-based reward system to incentivize sustained participation and motivation. Over the next few months, as we gear up for expansion into medical education, we will be testing and iterating our platform to create the greatest study resource, the first ITS, and the only MCAT software to enable students to crush the test and deliver to their fullest potential.
References
Alkhatlan, A., & Kalita, J. (2019). Intelligent tutoring systems: A comprehensive historical survey with recent developments. International Journal of Computer Applications, 181(43), 1-20. https://doi.org/10.5120/ijca2019918451
Chen, W., & Corridon, P. (2020). The predictive value of full-length practice exams for the new MCAT exam for premedical students. Journal of Medical Education and Curricular Development, 7. https://doi.org/10.1177/2382120520981979
Roediger, H. L., & Butler, A. C. (2011). The critical role of retrieval practice in long-term retention. Trends in Cognitive Sciences, 15(1), 20-27. https://doi.org/10.1016/j.tics.2010.09.003
Salden, R., Aleven, V., Renkl, A., & Schwonke, R. (2009). Worked examples and tutored problem solving: Redundant or synergistic forms of support? Topics in Cognitive Science, 1(1), 203-213. https://doi.org/10.1111/j.1756-8765.2008.01011.x