Leveraging LLM Tutoring Systems for Non-Native English Speakers in Introductory CS Courses

Non-native English speakers appreciated being able to mix technical English vocabulary with their native languages when asking for help

Molina, I. V., Montalvo, A., Ochoa, B., Denny, P., & Porter, L. (2024). Leveraging LLM Tutoring Systems for Non-Native English Speakers in Introductory CS Courses (No. arXiv:2411.02725). arXiv. https://doi.org/10.48550/arXiv.2411.02725

The majority of non-native English speakers (NNES) in this study - over 75% - report not knowing computer science vocabulary in their first language. Being able to ask for help in mixed language sentences (e.g., multi-headed attention は何ですか?), which they can easily do with LLMs but not with human tutors, can help them overcome a critical barrier to learning. NNES have a strong preference for getting support from LLMs.

In this study, we deployed an LLM tutor in an accelerated introductory computing course to evaluate its efficacy for NNES students. Our findings show that NNES students signed up for the LLM tutor at a similar rate as NES but used the system slightly less. Notably, NNES students asked significantly more questions in languages other than English, often mixing in English programming keywords…. These results suggest that LLM tutors can be a valuable resource for NNES students in computer science education, providing tailored support that enhances their learning experience and overcomes language barriers.