Table of Links
2. Background
2.1 Effective Tutoring Practice
2.2 Feedback for Tutor Training
2.3 Sequence Labeling for Feedback Generation
2.4 Large Language Models in Education
3. Method
3.1 Dataset and 3.2 Sequence Labeling
3.3 GPT Facilitated Sequence Labeling
4. Results
6. Limitation and Future Works
APPENDIX
B. Input for Fine-Tunning GPT-3.5
C. Scatter Matric of the Correlation on the Outcome-based Praise
D. Detailed Results of Fine-Tuned GPT-3.5 Model's Performance
2.3 Sequence Labeling for Feedback Generation
Sequence labeling, a fundamental task in natural language processing (NLP), plays a pivotal role in identifying and categorizing key segments of text according to predefined labels [26]. To elucidate the mechanism of sequence labeling, we consider Named Entity Recognition (NER) as a representative subtask, which is closed to the task in our study. NER seeks to automatically detect and classify named entities—words or phrases with specific attributes—into categories such as person, organization, and location [26, 36]. For instance, in the sentence “John said that Pittsburgh is wonderful in the winter,” the terms “John”, “Pittsburgh”, and “winter” would be labeled as Person, Location, and Time, respectively, showcasing how NER distinguishes and categorizes entities within a textual context.
Our study extends the application of sequence labeling to identify and highlight components related to different types of praise within tutor responses. This process involves discerning specific words or phrases that signify the kind of praise being used, thereby offering tutors insight into their feedback practices. For example, “You are doing great.”, the phrase “doing great” in this context is identified as an outcome-based praise. Leveraging sequence labeling allows our AI model to spotlight such instances of praise, enabling the provision of nuanced, explanatory feedback. An example of such feedback might be, “Saying “doing great” is praising the student for the outcome. You should focus on praising the students for their effort and process towards learning. Do you want to try responding again?” This approach facilitates the generation of targeted, template-based feedback for tutors.
Notably, while previous research has explored sequence labeling techniques for similar purposes [40], the accuracy of their proposed models in precisely identifying and categorizing feedback elements remains a challenge. This limitation underscores the need for leveraging more advanced models to provide accurate, informative feedback to tutors.
This paper is available on arxiv under CC BY 4.0 DEED license.
Authors:
(1) Jionghao Lin, Carnegie Mellon University ([email protected]);
(2) Eason Chen, Carnegie Mellon University ([email protected]);
(3) Zeifei Han, University of Toronto ([email protected]);
(4) Ashish Gurung, Carnegie Mellon University ([email protected]);
(5) Danielle R. Thomas, Carnegie Mellon University ([email protected]);
(6) Wei Tan, Monash University ([email protected]);
(7) Ngoc Dang Nguyen, Monash University ([email protected]);
(8) Kenneth R. Koedinger, Carnegie Mellon University ([email protected]).