Results 1 - 10
of
27
Predicting standardized test scores from Cognitive Tutor interactions
- In Proceedings of the 6th International Conference on Educational Data Mining (Memphis, TN
, 2013
"... Cognitive Tutors are primarily developed as instructional systems, with the goal of helping students learn. However, the systems are inherently also data collection and assessment systems. In this paper, we analyze data from over 3,000 students in a school district using Carnegie Learning’s Middle S ..."
Abstract
-
Cited by 3 (3 self)
- Add to MetaCart
(Show Context)
Cognitive Tutors are primarily developed as instructional systems, with the goal of helping students learn. However, the systems are inherently also data collection and assessment systems. In this paper, we analyze data from over 3,000 students in a school district using Carnegie Learning’s Middle School Mathematics tutors and model performance on standardized tests. Combining a standardized pretest score with interaction data from Cognitive Tutor predicts outcomes of standardized tests better than the pretest alone. In addition, a model built using only 7th grade data and a single standardized test outcome (Virginia’s SOL) generalizes to additional grade levels (6 and 8) and standardized test outcomes (NWEA’s MAP).
Constructing Variables that Support Causal Inference
, 2013
"... to many individuals. David Danks has been an ideal adviser and dissertation committee co-chair. Despite a busy schedule and many advisees, he always found time to read drafts and provide careful feedback. His understanding ear and sage advice (both personal and professional) I value immensely. Thank ..."
Abstract
-
Cited by 2 (2 self)
- Add to MetaCart
to many individuals. David Danks has been an ideal adviser and dissertation committee co-chair. Despite a busy schedule and many advisees, he always found time to read drafts and provide careful feedback. His understanding ear and sage advice (both personal and professional) I value immensely. Thank you. Richard Scheines, my other co-chair, provided the intellectual inspiration for much of this work. His long-standing interests in applied causal inference and matters of variable construction and definition in the social sciences led him to ask challenging questions, seriously improving this work. My committee members have provided guidance and inspiration in a variety of ways. Daniel Neill, some time ago, stimulated my interest in de-veloping interpretable models for predictive and causal inference that might prove useful for real-world policymakers. Partha Saha and Steven Ritter, in different settings and over the course of several years, have demonstrated that the type of questions we ask in this work are important in real educa-
The Opportunities and Limitations of Scaling Up Sensor-Free Affect Detection
"... We develop and analyze affect detectors for four affective states: confidence, excitement, frustration and interest. We utilize easy to implement self-report based “ground truth ” measurements of affect within a tutor, and model them as continuous variables that are later discretized into positive, ..."
Abstract
-
Cited by 2 (0 self)
- Add to MetaCart
(Show Context)
We develop and analyze affect detectors for four affective states: confidence, excitement, frustration and interest. We utilize easy to implement self-report based “ground truth ” measurements of affect within a tutor, and model them as continuous variables that are later discretized into positive, neutral, and negative valence classifications; this distinguishes our work from detectors which model affective states as binary. We explore the opportunities and limitations of cross validation with regard to potentially distinct sample groups.
71 Toward “Hyper-Personalized ” Cognitive Tutors Non-Cognitive Personalization in the Generalized Intelligent Framework for Tutoring
"... Abstract. We are starting to integrate Carnegie Learning’s Cognitive Tutor (CT) into the Army Research Laboratory’s Generalized Intelligent Framework for Tutoring (GIFT), with the aim of extending the tutoring systems to understand the impact of integrating non-cognitive factors into our tutoring. A ..."
Abstract
-
Cited by 1 (1 self)
- Add to MetaCart
(Show Context)
Abstract. We are starting to integrate Carnegie Learning’s Cognitive Tutor (CT) into the Army Research Laboratory’s Generalized Intelligent Framework for Tutoring (GIFT), with the aim of extending the tutoring systems to understand the impact of integrating non-cognitive factors into our tutoring. As part of this integration, we focus on ways in which non-cognitive factors can be assessed, measured, and/or “detected. ” This research provides the groundwork for an Office of the Secretary of Defense (OSD) Advanced Distributed Learning (ADL)-funded project on developing a “Hyper-Personalized ” Intelligent Tutor (HPIT). We discuss the integration of the HPIT project with GIFT, highlighting several important questions that such integration raises for the GIFT architecture and explore several possible resolutions.
Exploring the relationships between design, students’ affective states, and disengaged behaviors within an ITS
"... Abstract. Recent research has shown that differences in software design and content are associated with differences in how much students game the system and go off-task. In particular the design features of a tutor have found to predict substantial amounts of variance in gaming and off-task behavior ..."
Abstract
-
Cited by 1 (0 self)
- Add to MetaCart
(Show Context)
Abstract. Recent research has shown that differences in software design and content are associated with differences in how much students game the system and go off-task. In particular the design features of a tutor have found to predict substantial amounts of variance in gaming and off-task behavior. However, it is not yet understood how this influence takes place. In this paper we investigate the relationship between a student’s affective state, their tendency to engage in disengaged behavior, and the design aspects of the learning environments, towards understanding the role that affect plays in this process. To investigate this question, we integrate an existing taxonomy of the features of tutor lessons [3] with automated detectors of affect [8]. We find that confusion and frustration are significantly associated with lesson features which were found to be associated with disengaged behavior in past research. At the same time, we find that the affective state of engaged concentration is significantly associated with features associated with lower frequencies of disengaged behavior. This analysis suggests that simple re-designs of tutors along these lines may lead to both better affect and less disengaged behavior.
To appear. Generalizing and extending a predictive model for standardized test scores based on Cognitive Tutor interactions
- Proceedings of the 7th International Conference on Educational Data Mining
"... Recent work demonstrates that process data from intelligent tutoring systems (ITSs) can be used to predict student outcomes on high-stakes, standardized tests. Such models are important if ITSs are to be used for formative assessment and as replacements for external assessments. Recent work used var ..."
Abstract
-
Cited by 1 (1 self)
- Add to MetaCart
(Show Context)
Recent work demonstrates that process data from intelligent tutoring systems (ITSs) can be used to predict student outcomes on high-stakes, standardized tests. Such models are important if ITSs are to be used for formative assessment and as replacements for external assessments. Recent work used various measures of learning efficiency and performance from problem-level, aggregate data from Carnegie Learning’s Cognitive Tutor to predict standardized test scores on the state of Virginia’s Standards of Learning exam. We generalize this model to a different school district, state, and standardized test and examine extending the model using finer-grained data.
Reengineering the Feature Distillation Process: A Case Study in the Detection of Gaming the System
"... As education technology matures, researches debate whether data mining (EDM) or knowledge engineering (KE) paradigms are best for modeling complex learning constructs. A hybrid paradigm may capture strengths from both approaches. In particular, recent work has argued that successful data mining depe ..."
Abstract
-
Cited by 1 (1 self)
- Add to MetaCart
(Show Context)
As education technology matures, researches debate whether data mining (EDM) or knowledge engineering (KE) paradigms are best for modeling complex learning constructs. A hybrid paradigm may capture strengths from both approaches. In particular, recent work has argued that successful data mining depends on thought-ful feature engineering. In this paper, we explore the use of cogni-tive modeling (a form of knowledge engineering) to enhance the feature engineering process for detectors of gaming the system, one of the most studied complex constructs in EDM. Using this construct enables us to measure the extent to which our techniques improve performance over previous models.
Modifying Field Observation Methods on the Fly: Creative Metanarrative and Disgust in an Environmental
"... Abstract. Automated detection of constructs associated with student engage-ment, disengagement, and meta-cognition plays an increasingly prominent part of personalized online education. Often these detectors are trained with ground truth labels obtained from field observations, a method that balance ..."
Abstract
-
Cited by 1 (1 self)
- Add to MetaCart
(Show Context)
Abstract. Automated detection of constructs associated with student engage-ment, disengagement, and meta-cognition plays an increasingly prominent part of personalized online education. Often these detectors are trained with ground truth labels obtained from field observations, a method that balances collection speed with label quality. Some behaviors and affective states (e.g., boredom) are regularly modeled across learning environments, but other constructs (e.g., gaming the system) manifest in fewer systems. New environments create the possibility of entirely unexpected constructs. In this paper, we describe how a field observation protocol (already proven effective for affect and behavior de-tection in several systems) was adapted to provide the flexibility needed to document previously unidentified or rare constructs. Specifically, we describe the in-field modification of the Baker Rodrigo Ocumpaugh Monitoring Protocol (BROMP) to accommodate categories not previously established (e.g., creative metanarrative) during observations of an educational multi-user virtual envi-ronment (MUVE). We also discuss the importance of developing methods that allow researchers to conduct such explorations while still capturing standard data constructs. 1
Data mining and education
"... Abstract An emerging field of educational data mining (EDM) is building on and contributing to a wide variety of disciplines through analysis of data coming from many kinds of educational technologies. EDM researchers are addressing questions of cognition, metacognition, motivation, affect, languag ..."
Abstract
- Add to MetaCart
Abstract An emerging field of educational data mining (EDM) is building on and contributing to a wide variety of disciplines through analysis of data coming from many kinds of educational technologies. EDM researchers are addressing questions of cognition, metacognition, motivation, affect, language, social discourse, etc. using data from intelligent tutoring systems, massive open online courses, educational games and simulations, and discussion forums. The data include detailed action and timing logs of student interactions in user interfaces such as graded responses to questions or essays, steps in rich problem solving environments, games or simulations, discussion forum posts, or chat dialogs. They might also include external sensors such as eye tracking, facial expression, body movement, etc. We review how EDM has addressed the research questions that surround the psychology of learning with an emphasis on assessment, transfer of learning and model discovery, the role of affect, motivation and metacognition on learning, and analysis of language data and collaborative learning. For example, we discuss 1) how different statistical assessment methods were used in a data mining competition to improve prediction of student responses to intelligent tutor tasks, 2) how better cognitive models can be discovered from data and used to improve instruction, 3) how data-driven models of student affect can be used to focus discussion in a dialog-based tutoring system, and 4) how machine learning techniques applied to discussion data can be used to produce automated agents that support student learning as they collaborate in a chat room or discussion board.
La Mort du Chercheur: How well do students' subjective understandings of affective representations used in self- report align with one another's, and researchers'?
"... Abstract. We address empirical methods to assess the reliability and design of affective self-reports. Previous research has shown that students may have subjectively different understandings of the affective state they are reporting ..."
Abstract
- Add to MetaCart
(Show Context)
Abstract. We address empirical methods to assess the reliability and design of affective self-reports. Previous research has shown that students may have subjectively different understandings of the affective state they are reporting
