Results 1 
9 of
9
Sparse factor analysis for learning and content analytics
 J. OF MACHINE LEARNING RESEARCH
, 2014
"... We develop a new model and algorithms for machine learningbased learning analytics, which estimate a learner’s knowledge of the concepts underlying a domain, and content analytics, which estimate the relationships among a collection of questions and those concepts. Our model represents the probabil ..."
Abstract

Cited by 16 (10 self)
 Add to MetaCart
We develop a new model and algorithms for machine learningbased learning analytics, which estimate a learner’s knowledge of the concepts underlying a domain, and content analytics, which estimate the relationships among a collection of questions and those concepts. Our model represents the probability that a learner provides the correct response to a question in terms of three factors: their understanding of a set of underlying concepts, the concepts involved in each question, and each question’s intrinsic difficulty. We estimate these factors given the graded responses to a collection of questions. The underlying estimation problem is illposed in general, especially when only a subset of the questions are answered. The key observation that enables a wellposed solution is the fact that typical educational domains of interest involve only a small number of key concepts. Leveraging this observation, we develop both a biconvex maximumlikelihoodbased solution and a Bayesian solution to the resulting SPARse Factor Analysis (SPARFA) problem. We also incorporate userdefined tags on questions to facilitate the interpretability of the estimated factors. Experiments with synthetic and realworld data demonstrate the efficacy of our approach. Finally, we make a connection between SPARFA and noisy, binaryvalued (1bit) dictionary learning that is of independent interest.
Comparison of methods to trace multiple subskills: Is LRDBN best?
"... A longstanding challenge for knowledge tracing is how to update estimates of multiple subskills that underlie a single observable step. We characterize approaches to this problem by how they model knowledge tracing, fit its parameters, predict performance, and update subskill estimates. Previous me ..."
Abstract

Cited by 5 (3 self)
 Add to MetaCart
(Show Context)
A longstanding challenge for knowledge tracing is how to update estimates of multiple subskills that underlie a single observable step. We characterize approaches to this problem by how they model knowledge tracing, fit its parameters, predict performance, and update subskill estimates. Previous methods allocated blame or credit among subskills in various ways based on strong assumptions about their relation to observed performance. LRDBN relaxes these assumptions by using logistic regression in a Dynamic Bayes Net. LRDBN significantly outperforms previous methods on data sets from reading and algebra tutors in terms of predictive accuracy on unseen data, cutting the error rate by half. An ablation experiment shows that using logistic regression to predict performance helps, but that using it to jointly estimate subskills explains most of this dramatic improvement. An implementation of LRDBN is now publicly available in the BNTSM student modeling toolkit.
Multiarmed bandits for intelligent tutoring systems. (submitted
, 2015
"... We present an approach to Intelligent Tutoring Systems which adaptively personalizes sequences of learning activities to maximize skills acquired by students, taking into account the limited time and motivational resources. At a given point in time, the system proposes to the students the activity ..."
Abstract

Cited by 4 (4 self)
 Add to MetaCart
We present an approach to Intelligent Tutoring Systems which adaptively personalizes sequences of learning activities to maximize skills acquired by students, taking into account the limited time and motivational resources. At a given point in time, the system proposes to the students the activity which makes them progress faster. We introduce two algorithms that rely on the empirical estimation of the learning progress, RiARiT that uses information about the difficulty of each exercise and ZPDES that uses much less knowledge about the problem. The system is based on the combination of three approaches. First, it leverages recent models of intrinsically motivated learning by transposing them to active teaching, relying on empirical estimation of learning progress provided by specific activities to particular students. Second, it uses stateoftheart MultiArm Bandit (MAB) techniques to efficiently manage the exploration/exploitation challenge of this optimization process. Third, it leverages expert knowledge to constrain and bootstrap initial exploration of the MAB, while requiring only coarse guidance information of the expert and allowing the system to deal with didactic gaps in its knowledge. The system is evaluated in a scenario where 78 year old schoolchildren learn how to decompose numbers while manipulating money. Systematic experiments are presented with simulated students, followed by results of a user study across a population of 400 school children.
Timevarying learning and content analytics via sparse factor analysis
 In Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining
, 2014
"... We propose SPARFATrace, a new machine learningbased framework for timevarying learning and content analytics for educational applications. We develop a novel message passingbased, blind, approximate Kalman filter for sparse factor analysis (SPARFA) that jointly traces learner concept knowledge ..."
Abstract

Cited by 3 (0 self)
 Add to MetaCart
We propose SPARFATrace, a new machine learningbased framework for timevarying learning and content analytics for educational applications. We develop a novel message passingbased, blind, approximate Kalman filter for sparse factor analysis (SPARFA) that jointly traces learner concept knowledge over time, analyzes learner concept knowledge state transitions (induced by interacting with learning resources, such as textbook sections, lecture videos, etc., or the forgetting effect), and estimates the content organization and difficulty of the questions in assessments. These quantities are estimated solely from binaryvalued (correct/incorrect) graded learner response data and the specific actions each learner performs (e.g., answering a question or studying a learning resource) at each time instant. Experimental results on two online course datasets demonstrate that SPARFATrace is capable of tracing each learner’s concept knowledge evolution over time, analyzing the quality and content organization of learning resources, and estimating the question–concept associations and the question difficulties. Moreover, we show that SPARFATrace achieves comparable or better performance in predicting unobserved learner responses compared to existing collaborative filtering and knowledge tracing methods.
Different parameters  same predictions: An analysis of learning curves
, 2014
"... Using data from student use of educational technologies to evaluate and improve cognitive models of learners is now a common approach in EDM. Such naturally occurring data poses modeling challenges when nonrandom factors drive what data is collected. Prior work began to explore the potential parame ..."
Abstract

Cited by 2 (0 self)
 Add to MetaCart
Using data from student use of educational technologies to evaluate and improve cognitive models of learners is now a common approach in EDM. Such naturally occurring data poses modeling challenges when nonrandom factors drive what data is collected. Prior work began to explore the potential parameter estimate biases that may result from data from tutoring systems that employ a mastery learning mechanism whereby poorer students get assigned tasks that better students do not. We extend that work both by exploring a wider set of modeling techniques and by using a data set with additional observations of longerterm retention that provide a check on whether judged mastery is maintained. The data set at hand contains math learning data from children with and without developmental dyscalculia. We test variations on logistic regression, including the Additive Factors Model and others explicitly designed to adjust for masterybased data, as well as Bayesian Knowledge Tracing (BKT). We find these models produce similar prediction accuracy (though BKT is worse), but have different parameter estimation patterns. We discuss implications for use and interpretation of these different models.
Automatic Discovery of Cognitive Skills to Improve the Prediction of Student Learning
"... To master a discipline such as algebra or physics, students must acquire a set of cognitive skills. Traditionally, educators and domain experts use intuition to determine what these skills are and then select practice exercises to hone a particular skill. We propose a technique that uses student pe ..."
Abstract

Cited by 2 (1 self)
 Add to MetaCart
(Show Context)
To master a discipline such as algebra or physics, students must acquire a set of cognitive skills. Traditionally, educators and domain experts use intuition to determine what these skills are and then select practice exercises to hone a particular skill. We propose a technique that uses student performance data to automatically discover the skills needed in a discipline. The technique assigns a latent skill to each exercise such that a student’s expected accuracy on a sequence of sameskill exercises improves monotonically with practice. Rather than discarding the skills identified by experts, our technique incorporates a nonparametric prior over the exerciseskill assignments that is based on the expertprovided skills and a weighted Chinese restaurant process. We test our technique on datasets from five different intelligent tutoring systems designed for students ranging in age from middle school through college. We obtain two surprising results. First, in three of the five datasets, the skills inferred by our technique support significantly improved predictions of student performance over the expertprovided skills. Second, the expertprovided skills have little value: our technique predicts student performance nearly as well when it ignores the domain expertise as when it attempts to leverage it. We discuss explanations for these surprising results and also the relationship of our skilldiscovery technique to alternative approaches. 1
Computational Education using Latent Structured Prediction
, 2014
"... Computational education offers an important addon to conventional teaching. To provide optimal learning conditions, accurate representation of students’ current skills and adaptation to newly acquired knowledge are essential. To obtain sufficient representational power we investigate suitability of ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
Computational education offers an important addon to conventional teaching. To provide optimal learning conditions, accurate representation of students’ current skills and adaptation to newly acquired knowledge are essential. To obtain sufficient representational power we investigate suitability of general graphical models and discuss adaptation by learning parameters of a loglinear distribution. For interpretability we propose to constrain the parameter space apriori by leveraging domain knowledge. We show the benefits of general graphical models and of regularizing the parameter space by evaluation of our models on data collected from a computational education software for children having difficulties in learning mathematics.
A Unified 5Dimensional Framework for Student Models
"... This paper defines 5 key dimensions of student models: whether and how they model time, skill, noise, latent traits, and multiple influences on student performance. We use this framework to characterize and compare previous student models, analyze their relative accuracy, and propose novel models s ..."
Abstract
 Add to MetaCart
(Show Context)
This paper defines 5 key dimensions of student models: whether and how they model time, skill, noise, latent traits, and multiple influences on student performance. We use this framework to characterize and compare previous student models, analyze their relative accuracy, and propose novel models suggested by gaps in the multidimensional space. To illustrate the generative power of this framework, we derive one such model, called HOTDINA (Higher Order Temporal, Deterministic Input, NoisyAnd) and evaluate it on synthetic and real data. We show it predicts student performance better than previous methods, when, and why.