Results 1 - 10
of
21
Automated feedback generation for introductory programming assignments
, 2013
"... We present a new method for automatically providing feedback for introductory programming problems. In order to use this method, we need a reference implementation of the assignment, and an error model consisting of potential corrections to errors that students might make. Using this information, th ..."
Abstract
-
Cited by 25 (7 self)
- Add to MetaCart
(Show Context)
We present a new method for automatically providing feedback for introductory programming problems. In order to use this method, we need a reference implementation of the assignment, and an error model consisting of potential corrections to errors that students might make. Using this information, the system automatically derives minimal corrections to student’s incorrect solutions, providing them with a measure of exactly how incorrect a given solution was, as well as feedback about what they did wrong. We introduce a simple language for describing error models in terms of correction rules, and formally define a rule-directed translation strategy that reduces the problem of finding minimal corrections in an incorrect program to the problem of synthesizing a correct program from a sketch. We have evaluated our system on thousands of real student attempts obtained from the Introduction to
Synthesis from examples: Interaction models and algorithms
- 14th International Symposium on Symbolic and Numeric Algorithms for Scientific Computing
, 2012
"... Abstract—Examples are often a natural way to specify various computational artifacts such as programs, queries, and sequences. Synthesizing such artifacts from example based specifications has various applications in the domains of enduser programming and intelligent tutoring systems. Synthesis from ..."
Abstract
-
Cited by 21 (12 self)
- Add to MetaCart
(Show Context)
Abstract—Examples are often a natural way to specify various computational artifacts such as programs, queries, and sequences. Synthesizing such artifacts from example based specifications has various applications in the domains of enduser programming and intelligent tutoring systems. Synthesis from examples involves addressing two key technical challenges: (i) design of a user interaction model to deal with the inherent ambiguity in the example based specification. (ii) design of an efficient search algorithm- these algorithms have been based on paradigms from various communities including use of SAT/SMT solvers (formal methods community), version space algebras (machine learning community), and A*-style goal-directed heuristics (AI community). This paper describes some effective user interaction models and algorithmic methodologies for synthesis from examples while discussing synthesizers for a variety of artifacts ranging from tricky bitvector algorithms, spreadsheet macros for automating repetitive data manipulation tasks, ruler/compass based geometry constructions, algebraic identities, and predictive intellisense for repetitive drawings and mathematical terms.
Synthesizing number transformations from input-output examples.
, 2012
"... Abstract. Numbers are one of the most widely used data type in programming languages. Number transformations like formatting and rounding present a challenge even for experienced programmers as they find it difficult to remember different number format strings supported by different programming lan ..."
Abstract
-
Cited by 19 (10 self)
- Add to MetaCart
(Show Context)
Abstract. Numbers are one of the most widely used data type in programming languages. Number transformations like formatting and rounding present a challenge even for experienced programmers as they find it difficult to remember different number format strings supported by different programming languages. These transformations present an even bigger challenge for end-users of spreadsheet systems like Microsoft Excel where providing such custom format strings is beyond their expertise. In our extensive case study of help forums of many programming languages and Excel, we found that both programmers and end-users struggle with these number transformations, but are able to easily express their intent using input-output examples. In this paper, we present a framework that can learn such number transformations from very few input-output examples. We first describe an expressive number transformation language that can model these transformations, and then present an inductive synthesis algorithm that can learn all expressions in this language that are consistent with a given set of examples. We also present a ranking scheme of these expressions that enables efficient learning of the desired transformation from very few examples. By combining our inductive synthesis algorithm for number transformations with an inductive synthesis algorithm for syntactic string transformations, we are able to obtain an inductive synthesis algorithm for manipulating data types that have numbers as a constituent sub-type such as date, unit, and time. We have implemented our algorithms as an Excel add-in and have evaluated it successfully over several benchmarks obtained from the help forums and the Excel product team.
A trace-based framework for analyzing and synthesizing educational progressions
- In Proc. CHI
, 2013
"... A key challenge in teaching a procedural skill is finding an effective progression of example problems that the learner can solve in order to internalize the procedure. In many learning domains, generation of such problems is typically done by hand and there are few tools to help automate this proce ..."
Abstract
-
Cited by 17 (9 self)
- Add to MetaCart
(Show Context)
A key challenge in teaching a procedural skill is finding an effective progression of example problems that the learner can solve in order to internalize the procedure. In many learning domains, generation of such problems is typically done by hand and there are few tools to help automate this process. We reduce this effort by borrowing ideas from test input generation in software engineering. We show how we can use execution traces as a framework for abstracting the characteristics of a given procedure and defining a partial ordering that reflects the relative difficulty of two traces. We also show how we can use this framework to analyze the completeness of expert-designed progressions and fill in holes. Furthermore, we demonstrate how our framework can automatically synthesize new problems by generating large sets of problems for elementary and middle school mathematics and synthesizing hundreds of levels for a popular algebra-learning game. We present the results of a user study with this game confirming that our partial ordering can predict user evaluation of procedural difficulty better than baseline methods.
New Potentials for Data-Driven Intelligent Tutoring System Development and Optimization Short title: Data-Driven Improvement of Intelligent Tutors
"... learning for student modeling Increasing widespread use of educational technologies is producing vast amounts of data. Such data can be used to help advance our understanding of student learning and enable more intelligent, interactive, engaging, and effective education. In this paper, we discuss th ..."
Abstract
-
Cited by 10 (0 self)
- Add to MetaCart
learning for student modeling Increasing widespread use of educational technologies is producing vast amounts of data. Such data can be used to help advance our understanding of student learning and enable more intelligent, interactive, engaging, and effective education. In this paper, we discuss the status and prospects of this new and powerful opportunity for data-driven development and optimization of educational technologies, focusing on Intelligent Tutoring Systems. We provide examples of use of a variety of techniques to develop or optimize the select, evaluate, suggest, and update functions of intelligent tutors, including probabilistic grammar learning, rule induction, Markov decision process, classification, and integrations of symbolic search and statistical inference. 1.
Synthesis from Examples
"... Examples are often a natural way to specify computational structures such as programs, queries, and sequences. Synthesizing such structures from example based specification has applications in automating end-user programming and in building intelligent tutoring systems. Synthesis from examples invol ..."
Abstract
-
Cited by 10 (6 self)
- Add to MetaCart
(Show Context)
Examples are often a natural way to specify computational structures such as programs, queries, and sequences. Synthesizing such structures from example based specification has applications in automating end-user programming and in building intelligent tutoring systems. Synthesis from examples involves addressing two key technical challenges: (i) design of an efficient search algorithm – these algorithms have been based on various paradigms including versionspace algebras, SAT/SMT solvers, numerical methods, and even exhaustive search, (ii) design of a user interaction model to deal with the inherent ambiguity in the example based specification. This paper illustrates various algorithmic techniques and user interaction models by describing inductive synthesizers for varied applications including synthesis of tricky bitvector algorithms, spreadsheet macros for automating repetitive data manipulation tasks, ruler/compass based geometry constructions, new algebra problems, sequences for mathematical intellisense, and grading of programming problems. 1.
From relational verification to simd loop synthesis
- In PPoPP. ACM
, 2013
"... Existing pattern-based compiler technology is unable to effectively exploit the full potential of SIMD architectures. We present a new program synthesis based technique for auto-vectorizing perfor-mance critical innermost loops. Our synthesis technique is appli-cable to a wide range of loops, consis ..."
Abstract
-
Cited by 9 (2 self)
- Add to MetaCart
(Show Context)
Existing pattern-based compiler technology is unable to effectively exploit the full potential of SIMD architectures. We present a new program synthesis based technique for auto-vectorizing perfor-mance critical innermost loops. Our synthesis technique is appli-cable to a wide range of loops, consistently produces performant SIMD code, and generates correctness proofs for the output code. The synthesis technique, which leverages existing work on rela-tional verification methods, is a novel combination of deductive loop restructuring, synthesis condition generation and a new induc-tive synthesis algorithm for producing loop-free code fragments. The inductive synthesis algorithm wraps an optimized depth-first exploration of code sequences inside a CEGIS loop. Our technique is able to quickly produce SIMD implementations (up to 9 instruc-tions in 0.12 seconds) for a wide range of fundamental looping structures. The resulting SIMD implementations outperform the original loops by 2.0×-3.7×.
Educational Software Engineering: Where Software Engineering, Education, and Gaming Meet
"... Abstract—We define and advocate the subfield of educational software engineering (i.e., software engineering for education), which develops software engineering technologies (e.g., software testing and analysis, software analytics) for general educational tasks, going beyond educational tasks for so ..."
Abstract
-
Cited by 8 (7 self)
- Add to MetaCart
(Show Context)
Abstract—We define and advocate the subfield of educational software engineering (i.e., software engineering for education), which develops software engineering technologies (e.g., software testing and analysis, software analytics) for general educational tasks, going beyond educational tasks for software engineering. In this subfield, gaming technologies often play an important role together with software engineering technologies. We expect that researchers in educational software engineering would be among key players in the education domain and in the coming age of Massive Open Online Courses (MOOCs). Educational software engineering can and will contribute significant solutions to address various critical challenges in education especially MOOCs such as automatic grading, intelligent tutoring, problem generation, and plagiarism detection. In this position paper, we define educational software engineering and illustrate Pex for Fun (in short as Pex4Fun), one of our recent examples
Programming by demonstration framework applied to procedural math problems
, 2013
"... K-12 mathematics includes many procedures to be learned, such as addition and subtraction, and there are many “buggy ” or incor-rect procedures that students demonstrate during this learning pro-cess. Learning such procedures (both correct and incorrect) from demonstration traces has various applica ..."
Abstract
-
Cited by 2 (2 self)
- Add to MetaCart
(Show Context)
K-12 mathematics includes many procedures to be learned, such as addition and subtraction, and there are many “buggy ” or incor-rect procedures that students demonstrate during this learning pro-cess. Learning such procedures (both correct and incorrect) from demonstration traces has various applications in computer-aided education. We formalize mathematical procedures as spreadsheet programs, involving loops and conditionals over a given set of base operators, and present a novel algorithm for synthesizing such pro-cedures from demonstrations. Our algorithm is based on dynamic programming and leverages ideas from version-space algebras and template-based program synthesis. Our implementation efficiently synthesized programs to solve 20 common math procedures and re-produce 28 different kinds of bugs that were demonstrated by real students across 9 procedures. Our implementation significantly out-performs SKETCH, a state of the art program synthesizer, on these tasks. We also demonstrate the applicability of our generic program synthesis technology to spreadsheet table transformations, an im-portant domain in end-user programming. 1.