Introduction During the 1997-1998 school year, answering programming exercises in the computer was introduced in the curricula of several Computer Science undergraduate courses in our department. The courses in question involved such distinct programming languages as the assembly language for the 68000 Motorola microprocessor, Prolog programming, and Graphical Users Interfaces (GUI) construction using tcl-tk [Ous94]. For this purpose a grading tool was developed whose goal was to evaluate the students programming skills better than just with written exercises. The motivation is that outside academia, program development is mostly seen as a cycle that includes program design, coding and debugging, possibly leading to recoding (or even redesign) until a program meeting the requirements is obtained. Traditional grading methods test only the design and coding skills. We wanted to test whether the students had acquired the skills required to complete the full programming cycle. A

The traditional method of automatically assessing programming exercises in Computer Science uses a black-box approach where a set of test data is inputed to both students and teachers programs and their outputs compared. This approach is useful for grading but inadequate for detecting and correcting students errors. In this paper we present several cases where we were able to develop matching algorithms to compare answers with solutions and pinpoint dierences between them. In some cases the matching is based on the actual structure of answers and solutions. In other cases we use execution side-eects to collect a structure that can be compared using a matching algorithm. This approach is currently being used in Ganesh - a web environment for learning Computer Science. 1 Introduction The web is the natural environment for learning Computer Science (CS) nowadays. As computers are both the subject and the medium, students are naturally inclined to explore all the potential of this form...

This article appeared in a journal published by Elsevier. The attached copy is furnished to the author for internal non-commercial research and education use, including for instruction at the authors institution and sharing with colleagues. Other uses, including reproduction and distribution, or selling or licensing copies, or posting to personal, institutional or third party websites are prohibited. In most cases authors are permitted to post their version of the article (e.g. in Word or Tex form) to their personal website or institutional repository. Authors requiring further information regarding Elsevier’s archiving and manuscript policies are encouraged to visit: http://www.elsevier.com/copyright Author's personal copy

Abstract: The ability to program is one of the core tools used by computer scientists, and programming proficiency is a recommended requirement for ABET accreditation. In our experience, students learn programming skills best by writing many programs, ranging from simple to complex. Overworked teachers can be dismayed by the prospect of grading still more programs per student as well as teaching introductory classes with large enrollments. The automatic grading approach offers substantial advantages and opportunities, but also some challenges. We present WebBot, a web-based automatic grader for computer programming assignments. This program is an expansion of GradeBot, an automatic program grader used for several years. This newest version of GradeBot introduces a web-based interface. GradeBot evaluates student programs written in any of several languages, including C, C++, Java, Perl, Python, Tcl, and MIPS assembler. Guidance for similar projects is provided through a discussion of the development and use of GradeBot and WebBot. Keywords: automated grading, grading, courseware, CS1, CS2, web-based This paper is part of the group that was selected for inclusion in the journal based on preliminary ratings in the top 30% of papers submitted, and a second review placing it in the top 15% by persons unconnected with the conference or the journal, and whose names are withheld to preserve their anonymity. EDSIG activities include the publication of ISEDJ, the organization and execution of the annual ISECON conference held each fall, the publication of the Journal of Information Systems Education (JISE), and the designation and honoring of an IS Educator of the Year. • The Foundation for Information Technology Education has been the key sponsor of ISECON over the years. • The Association for Information Technology Professionals (AITP) provides the corporate umbrella under which EDSIG operates. c Copyright 2006 EDSIG. In the spirit of academic freedom, permission is granted to make and distribute unlimited copies of this issue in its PDF or printed form, so long as the entire document is presented, and it is not modified in any substantial way. ABSTRACT The ability to program is one of the core tools used by computer scientists, and programming proficiency is a recommended requirement for ABET accreditation. In our experience, students learn programming skills best by writing many programs, ranging from simple to complex. Overworked teachers can be dismayed by the prospect of grading still more programs per student as well as teaching introductory classes with large enrollments. The automatic grading approach offers substantial advantages and opportunities, but also some challenges. We present WebBot, a web-based automatic grader for computer programming assignments. This program is an expansion of GradeBot, an automatic program grader used for several years. This newest version of GradeBot introduces a web-based interface. GradeBot evaluates student programs written in any of several languages, including C, C++, Java, Perl, Python, Tcl, and MIPS assembler. Guidance for similar projects is provided through a discussion of the development and use of GradeBot and WebBot.

Abstract- Do students really welcome failure, the mother of success? This paper reports the implementation of a trial-and-failure learning strategy, in both entry and senior level computer science classes. The idea is simple: given a sophisticated course project, let students try project submissions as many times as they want before the project deadline. For each submission, a thorough inspection is performed by an automated grading system called APOGEE. A student project has to accomplish not only the functional requirements but also many desired quality attributes such as robustness and security. In 2009, the trial-and-failure strategy has been adopted by four universities in five class sections. We report some interesting observations from student survey results, e.g., one can find out if factors like students ’ positive experiences of programming, choice of programming language, years of working experience, and instructors are predictive variables for positive attitudes toward the trial-and-failure learning experience as a whole.

The ability to program is one of the core tools used by computer scientists, and programming proficiency is a recommended requirement for ABET accreditation. In our experience, students learn programming skills best by writing many programs, ranging from simple to complex. Overworked teachers can be dismayed by the prospect of grading still more programs per student as well as teaching introductory classes with large enrollments. The automatic grading approach offers substantial advantages and opportunities, but also some challenges. We present WebBot, a web-based automatic grader for computer programming assignments. This program is an expansion of GradeBot, an automatic program grader used for several years. This newest version of GradeBot introduces a web-based interface. GradeBot evaluates student programs written in any of several languages, including C, C++, Java, Perl, Python, Tcl, and MIPS assembler. Guidance for similar projects is provided through a discussion of the development and use of GradeBot and WebBot.

It's hard to know where to begin. If ten years ago someone had said I'd be sitting here writing the acknowledgments to a Ph.D., I'd wouldn't have believed them. That I'm here today is, in part, due to the support and encouragement of some rather special people. First and foremost among these people are my parents who- despite not knowing quite what I've been doing over the last few years- have been supportive. I also wish to thank Bob Hamilton (my supervisor) who had faith in my ability to do a Ph.D. in the first instance. Whether or not this was such a good idea is open to debate, but nevertheless I'm sure Bob's intended well, and I appreciate this. There are several people around the Department who I'd like to thank. In particular, I appreciate the support and encouragement of Peter Cammock. Thanks also go to

Abstract — Computer-based grading tools have existed for nearly as long as computing courses. The majority of these tools have focused on completely automatic grading of functional requirements, leaving no room for subjectivity, and generally eschewing human feedback in favor of total automation. We argue that these tools are of little practical use because they severely limit the types of assessments that can be graded, and force the user to adopt to the paradigms of the grading tool, rather than vice versa. We present Agar, a tool designed to compensate for those possible shortcomings, discuss the design of Agar, and discuss unanticipated usage patterns that have already sprung up in our user base. I.

To reduce the workload of teachers and to improve the effectiveness of face-to-face courses, it is desirable to supplement them with Web-based tools which support the creation, management, submission, and assessment of assignments and tests. This paper presents our approach for supporting computer science education with software components which are integrated into a general-purpose content management system (CMS). We describe the design and implementation of these components, and we report on our practical experience with deploying the software in our courses.

I’m thankful to my advisor for aiming me sky high. I’m thankful to my wife for keeping me grounded. I’m thankful to Eamonn for the inspiration and Christian for the direction. I’m thankful to Sue, Todd, Mark, and Orit, for the opportunities they have given me. I’m thankful to Phil for starting at the end of the alphabet. I’m thankful to Marian for putting everything in perspective. I’m thankful for Steve for getting excited. I’m thankful to Vinh and Eric for their help. I’m thankful for Dan for lunch. I’m thankful to Brian and Kris for their support. I’m thankful to Peter for isolating the nastiest bug of the bunch. I am particularly thankful for the support of my family and friends, for everything that has led me to this point. iv ABSTRACT OF THE DISSERTATION