Advances in graphical technology have now made it possible for us to interact with information in innovative ways, most notably by exploring multimedia environments and by manipulating three-dimensional virtual worlds. Many benefits have been claimed for this new kind of interactivity, a general assumption being that learning and cognitive processing are facilitated. We point out, however, that little is known about the cognitive value of any graphical representations, be they good old-fashioned (e.g. diagrams) or more advanced (e.g. animations, multimedia, virtual reality). In our paper, we critique the disparate literature on graphical representations, focusing on four representative studies. Our analysis reveals a fragmented and poorly understood account of how graphical representations work, exposing a number of assumptions and fallacies. As an alternative we propose a new agenda for graphical representation research. This builds on the nascent theoretical approach within cognitive science that analyses the role played by external representations in relation to internal mental ones. We outline some of the central properties of this relationship that are necessary for the processing of graphical representations. Finally, we consider how this analysis can inform the selection and design of both traditional and advanced forms of graphical technology.

Self-explaining is an effective metacognitive strategy that can help learners develop deeper understanding of the material they study. This experiment explored if the format of material (i.e., text or diagrams) influences the self-explanation effect. Twenty subjects were presented with information about the human circulatory system and prompted to self-explain; 10 received this information in text and 10 in diagrams. Results showed that students given diagrams performed significantly better on post-tests than students given text. Diagrams students also generated significantly more self-explanations that text students. Furthermore, the benefits of self-explaining were much greater in the diagrams condition. To discover why diagrams can promote the self-explanation effect, results are interpreted with reference to the multiple differences in the semantic, cognitive and affective properties of the texts and diagrams studied.

There is widespread anecdotal evidence that expert programmers make use of visual mental images when they are designing programs. This evidence is used to justify the use of diagrams and visual programming languages during software design. This paper reports the results of two studies. In the first, expert programmers were directly questioned regarding the nature of their mental representations while they were engaged in a design task. This investigative technique was used with the explicit intention of eliciting introspective reports of mental imagery. In the second, users of a visual programming language responded to a questionnaire in which they were asked about cognitive processes. The resulting transcripts displayed a considerable number of common elements. These suggest that software design shares many characteristics of more concrete design disciplines. The reports from participants in the two studies, together with previous research into imagery use, indicate potential...

This paper describes an evaluation of an intelligent labelling explorer (ILEX),

Modern computer systems routinely present information to the user as a combination of text and diagrammatic images, described as "graphical user interfaces". Practitioners and researchers in Human-Computer Interaction (HCI) generally believe that the value of these diagrammatic representations is derived from metaphorical reasoning; they communicate abstract information by depicting a physical situation from which the abstractions can be inferred. This assumption has been prevalent in HCI research for over 20 years, but has seldom been tested experimentally. This thesis analyses the reasons why diagrams are believed to assist with abstract reasoning. It then presents the results of a series of experiments testing the contribution of metaphor to comprehension, problem solving, explanation and memory tasks carried out using a range of different diagrams. The results indicate that explicit metaphors provide surprisingly little benefit for cognitive tasks using diagrams as an external re...

Abstract not found

Abstract: Multimedia learning environments combine multiple forms of representations like texts, static and animated pictures or graphs. Knowledge acquisition from multiple representations requires that learner create referential connections between corresponding elements and corresponding structures in different representations. As this process is usually difficult, learners frequently fail to construct coherent mental representations and, thus, do not sufficiently understand the subject matter. This paper analyzes the effects of different kinds of instructional help on the process of coherence formation from multiple representations by learners with different prior knowledge. Three groups of university students with different domain-specific knowledge had to learn a complex subject matter from chemistry using six different forms of representation. In addition, a first group received directive help for coherence formation. A second group received non-directive help, and a third group received no instructional help. Results indicate that directive help is effective for recall performance because of its summarizing and repeating function. Furthermore learners with different levels of prior knowledge show different reactions when help is given. For learners with insufficient prior knowledge help is not helpful or in case of recall performance even harmful. Learners with a medium level of prior knowledge can increase

Recursion is an important problem solving technique used in programming. It is also a highly unfamiliar mental activity and many computing novices have difficult understanding recursion and applying recursive techniques in problem solving. Research studies have concluded that novices and experts differ in their mental models of recursion. Novices seem to possess various inadequate models of recursion especially the iterative or loop model. This paper examines whether novices who are aided in acquiring an expert's mental model of recursion (the copies model) can effectively use this model in evaluating recursive algorithms. Results of a study indicated that a large percentage of novices who had previously demonstrated an understanding of the copies model (using explicit diagrammatic traces) failed do so when not using diagrammatic traces. In fact, they appeared to demonstrate evidence for the incorrect iterative or loop model when trying to mentally evaluate recursive programs. The resu...

Abstract. DEMIST is a multi-representational simulation environment that supports understanding of the representations and concepts of population dynamics. We report on a study with 18 subjects with little prior knowledge that explored if DEMIST could support their learning and asked what decisions learners would make about how to use the many representations that DEMIST provides. Analysis revealed that using DEMIST for one hour significantly improved learners ’ understanding of population dynamics though their knowledge of the relation between representations remained weak. It showed that learners used many of DEMIST’s features. For example, they investigated the majority of the representational space, used dynalinking to explore the relation between representations and had preferences for representations with different computational properties. It also revealed that decisions made by designers impacted upon what is intended to be a free discovery environment. 1

this paper is to present a comparative study of learner modelling systems that involve the learner in diagnosis and open the learner model for inspection, change and discussion. We outline potentials of this approach and refer to some problems that need further investigation.