Abstract: This study examines the implicit and explicit heuristics and usability evaluation processes utilized by a leading game developer. Five people from a single game team, each contributing in a different way to the game development process, were observed for one business day and interviewed. At the time of this study, the participating game team was at the tail end of their first month of prototyping in the pre-production phase of development. The data collected and literature reviewed combine to suggest that instituting more formal usability evaluation processes could be helpful to the game development process. The heuristics created in this study are a starting point for the construction of a standard list of game heuristics for use by the game development community. 2 TABLE OF CONTENTS

Abstract: THEA is a technique designed for use by interactive system designers and engineers to help anticipate interaction failures. These may become problematic once designs become operational. The technique employs a cognitive error analysis based on an underlying model of human information processing. It is a highly structured approach, intended for use early in the development lifecycle as design concepts and requirements concerned with safety and usability – as well as functionality – are emerging. We believe the technique advances the systematic identification of human-computer interaction error through its straightforward application, requiring minimal formal knowledge of human factors or cognitive psychology.

There are two main kinds of approach to considering usability of any system: empirical and analytical. Empirical techniques involve testing systems with users, whereas analytical techniques involve usability personnel assessing systems using established theories and methods. We report here on a set of studies in which four different techniques were applied to various digital libraries, focusing on the strengths, limitations and scope of each approach. Two of the techniques, Heuristic Evaluation and Cognitive Walkthrough, were applied in text-book fashion, because there was no obvious way to contextualize them to the Digital Libraries (DL) domain. For the third, Claims Analysis, it was possible to develop a set of re-usable scenarios and personas that relate the approach specifically to DL development. The fourth technique, CASSM, relates explicitly to the DL domain by combining empirical data with an analytical approach. We have found that Heuristic Evaluation and Cognitive Walkthrough only address superficial aspects of interface design (but are good for that), whereas Claims Analysis and CASSM can help identify deeper conceptual difficulties (but demand greater skill of the analyst). However, none fit seamlessly with existing digital library development practices, highlighting an important area for further work to support improved usability.

Design representations in user-centered design serve intentions for directing design process and communication functions for enlisting interdisciplinary participation. To disentangle these two factors, a vocabulary for identifying communication functions in design is proposed. This vocabulary, drawn from a selective review of empirical studies of design activity in architecture and engineering, is then applied to three design cases from user-centered design. This analysis shows how representational use is subject to adaptive pressure from the communication demands in interdisciplinary teams. The consequences of this pressure for understanding the nature of design are discussed. ACM Classification: K.6.1 [Management of Computing

Think-aloud testing is a widely employed usability evaluation method, yet its use in practice is rarely studied. We report an explorative study of 14 think-aloud sessions, the audio recordings of which were examined in detail. The study shows that immediate analysis of observations made in the think-aloud sessions is done only sporadically, if at all. When testing, evaluators seem to seek confirmation of problems that they are already aware of. During testing, evaluators often ask users about their expectations and about hypothetical situations, rather than about experienced problems. In addition, evaluators learn much about the usability of the tested system but little about its utility. The study shows how practical realities rarely discussed in the literature on usability evaluation influence sessions. We discuss implications for usability researchers and professionals, including techniques for fast-paced analysis and tools for capturing observations during sessions. Author Keywords Usability evaluation, think aloud testing, industrial

In the course of this research project the output of four Task Analysis (TA) methods were investigated, explored and evaluated to ascertain whether they could support the Requirements Analysis (RA) phase and so contribute directly to other activities in the development life cycle for Interactive MultiMedia (IMM) systems. The research discusses the success and failure factors particular TA methods. The problems of an IMM systems development life cycle are linked to the weaknesses of the Requirements Analysis phase and in particular to the incomplete support of TA methods and techniques used within the Requirements Analysis phase. The outputs of the selected TA methods are evaluated according to four factors, which are represented as an Evaluation Framework (EF). Each factor represents specific criteria and features that TA methods should cover in their processes and outputs. The findings show that TA methods have a number of weaknesses in the support of and the contributions they make. Therefore questions and recommendations are considered about how the methods can be improved in order to obtain better requirements.

Usability inspection methods, such as heuristic evaluation, the cognitive walkthrough, formal usability inspections, and the pluralistic usability walkthrough, were introduced fifteen years ago. Since then, these methods, analyses of their comparative effectiveness, and their use have evolved in different ways. In this paper, we track the fortunes of the methods and analyses, looking at which led to use and to further research, and which led to relative methodological dead ends. Heuristic evaluation and the cognitive walkthrough appear to be the most actively used and researched techniques. The pluralistic walkthrough remains a recognized technique, although not the subject of significant further study. Formal usability inspections appear to have been incorporated into other techniques or largely abandoned in practice. We conclude with lessons for practitioners and suggestions for future research.

A case study on integrating contextual information with analytical usability evaluation

In this paper we give an overview of existing approaches for capturing HCD(Human-Centred Design) process and design knowledge. We present an alternative approach that aims at fostering the integration of UE (Usability Engineering) activities and artifacts into existing software development processes. The approach is based on six claims that are derived from an analysis of existing UE process models and requirements of software developers. Our approach is embeddable in existing process improvement frameworks such as the UMM (Usability Maturity Model) and is supported by a web-based tool. An explorative study that we have conducted with software developers from various software development organizations confirms the potential of our approach. However the study indicates that our approach is more strongly preferred by developers with experience in user interface design.

In the context of a software development organization, two strategies are possible for introducing and institutionalizing new usability engineering methods. The first one, expert-based institutionalization, require to resort to third party companies or experts that can, based its previous expertise, assist the team in selecting, implementing and institutionalizing usability engineering methods and tools. The second one, a measurement-based strategy, is based on empirical evidence for learning and assessing the appropriateness, usefulness of a usability engineering method. This paper proposed to combine these two approaches in a single process metrics support environment for selecting and institutionalizing usability engineering methods. The proposed approach has been validated via in a cross-organizational empirical study involving several software engineers from five mediums to large sized software development companies.