The iStuff toolkit of physical devices, and the flexible software infrastructure to support it, were designed to simplify the exploration of novel interaction techniques in the post-desktop era of multiple users, devices, systems and applications collaborating in an interactive environment. The toolkit leverages an existing interactive workspace infrastructure, making it lightweight and platform independent. The supporting software framework includes a dynamically configurable intermediary to simplify the mapping of devices to applications. We describe the iStuff architecture and provide several examples of iStuff, organized into a design space of ubiquitous computing interaction components. The main contribution is a physical toolkit for distributed, heterogeneous environments with run-time retargetable device data flow. We conclude with some insights and experiences derived from using this toolkit and framework to prototype experimental interaction techniques for ubiquitous computing environments.

Bare hand pointing on touch screens both benefits and suffers from the nature of direct input. This work explores techniques to overcome its limitations. Our goal is to design interaction tools allowing pixel level pointing in a fast and efficient manner. Based on several cycles of iterative design and testing, we propose two techniques: Cross-Keys that uses discrete taps on virtual keys integrated with a crosshair cursor, and an analog Precision-Handle that uses a leverage (gain) effect to amplify movement precision from the user’s finger tip to the end cursor. We conducted a formal experiment with these two techniques, in addition to the previously known Zoom-Pointing and Take-Off as baseline anchors. Both subjective and performance measurements indicate that Precision-Handle and Cross-Keys complement existing techniques for touch screen interaction.

In ubiquitous computing environments, multiple users work with a wide range of different devices. In many cases, users interact and collaborate using multiple heterogeneous devices at the same time. The configuration of the devices should be able to change frequently due to a highly dynamic, flexible and mobile nature of new work practices. This produces new requirements for the architecture of an appropriate software infrastructure. In this paper, an architecture designed to meet these requirements is proposed. To test its applicability, this architecture was used as the basis for the implementation of BEACH, the software infrastructure of i-LAND (the ubiquitous computing environment at GMD-IPSI). It provides the functionality for synchronous cooperation and interaction with roomware components, i.e. room elements with integrated information technology. In conclusion, our experiences with the current implementation are presented.

This paper describes how our use of video artifacts affected the re-design of a graphical editor (Design/CPN) for building, simulating, and analyzing Coloured Petri Nets. The two primary goals of the project were to create design abstractions that integrate recent advances in graphical interaction techniques and to explicitly support the various patterns of using Petri Nets in real-world settings.

In this paper, a conceptual model for synchronous applications in ubiquitous computing environments is proposed. To test its applicability, it was used to structure the architecture of the BEACH software framework that is the basis for the software infrastructure of i-LAND (the ubiquitous computing environment at FhG-IPSI). The BEACH framework provides the functionality for synchronous cooperation and interaction with roomware components, i.e. room elements with integrated information technology. To show how the BEACH model and framework can be applied, the design of a sample application is explained. Also, the BEACH model is positioned against related work. In conclusion, we provide our experiences with the current implementation.

Toolglass [Bier et al. 1993] demonstrated a two-handed command selection technique that combined command selection and direct manipulation. While empirical evaluations showed the speed advantage of ToolGlass, they did not show the relative importance of two possible factors in its improved performance: 1) the use of two hands and 2) the merging of command selection and direct manipulation. We conducted a study comparing the relative benefits of three command selection techniques that merge command selection and direct manipulation: one two-handed technique, Toolglass, and two one-handed techniques, control menus [Pook et al., 2000] and FlowMenu [Guimbretière and Winograd, 2000]. Participants performed sequences of operations that required both selecting a color and designating the endpoints of a line. Our results show that control menus and FlowMenu are significantly faster than Toolglass. Further analysis suggests that the merging of command selection and direct manipulation is the most important factor in the performance of all three techniques.

We present a method for browsing videos by directly dragging their content. This method brings the benefits of direct manipulation to an activity typically mediated by widgets. We support this new type of interactivity by: 1) automatically extracting motion data from videos; and 2) a new technique called relative flow dragging that lets users control video playback by moving objects of interest along their visual trajectory. We show that this method can outperform the traditional seeker bar in video browsing tasks that focus on visual content rather than time. ACM Classification: H5.2 [Information interfaces and presentation]: User Interfaces.- Graphical user interfaces.

We have developed a series of spatial hypertext systems that support early stages of linear-information authoring, such as paper writing and movie editing. They are designed based on the ART (Amplifying Representational Talkback) principle, which emphasizes the importance of visual interaction and the power of external representations. The systems use spatial hypertext not as a medium for representing final artifacts but as a means of interacting with linear information during an authoring process. This paper first describes the role and the effect of the spatial hypertext representation plays in support of early stages of authoring linear information, and explains the ART interaction model for the approach. The ART#001 system, which supports early stages of writing, is described in detail and the other three ART systems are used to illustrate the essential aspects of our approach. The paper concludes with a discussion on the semiotic interpretation of spatial hypertext as a representation, and on the innovative use of spatial hypertext as an instrument to compose information, rather than as an information medium.

We are in the midst of an explosion of emerging human-computer interaction techniques that have redefined our understanding of both computers and interaction. We propose the notion of Reality-Based Interaction (RBI) as a unifying concept that ties together a large subset of these emerging interaction styles. Through RBI we are attempting to provide a framework that can be used to understand, compare, and relate current paths of HCI research. Viewing interaction through the lens of RBI can provide insights for designers and allows us to find gaps or opportunities for future development. Furthermore, we are using RBI to develop new evaluation techniques for features of emerging interfaces that are currently unquantifiable.

Figure 1: Designing a puzzle UI with MaggLite. (a) Drawing UI. (b) Configuring interactions. (c) Using the application. This article presents MaggLite, a toolkit and sketch-based interface builder allowing fast and interactive design of post-WIMP user interfaces. MaggLite improves design of advanced UIs thanks to its novel mixed-graph architecture that dynamically combines scene-graphs with interactiongraphs. Scene-graphs provide mechanisms to describe and produce rich graphical effects, whereas interaction-graphs allow expressive and fine-grained description of advanced interaction techniques and behaviors such as multiple pointers management, toolglasses, bimanual interaction, gesture, and speech recognition. Both graphs can be built interactively by sketching the UI and specifying the interaction using a dataflow visual language. Communication between the two graphs is managed at runtime by components we call Interaction Access Points. While developers can extend the toolkit by refining built-in generic mechanisms, UI designers can quickly and interactively design, prototype and test advanced user interfaces by applying the MaggLite principle: “draw it, connect it and run it”. Categories and Subject Descriptors: H5.2 [Information Interfaces]: User Interfaces―Graphical user interfaces