Douglas B. Moran, Adam J. Cheyer, Luc E. Julia, David L. Martin, Sangkyu Park, "Multimodal User Interfaces in the Open Agent Architecture", Intelligent User Interfaces 1997, pp. 1-9.  Home/Search   Document Details and Download   Summary   Related Articles   Check

....wrappers and agent libraries that support interagent message passing, multithreaded processing, tracing of messages on the screen, and logging of messages in a history. Eventually an agent construction tool would be desirable, such as the one that is part of the Open Agent Architecture (OAA) [7], but such a tool is not possible under the current level of effort. To allow more complex responses to incoming requests, such as interrupting certain responses to reprioritize the computation, it is possible to combine SRI s PRS (Procedural Reasoning System) 20] with the MPA LISP wrapper. The ....

.... substrate for invoking procedures over a network: 9 # Knowledge Query and Manipulation Language (KQML) 4] developed with ARPI support by Loral and the University of Maryland Baltimore County # Inter Language Unification (ILU) 5] developed at Xerox PARC # Open Agent Architecture (OAA) [7], developed at SRI International Currently, MPA is demonstrated on top of KQML, until such time as efficiency or robustness considerations force us to consider another approach. A preliminary implementation of MPA on top of OAA also exists, and a partial implementation on top of ILU was ....

Douglas B. Moran, Adam J. Cheyer, Luc E. Julia, David L. Martin, and Sangkyu Park. Multimodal user interfaces in the Open Agent Architecture. In Proc. of the 1997.

....interactions rannatural and stilted. These behavioral agents are quite similar in motivation and functionality to the event recognizer architecture described here. This idea of a separate module for monitoring input from multiple modalities is common. For example, the Open Agent Architecture [11, 12] has a modality coordination agent that is responsible for producing a single meaning that matches the user s intention from multiple inputs. In general, understanding a person s intentions involves much more than recognizing patterns of behavior. However, the event recognizers we propose ....

Moran, D.B., et al. Multimodal user interfaces in the Open Agent Architecture. Proceedings of Intelligent User Interfaces. 1997. Orlando, FL: ACM.

....(pizza ( topping) extra cheese) Figure 2: Example for an translation dictionary. 6 Conclusions We propose a dialog system that allows a multi modal interaction between user and mobile software agents. This is an extension of earlier approaches that focuses just on stationary agents [5, 4, 2]. The user interacts with the agent by means of a complex dialog, which is summarized and translated in an agent communication language. Hence, this approach simplifies the problem of natural language understanding in a mobile environment significantly. The innovative aspects of this approach are ....

MORAN, D. B., CHEYER, A. J., JULIA, L. E., MARTIN, D. L., AND PARK, S. Multimodal user interfaces in the Open Agent Architecture. In Proc. of the 1997.

....often draw the first draft of their slides; we designed Multipoint as a tool for creating the first draft of a set of slides on a pen computer. MULTIPOINT IMPLEMENTATION Multipoint is implemented as an add on to Microsoft PowerPoint and uses SRI s Open Agent Architecture (OAA) [MORAN]. It communicates to PowerPoint via Visual Basic for Applications (VBA) OAA facilitates Engaging the audience Make a Rectangle Figure 1. The MultiPoint command make a rectangle converts a sketched rectangle into neatened form synchronous and asynchronous communication managed by a ....

Moran, D. B. and A. J. Cheyer, L. E. Julia, D. L. Martin, and S. Park, "Multimodal User Interfaces in the Open Agent Architecture," in Proc. of the 1997 International Conference on Intelligent User Interfaces, (Orlando, Florida), pp. 61--68, 1997.

....challenging space of perceptual user interface design to non programmers. Keywords multimodal, informal prototyping, design. 1. INTRODUCTION Significant progress has been made in developing multimodal architectures that fuse synergistic perceptual input modalities and execute across devices [1, 2, 3, 4]. However, these systems have not made multimodal user interface design easily approachable by non programmers or designers. We are working on a set of techniques and tools to make perceptual user interface systems easier to prototype. Here, we describe work to date on a tool for multimodal ....

....executes the application in Figure 1 in the multimodal browser. The browser shows the starting scene (a) the participant then draws the gesture n on the scene (b) the browser then transitions to the next pane in the multimodal storyboard shown in (c) using the Open Agent Architecture [1], which enables it to respond to gestures, speech commands, keyboard input, or any other input mechanism for which there is an associated input agent. With this architecture, we can also substitute humans as Wizard of Oz recognition agents. 3. DEFINING INTERACTION The fluid interaction in ....

[Article contains additional citation context not shown here]

Moran, L.B., et al., Multimodal user interfaces in the open agent architecture. Knowledge-Based Systems, 1998. 10(5): p. 295-304.

....such as Jini [18] Our user interface adaptation work can be seen as a similar thrust but focusing on interface component adaptation rather than service look up and adaptation. Work on adaptable user interface systems [12, 5, 27] intelligent user interfaces [20, 21] and agent based systems [15, 23] has contributed to the development of techniques supporting various kinds of interface adaptation. Some adaptable and agent based systems support techniques for designing and implementing user interface adaptation facilities. However, such approaches use custom architectures and implementations ....

Moran DR, Cheyer AJ, Julia LE, Martin DL, Sangkyu Park. Multimodal user interfaces in the Open Agent Architecture.

....on the screen, and logging of messages in a history. MPA agents have been implemented in C, C , LISP, and Java. Further documentation can be found at the MPA website. Eventually, an agent construction tool would be desirable, such as the one that is part of the Open Agent Architecture (OAA) [7]. To allow more complex responses to incoming requests, such as interrupting certain responses to reprioritize the computation, it is possible to combine SRI s PRS (Procedural Reasoning System) 21] with the MPA LISP wrapper. PRS is the basis for our cell manager, plan server, and meta ....

.... substrate for invoking procedures over a network: # Knowledge Query and Manipulation Language (KQML) 4] developed with ARPI support by Loral and the University of Maryland Baltimore County # Inter Language Unification (ILU) 5] developed at Xerox PARC # Open Agent Architecture (OAA) [7], developed at SRI International Currently, MPA is demonstrated on top of KQML, until such time as efficiency or robustness considerations force us to consider another approach. A preliminary implementation of MPA on top of OAA also exists, and a partial implementation on top of ILU was ....

Douglas B. Moran, Adam J. Cheyer, Luc E. Julia, David L. Martin, and Sangkyu Park. Multimodal user interfaces in the Open Agent Architecture. In Proc. of the

....have the possibility of providing the system with additional information by clicking on an apartment icon (see Figure 1) or marking areas on an interactive map of Stockholm. Considerable efforts to study multimodal input in map task domains have been reported by the SRI group, see for example [11]. We will therefore address the question of how to integrate the results from a speech recognizer with mouse input such as the selection of an icon or the specification of an area on the map. The system requires a dialogue manager capable of integrating input output modalities [14] resolving ....

Moran D, Cheyer A, Julia L, Martin D and Park S (1998) Multimodal User Interfaces in the Open Agent Architecture. Journal of Knowledge Based Systems, 10:295-303.

....systems are not able to influence the order in which a user processes a document, 22 or the speed of processing. Unlike these systems, MAGIC s and PPP s design and realization components also handle the temporal layout of presentations. The second group is represented by Moran and colleagues [55] who focus on the combination and reuse of existing software components in new multimedia applications. To support the creation of applications from agents written in multiple languages and running on different platforms, e.g. a text generator running on a Sun and a graphics generator running on a ....

D.B. Moran, A.J. Cheyer, L.E. Julia, D.L. Martin, and S. Park. Multimodal User Interfaces in the Open Agent Architecture. In Proceedings of the 1997 International Conference on Intelligent User Interfaces, pages 61--70, Orlando, Florida, 1997.

....MPA provides wrappers and agent libraries that support asynchronicity, interagent communication, tracing of messages on the screen, and logging of messages in a history. Eventually an agent construction tool would be desirable, such as the one that is part of the Open Agent Architecture (OAA) [4], but such a tool is not possible under the current level of effort. 3 Grasper CL [3] is a programming language extension to LISP that introduces graphs arbitrarily connected networks as a primitive data type. Grasper CL is used as a uniform basis for the man machine interface of many ....

Douglas B. Moran, Adam J. Cheyer, Luc E. Julia, David L. Martin, and Sangkyu Park. Multimodal user interfaces in the open agent architecture. In Proc. of the 1997 International Conference on Intelligent User Interfaces (IUI97), Orlando, Florida, 6-9 January 1997.

....in a variety of domains. Interface Agents are in turn a branch of the field of software agents which exist as the primary point of contact between a human and a computer. Their style of implementation can range as wide as currently existing graphical user interfaces or a voice over a telephone (Moran, et al. 1997) to a life size, full body 3D image in a virtual room (Maes 1995) The type of interface agent most frequently implemented to date however is a 2D miniature full body animated cartoon figure which appears to float over a graphical user interface windows environment or web page. Pedagogical Agent ....

Moran, D B, et al. 1997. Multimodal User Interfaces in the Open Agent Architecture, IUI Conference Proceedings 1997.

....Dragon Dictate [14] and IBM ViaVoice [19] 4. Speech manipulating interfaces: Speech Skimmer, a system for scanning a collection of speech segments [2] and Storywriter, a speech based document editing system [13] 5. Multimodal applications: Multimodal maps that use both pen and speech input [9, 28]. We have designed SUEDE to support the first two of these speech interface styles. In its current form, SUEDE is not suited for prototyping applications that have alternative modes of feedback (such as the text in a dictation or the graphics in a multimodal application) or involve manipulating ....

Moran, L.B., A.J. Cheyer, L.E. Julia, D.L. Martin, and S. Park, Multimodal user interfaces in the open agent architecture. Knowledge-Based Systems, 1998. 10(5): p. 295-304.

....system independent . provide a component based software architecture that can distribute tasks to other agents on the network Our research in the Center for Human Computer Communication at the Oregon Graduate Institute and earlier work by Cohen, Oviatt, and colleagues at SRI International [2, 3] has focused on empirical and applied research in the areas of multiagent 2 Figure 1a. QuickSet on an HP620LX running Windows CE Figure 1b. QuickSet on an ITI Digital Desk architectures, multimodal humancomputer interaction and humancomputer collaboration. This research led to the ....

D. B. Moran, A. J. Cheyer, L. E. Julia, D. L. Martin, and S. Park, "Multimodal user interfaces in the Open Agent Architecture," In the proceedings of the 1997 International Conference on Intelligent User Interfaces (IUI97), Orlando, Florida, 1997.

....it should be general enough to be used for displaying data similar to the output from UCP produced by any NL program. Some e orts related to GATE has been made both in the US and in Europe, e.g. the multimodal Open Agent Architecture TM (OAA) developed by SRI International [Cohen et al. 1989, Moran et al. 1997] the Advanced Language Engineering Platform (ALEP) which is an initiative of the European Commission [Simpkins Groenendijk 1994, Bredenkamp et al. 1997] and the German Verbmobil architecture [Bub Schwinn 1996] Although the latter architecture is primarily intended to t the needs of a ....

Douglas B. Moran, Adam J. Cheyer, Luc E. Julia, David L. Martin, and Sangkyu Park. \Multimodal User Interfaces in the Open Agent Architecture". In Proceedings of the International Conference on Intelligent User Interfaces, pp. 61-68, Orlando, Florida, 1997. ACM. BIBLIOGRAPHY 58

.... SS = Speech Synthesis, FS = Face Synthesis, A = Audio Feedback, H = Haptic Feedback, VC = Video Conferencing ffl Cooperation: E = Equivalence, C = Complementarity, R = Redundancy, CC = Concurrency, S = Specialization, T = Transfer ffl Architectures: MMI [383] PAC Amodeus [272] OAA [261] For illustration we briefly describe two multimodal applications for which images were available to the authors. QuickDoc [389] allows the user to view, manipulate, and summarize multimedia data using multimodal interaction techniques. A doctor looks through a series of X ray images or CT ....

....applications with synergetic (and thus any) cooperation of modalities. The core component of PAC Amodeus is the Dialogue Controller a set of cooperating agents that capture parallelism and information processing at multiple levels of abstraction. 68 ffl SRI s Open Agent Architecture (OAA) [261] provides access to agent based applications through intelligent, cooperative, distributed agent based user interfaces. It currently supports a mix of spoken language, handwriting, 2D gestures, in addition to standard input modalities (keyboard and mouse input) Since only the primary user ....

D.B. Moran, A.J. Cheyer, L.E. Julia, and D.L. Martin. Multimodal user interfaces in the Open Agent Architecture. In Intelligent User Interfaces, pages 61--68, Orlando (FL), 1997.

.... SS = Speech Synthesis, FS = Face Synthesis, A = Audio Feedback, H = Haptic Feedback, VC = Video Conferencing ffl Cooperation: E = Equivalence, C = Complementarity, R = Redundancy, CC = Concurrency, S = Specialization, T = Transfer ffl Architectures: MMI [323] PAC Amodeus [224] OAA [214] For illustration we briefly describe two multimodal applications for which images were available to the authors. QuickDoc [329] allows the user to view, manipulate, and summarize multimedia data using multimodal interaction techniques. A doctor looks through a series of X ray images or CT scans, ....

....applications with synergetic (and thus any) cooperation of modalities. The core component of PAC Amodeus is the Dialogue Controller a set of cooperating agents that capture parallelism and information processing at multiple levels of abstraction. ffl SRI s Open Agent Architecture (OAA) [214] provides access to agent based applications through intelligent, cooperative, distributed agent based user interfaces. It currently supports a mix of spoken language, handwriting, 2D gestures, in addition to standard input modalities (keyboard and mouse input) Since only the primary user ....

D.B. Moran, A.J. Cheyer, L.E. Julia, and D.L. Martin. Multimodal user interfaces in the Open Agent Architecture. In Intelligent User Interfaces, pages 61--68, Orlando (FL), 1997.

....the coordination of the subtasks required to accomplish the end goal. The OAA has been used to implement more than twenty different applications, including x Multi robot control and coordination [4] x Office automation and unified messaging [2] x Collaborative multimodal user interfaces [1, 12] x Frontends [8] and backends [11] for the Web x Development tools [10] for creating and assembling new agents with the OAA Each OAA project can take advantage of the core services provided by the architecture as well as of the growing number of technologies now accessible through an agent ....

Moran, D., Cheyer, A., Julia, L., and Park, S. Multimodal user interfaces in the Open Agent Architecture. In Proc. of IUI-97, pages 61---68. Orlando, Jan. 1997.

....with a multimodal (pen and voice) interface over the web. Our implementation approach was to add Java to the set of languages accepted by the Open Agent Architecture (OAA) a framework for rapidly prototyping complex applications, and particularly suited to those with multimodal interfaces [1]. Given the OAA s distributed nature, an OAA based application can be run from a lightweight computer by downloading only the small user interface component, while the core of the application is implemented on a server composed of larger agents (e.g. speech recognition (SR) natural language, ....

Moran Douglas, Cheyer Adam, Julia Luc, Martin David. and Park Sangkyu. Multimodal User Interfaces in the Open Agent Architecture. In Proceedings of IUI'97 (Orlando, FL), 1997, pp. 61-68.

No context found.

Douglas B. Moran, Adam J. Cheyer, Luc E. Julia, David L. Martin, Sangkyu Park, "Multimodal User Interfaces in the Open Agent Architecture", Intelligent User Interfaces 1997, pp. 1-9.

No context found.

Douglas B. Moran, Adam J. Cheyer, Luc E. Julia, David L. Martin, Sangkyu Park, "Multimodal User Interfaces in the Open Agent Architecture", Intelligent User Interfaces 1997, pp. 1-9.

No context found.

Moran, D.B., Cheyer, A.J., Julia, L.E., Martin, D.L., and Park, S., Multimodal User Interfaces in the Open Agent Architecture, in Proceedings of IUI'97 Orlando, FL, ACM Press, 1997, 61-68.

No context found.

Moran, D. B. and A. J. Cheyer, L. E. Julia, D. L. Martin, and S. Park, "Multimodal User Interfaces in the Open Agent Architecture," in Proc. of the 1997 International Conference on Intelligent User Interfaces,

No context found.

Moran, D. B., A. J. Cheyer, L. E. Julia, D. L. Martin, and S. Park, "Multimodal user interfaces in the Open Agent Architecture," in Proc. of the 1997 International Conference on Intelligent User Interfaces (IUI97). pp. 61--68, Orlando, FL, January 1997 http://www.ai.sri.com/pubs/papers/Mora97:Multimodal/document.ps

No context found.

Moran, D.B., Cheyer, A.J., Julia, L.E., Martin, D.L., and Park, S. (1997) Multimodal user interfaces in the Open Agent Architecture. Proceedings of IUI97. pp 61-68.

No context found.

Douglas B. Moran, Adam J. Cheyer, Luc E. Julia, David L. Martin, and Sangkyu Park. 1997. Multimodal user interfaces in the Open Agent Architecture. In Proceedings of the International Conference on Intelligent User Interfaces, pages 61--68, Orlando, Florida, January. ACM.

First 50 documents

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC