This work shows how a dialogue model can be represented as a Partially Observable Markov Decision Process (POMDP) with observations composed of a discrete and continuous component. The continuous component enables the model to directly incorporate a confidence score for automated planning. Using a testbed simulated dialogue management problem, we show how recent optimization techniques are able to find a policy for this continuous POMDP which outperforms a traditional MDP approach. Further, we present a method for automatically improving handcrafted dialogue managers by incorporating POMDP belief state monitoring, including confidence score information. Experiments on the testbed system show significant improvements for several example handcrafted dialogue managers across a range of operating conditions. 1

Multimodal interfaces are becoming increasingly ubiquitous with the advent of mobile devices, accessibility considerations, and novel software technologies that combine diverse interaction media. In addition to improving access and delivery capabilities, such interfaces enable flexible and personalized dialogs with websites, much like a conversation between humans. In this paper, we present a software framework for multimodal web interaction management that supports mixed-initiative dialogs between users and websites. A mixed-initiative dialog is one where the user and the website take turns changing the flow of interaction. The framework supports the functional specification and realization of such dialogs using staging transformations -- a theory for representing and reasoning about dialogs based on partial input. It supports multiple interaction interfaces, and offers sessioning, caching, and co-ordination functions through the use of an interaction manager. Two case studies are presented to illustrate the promise of this approach.

Personalization constitutes the mechanisms and technologies necessary to customize information access to the end-user. It can be defined as the automatic adjustment of information content, structure, and presentation tailored to the individual. In this chapter, we study personalization from the viewpoint of personalizing interaction. The survey covers mechanisms for information-finding on the web, advanced information retrieval systems, dialogbased applications, and mobile access paradigms. Specific emphasis is placed on studying how users interact with an information system and how the system can encourage and foster interaction. This helps bring out the role of the personalization system as a facilitator which reconciles the user's mental model with the underlying information system's organization. Three tiers of personalization systems are presented, paying careful attention to interaction considerations. These tiers show how progressive levels of sophistication in interaction can be achieved. The chapter also surveys systems support technologies and niche application domains.

Personalization constitutes the mechanisms necessary to automatically customize information content, structure, and presentation to the end-user to reduce information overload. Unlike traditional approaches to personalization, the central theme of our approach is to model a website as a program and conduct website transformation for personalization by program transformation (e.g., partial evaluation, program slicing). The goal of this paper is study personalization through a program transformation lens, and develop a formal model, based on program transformations, for personalized interaction with hierarchical hypermedia. The specific research issues addressed involve identifying and developing program representations and transformations suitable for classes of hierarchical hypermedia, and providing supplemental interactions for improving the personalized experience. The primary form of personalization discussed is out-of-turn interaction – a technique which empowers a user navigating a hierarchical website to postpone clicking on any of the hyperlinks presented on the current page and, instead, communicate the

Internet telephony — voice transmission and call signalling over IP networks — can pro-vide services far beyond those of the circuit-switched telephone network. This thesis discusses Internet telephony services in four broad areas: user-location services; multi-party conferenc-ing; the interworking of Internet telephony and mobile telephony; and Internet telephony feature interaction. User-location services are services which modify how a telephony server locates a user. Service authors need a way to control this process; this thesis presents two of them. The SIP Common Gateway Interface (SIP CGI) is a low-level server interface which allows fine-grained control of message processing in Session Initiation Protocol (SIP) servers. The Call Processing Language (CPL) is a protocol-independent, inherently safe high-level language for describing services in a way that is easily created and edited. The thesis also describes a general service framework providing a straightforward and powerful API atop which these and other service execution environments can be implemented, and an event thread architecture that makes imple-mentation of transaction-based protocols such as SIP efficient and scalable. Multi-party conferencing involves calls in which three or more people communicate si-

Recommendation and personalization attempt to reduce information overload and retain customers. While research in both recommender systems and personalization grew mainly out of information retrieval, both areas have emerged from nascent levels to veritable and challenging research areas in their own right. Whereas no technical or sophisticated methodologies exist by which to build such systems, the field also lacks a comprehensive, yet manageable survey by which to study recommenda-tion systems and personalization facilities. In this paper, we attempt to fill that gap by presenting a thematic approach toward studying recommendation and personalization. Specifically, we present three major representative personalization themes: rec-ommendation; induction, exploration, and exploitation of social networks; and personalization of information access. We unify the presentation of the three themes which we have extracted from the rich landscape of recommender system and personal-ization research via a functional metaphor, where inputs and output to a function are identified in each theme and instantiated through a number of systems and projects visited. In addition, we examine how a number of systems implement the function through various operators and techniques. Finally, we cover several broadening aspects, such as targeting, privacy and trust,

We describe the architecture and implementation of our comprehensive multi-platform collaboration framework known as Columbia InterNet Extensible Multimedia Architecture (CINEMA). It provides a distributed architecture for collaboration using synchronous communications like multimedia conferencing, instant messaging, shared web-browsing, and asynchronous communications like discussion forums, shared files, voice and video mails. It allows seamless integration with various communication means like telephones, IP phones, web and electronic mail. In addition, it provides value-added services such as call handling based on location information and presence status. The paper discusses the media services needed for collaborative environment, the components provided by CINEMA and the interaction among those components.

The rise of ubiquitous computing devices has provided the catalyst for the next generation World Wide Web, one that shifts the focus from the desktop computer to mobile devices such as cell phones and PDAs, in an ever increasing range of modalities. Web interaction management in this setting must contend with a plethora of interaction interfaces and a diverse range of content types in addition to helping realize the full potential of multimodality (i.e., supporting flexible and personalized interactions between humans and sites). This thesis presents WS://IM, a new software framework for web interaction management that is capable of supporting multimodal interactions. In addition to presenting a loosely bundled, factorized architecture that supports hyperlink interaction, WS://IM has the unique facilitation for out-of-turn interaction. Out-of-turn interaction is a novel technique that helps realize mixed-initiative interactions between humans and Web sites. Design methodology, implementation details, and exposition through three implemented case studies are provided. For My parents and grandparents. Your guidance, support, and encouragement have made all the difference in the world.

The increase in connected mobile computing devices has created the need for ubiquitous Web access. In many usage scenarios, it would be beneficial to interact multimodally. Current Web user interface description languages, such as HTML and VoiceXML, concentrate only on one modality. Some languages, such as SALT and X+V, allow combining aural and visual modalities, but they lack ease-of-authoring, since both modalities have to be authored separately. Thus, for ease-of-authoring and maintainability, it is necessary to provide a cross-modal user interface language, whose semantic level is higher. We propose a novel model, called XFormsMM, which includes XForms 1.0 combined with modality-dependent stylesheets and a multimodal interaction manager. The model separates modality-independent parts from the modality-dependent parts, thus automatically providing most of the user interface to all modalities. The model allows flexible modality changes, so that the user can decide, which modalities to use and when.

The public switched telephone network (PSTN) provides ubiquitous availability and very high scalability of more than a million busy hour call attempts per switch. If large carriers are to adopt Internet telephony, then Internet telephony servers should offer at least similar quantifi-able guarantees for scalability and reliability using metrics such as call setup latency, server call handling capacity, busy hour call arrivals, mean-time between failures and mean-time to recover. This thesis presents a reliable, scalable and interoperable Internet telephony architecture for user registration, call routing, conferencing and unified messaging using commodity hardware. The results extend beyond Internet telephony to encompass multimedia communication in general. The architecture presented in this thesis deals with two aspects: at least PSTN-grade re-liability and scalability of the Internet telephony servers, and interoperable Internet telephony services such as conferencing and voice mail using existing protocols. We describe the archi-tecture and implementation of our Session Initiation Protocol (SIP)-based enterprise Internet telephony architecture known as Columbia InterNet Extensible Multimedia Architecture (CIN-EMA). It consists of a SIP registration and proxy server, a multi-party conferencing server, a gateway for interworking SIP with ITU’s H.323, an interactive voice response system and a