Top end mobile phones include a number of specialized (e.g., accelerometer, compass, GPS) and general purpose sensors (e.g., microphone, camera) that enable new people-centric sensing applications. Perhaps the most ubiquitous and unexploited sensor on mobile phones is the microphone – a powerful sensor that is capable of making sophisticated inferences about human activity, location, and social events from sound. In this paper, we exploit this untapped sensor not in the context of human communications but as an enabler of new sensing applications. We propose SoundSense, a scalable framework for modeling sound events on mobile phones. SoundSense is implemented on the Apple iPhone and represents the first general purpose sound sensing system specifically designed to work on resource limited phones. The architecture and algorithms are designed for scalability and SoundSense uses a combination of supervised and unsupervised learning techniques to classify both general sound types (e.g., music, voice) and discover novel sound events specific to individual users. The system runs solely on the mobile phone with no back-end interactions. Through implementation and evaluation of two proof of concept peoplecentric sensing applications, we demostrate that SoundSense is capable of recognizing meaningful sound events that occur in users ’ everyday lives. Categories and Subject Descriptors

A growing number of mobile computing applications are centered around the user’s location. The notion of location is broad, ranging from physical coordinates (latitude/longitude) to logical labels (like Starbucks, McDonalds). While extensive research has been performed in physical localization, there have been few attempts in recognizing logical locations. This paper argues that the increasing number of sensors on mobile phones presents new opportunities for logical localization. We postulate that ambient sound, light, and color in a place convey a photo-acoustic signature that can be sensed by the phone’s camera and microphone. In-built accelerometers in some phones may also be useful in inferring broad classes of user-motion, often dictated by the nature of the place. By combining these optical, acoustic, and motion attributes, it may be feasible to construct an identifiable fingerprint for logical localization. Hence, users in adjacent stores can be separated logically, even when their physical positions are extremely close. We propose SurroundSense, a mobile phone based system that explores logical localization via ambience fingerprinting. Evaluation results from 51 different stores show that SurroundSense can achieve an average accuracy of 87 % when all sensing modalities are employed. We believe this is an encouraging result, opening new possibilities in indoor localization.

Urban sensing, participatory sensing, and user activity recognition can provide rich contextual information for mobile applications such as social networking and location-based services. However, continuously capturing this contextual information on mobile devices is difficult due to battery life limitations. In this paper, we present the framework design for an Energy Efficient Mobile Sensing System (EEMSS) that powers only necessary and energy efficient sensors and manages sensors hierarchically to recognize user state as well as detect state transitions. We also present the design, implementation, and evaluation of EEMSS that automatically recognizes user daily activities in real time using sensors on an off-the-shelf high-end smart phone. Evaluation of EEMSS with 10 users over one week shows that it increases the smart phone’s battery life by more than 75% while maintaining both high accuracy and low latency in identifying transitions between end-user activities.

This paper describes Mercury, a wearable, wireless sensor platform for motion analysis of patients being treated for neuromotor disorders, such as Parkinson’s Disease, epilepsy, and stroke. In contrast to previous systems intended for short-term use in a laboratory, Mercury is designed to support long-term, longitudinal data collection on patients in hospital and home settings. Patients wear up to 8 wireless nodes equipped with sensors for monitoring movement and physiological conditions. Individual nodes compute high-level features from the raw signals, and a base station performs data collection and tunes sensor node parameters based on energy availability, radio link quality, and application specific policies. Mercury is designed to overcome the core challenges of long battery lifetime and high data fidelity for long-term studies where patients wear sensors continuously 12 to 18 hours a day. This requires tuning sensor operation and data transfers based on energy consumption of each node and processing data under severe computational constraints. Mercury provides a high-level programming interface that allows a clinical researcher to rapidly build up different policies for driving data collection and tuning sensor lifetime. We present the Mercury architecture and a detailed evaluation of two applications of the system for monitoring patients with Parkinson’s Disease and epilepsy.

Today’s mobile phones represent a rich and powerful computing platform, given their sensing, processing and communication capabilities. Phones are also part of the everyday life of billions of people, and therefore represent an exceptionally suitable tool for conducting social and psychological experiments in an unobtrusive way. In this paper we illustrate EmotionSense, a mobile sensing platform for social psychology studies based on mobile phones. Key characteristics include the ability of sensing individual emotions as well as activities, verbal and proximity interactions among members of social groups. Moreover, the system is programmable by means of a declarative language that can be used to express adaptive rules to improve power saving. We evaluate a system prototype on

Location-based social applications (LBSAs) rely on the location coordinates of the users to provide services. Today, smartphones using these applications act as simple clients and send out user locations to untrusted third-party servers. These servers have the application logic to provide the service, and in the process collect large amounts of user location information over time. This design, however, is shown to be susceptible to large-scale user privacy compromises even if several location cloaking techniques are employed. In this position paper, we argue that the LBSAs should adapt an approach where the untrusted third-party servers are treated simply as encrypted data stores, and the application functionality be moved to the client devices. The location coordinates are encrypted, when shared, and can be decrypted only by the users that the data is intended for. This approach significantly improves user location privacy. We argue that this approach not only improves privacy, but it is also flexible enough to support a wide variety of location-based applications used today. In this paper, we identify the key building blocks necessary to construct the applications in this approach, give examples of using the building blocks by constructing several applications, and outline the privacy properties provided by this approach. We believe our approach provides a practical alternative design for LBSAs that is deployable today. 1.

Abstract. There are endless possibilities for the next generation of mobile social applications that automatically determine your social context. A key element of such applications is ubiquitous and precise sensing of the people you interact with. Existing techniques that rely on deployed infrastructure to determine proximity are limited in availability and accuracy. Virtual Compass is a peer-based relative positioning system that relies solely on the hardware and operating system support available on commodity mobile handhelds. It uses multiple radios to detect nearby mobile devices and places them in a two-dimensional plane. It uses adaptive scanning and out-of-band coordination to explore trade-offs between energy consumption and the latency in detecting movement. We have implemented Virtual Compass on mobile phones and laptops, and we evaluate it using a sample application that senses social interactions between Facebook friends. 1

Location-based applications have become increasingly popular on smartphones over the past years. The active use of these applications can however cause device battery drain owing to their powerintensive location-sensing operations. This paper presents an adaptive location-sensing framework that significantly improves the energy efficiency of smartphones running location-based applications. The underlying design principles of the proposed framework involve substitution, suppression, piggybacking, and adaptation of applications ’ location-sensing requests to conserve energy. We implement these design principles on Android-based smartphones as a middleware. Our evaluation results show that the design principles reduce the usage of the power-intensive GPS (Global Positioning System) by up to 98 % and improve battery life by up to 75%.

Mobile devices are becoming increasingly sophisticated and the latest generation of smart cell phones now incorporates many diverse and powerful sensors. These sensors include GPS sensors, vision sensors (i.e., cameras), audio sensors (i.e., microphones), light sensors, temperature sensors, direction sensors (i.e., magnetic compasses), and acceleration sensors (i.e., accelerometers). The availability of these sensors in mass-marketed communication devices creates exciting new opportunities for data mining and data mining applications. In this paper we describe and evaluate a system that uses phone-based accelerometers to perform activity recognition, a task which involves identifying the physical activity a user is performing. To implement our system we collected labeled accelerometer data from twenty-nine users as they performed daily activities such as walking, jogging, climbing stairs, sitting, and standing, and then aggregated this time series data into examples that summarize the user activity over 10second intervals. We then used the resulting training data to induce a predictive model for activity recognition. This work is significant because the activity recognition model permits us to gain useful knowledge about the habits of millions of users passively—just by having them carry cell phones in their pockets. Our work has a wide range of applications, including automatic customization of the mobile device’s behavior based upon a user’s activity (e.g., sending calls directly to voicemail if a user is jogging) and generating a daily/weekly activity profile to determine if a user (perhaps an obese child) is performing a healthy amount of exercise.

In this paper, we identify mobile social networks as an important new direction of research in mobile computing, and show how an expanded definition of mobile social networks that includes sensor networks can enable exciting new contextaware applications, such as context-aware video screens, music jukeboxes, and mobile health applications. We offer SocialFusion as a system capable of systematically integrating such diverse mobile, social, and sensing input streams and effectuating the appropriate context-aware output action. We explain some of the major challenges that SocialFusion must overcome. We describe some preliminary results that we have obtained in implementing the SocialFusion vision. 1.