In this dissertation, we address the problem of understanding human activities in videos by developing a two-pronged approach: coarse level modeling of scene activities and fine level modeling of individual activities. At the coarse level, where the resolution of the video is low, we rely on person tracks. At the fine level, richer features are available to identify different parts of the human body, therefore we rely on the body joint tracks. There are three main goals of this dissertation: identifying unusual activities at the coarse level, recognizing different activities at the fine level, and predicting the behavior in order to synthesize activities at the fine level. The summary of the three proposed solutions is presented in the following. The first goal is addressed by modeling activities at the coarse level through two novel and complementing approaches. For this purpose, we rely on the tracks of all the moving objects in the scene observed by a static camera. First approach learns the behavior of individuals by modeling the patterns of motion and size of objects in a compact model. The proposed method provides a higher-level process to the traditional real-time surveillance pipeline for identifying unusual activities and feeding back the learned scene model to improve object detection. Pixel

user with raw sensor data from the physical world, we introduce visualization layers to abstract the internals of monitoring algorithms and provide a clean consumable computational output. Canvas provides a flexible backbone that lets us improve vision algorithms while providing a seamless visualization interface. This ultimately improves the effectiveness of the monitoring by focusing attention and presenting only the most relevant information. The visualization is built on Web technology to make the information available anywhere, anytime.