Citation Context

...arity, agglomerative clustering algorithms [10] is then applied to merge face tracks into face groups. Finally, the character relationship graph is formed by linking consecutively appearing face groups. 2 Face and Face Tracks A face track is defined as a group of spatially and temporally consecutive faces which contains variant head motions and illumination changes. Most methods align faces to its frontal view for matching. However, face alignment is time consuming. Moreover, non-frontal faces can also provide face descriptions for identifying characters. To obtain face tracks, face detection [11] is performed to find the first appearing face from the video. Once a face is detected in the t-th frame, we start the face tracking procedure using the position prior to reduce face detection time. The position prior defines ROI in frame t+1 of the detected face in frame t. If a face ft that appears in frame t overlays a face ft+1 in frame t + 1, ft+1 is considered as a candidate tracked face of ft. The appearance prior is proposed to verify if the tracked face belongs to the same person. If a candidate face ft+1 is visually similar to ft, it is then confirmed to be the following face of ft. ...

Citation Context

...esent these unaligned faces. One of the major problems to define the similarity between two face tracks is that the lengths of two face tracks are usually different. Although closest face pairs [6] and earth mover’s distance (EMD) [2] of face track distributions were proposed to solve the problem, these methods are still hard to model the real distributions of face tracks and lead false matching results. We propose a new face track similarity assessment based on the concept of principle angles [9] to solve the length problem. With the face track similarity, agglomerative clustering algorithms [10] is then applied to merge face tracks into face groups. Finally, the character relationship graph is formed by linking consecutively appearing face groups. 2 Face and Face Tracks A face track is defined as a group of spatially and temporally consecutive faces which contains variant head motions and illumination changes. Most methods align faces to its frontal view for matching. However, face alignment is time consuming. Moreover, non-frontal faces can also provide face descriptions for identifying characters. To obtain face tracks, face detection [11] is performed to find the first appearing f...

Citation Context

...s and scenes are considered as effective preprocessing for analyzing character relationships in movies. To identify the same characters in different shots, face detection is performed for each shot. Then, face recognition based [1][3] and face clustering based [2] methods are employed to identify repeatedly appearing characters. Finally, character relationships are obtained by using temporal scene constraints. Detailed reviews of character relationship analysis methods can be found in [5]. Besides using face information for character relationship analysis, some researchers combine transcripts [6], and cast lists [7] with face information to identify people. As mentioned above, most of the state-of-the-art methods retrieve character relationships based on temporal video segmentation and face recognition results. However, current scene detection methods such as [8] hardly perform promising results on different styles of movies. Moreover, because of variant head motions and scene illumination conditions in shots, the recognized characters are not accurate enough for character relationship analysis. In practice, when watching movies, the audience can understand character relationships wit...

Citation Context

...rithms to obtain face tracks appearing in the video. Contrastbased face features are employed to represent these unaligned faces. One of the major problems to define the similarity between two face tracks is that the lengths of two face tracks are usually different. Although closest face pairs [6] and earth mover’s distance (EMD) [2] of face track distributions were proposed to solve the problem, these methods are still hard to model the real distributions of face tracks and lead false matching results. We propose a new face track similarity assessment based on the concept of principle angles [9] to solve the length problem. With the face track similarity, agglomerative clustering algorithms [10] is then applied to merge face tracks into face groups. Finally, the character relationship graph is formed by linking consecutively appearing face groups. 2 Face and Face Tracks A face track is defined as a group of spatially and temporally consecutive faces which contains variant head motions and illumination changes. Most methods align faces to its frontal view for matching. However, face alignment is time consuming. Moreover, non-frontal faces can also provide face descriptions for identif...

Citation Context

... are employed to identify repeatedly appearing characters. Finally, character relationships are obtained by using temporal scene constraints. Detailed reviews of character relationship analysis methods can be found in [5]. Besides using face information for character relationship analysis, some researchers combine transcripts [6], and cast lists [7] with face information to identify people. As mentioned above, most of the state-of-the-art methods retrieve character relationships based on temporal video segmentation and face recognition results. However, current scene detection methods such as [8] hardly perform promising results on different styles of movies. Moreover, because of variant head motions and scene illumination conditions in shots, the recognized characters are not accurate enough for character relationship analysis. In practice, when watching movies, the audience can understand character relationships without knowing any temporal video structures. The interactions among characters (appear consecutively or in the same frame) provide audience sufficient evidence to interpret character relationships in the movie. Thus, we propose a novel bottom-up interpretation of videos as...

Citation Context

...he same character under variant head motions and scene illuminations. A novel measurement is developed to assess the similarity between two face tracks with different lengths. Then, based on temporal constraints, the character relationship graph of a movie is built. As shown in the experiments, our method can successfully retrieve character relationships from movies in real-time without any prior information or training. 1 Introduction Exploring character relationships from videos provides a cognitive way for content understanding and indexing. To achieve the goal, state-of-the-art methods [1][2][3] usually analyze temporal structures of videos, i.e. shots and scenes [4], at first. In general, a shot indicates continuous presence of characters and a scene contains interactions of characters in a relatively larger social group. Therefore, extracting shots and scenes are considered as effective preprocessing for analyzing character relationships in movies. To identify the same characters in different shots, face detection is performed for each shot. Then, face recognition based [1][3] and face clustering based [2] methods are employed to identify repeatedly appearing characters. Finally...

Citation Context

...same character under variant head motions and scene illuminations. A novel measurement is developed to assess the similarity between two face tracks with different lengths. Then, based on temporal constraints, the character relationship graph of a movie is built. As shown in the experiments, our method can successfully retrieve character relationships from movies in real-time without any prior information or training. 1 Introduction Exploring character relationships from videos provides a cognitive way for content understanding and indexing. To achieve the goal, state-of-the-art methods [1][2][3] usually analyze temporal structures of videos, i.e. shots and scenes [4], at first. In general, a shot indicates continuous presence of characters and a scene contains interactions of characters in a relatively larger social group. Therefore, extracting shots and scenes are considered as effective preprocessing for analyzing character relationships in movies. To identify the same characters in different shots, face detection is performed for each shot. Then, face recognition based [1][3] and face clustering based [2] methods are employed to identify repeatedly appearing characters. Finally, c...

Citation Context

...f the same character under variant head motions and scene illuminations. A novel measurement is developed to assess the similarity between two face tracks with different lengths. Then, based on temporal constraints, the character relationship graph of a movie is built. As shown in the experiments, our method can successfully retrieve character relationships from movies in real-time without any prior information or training. 1 Introduction Exploring character relationships from videos provides a cognitive way for content understanding and indexing. To achieve the goal, state-of-the-art methods [1][2][3] usually analyze temporal structures of videos, i.e. shots and scenes [4], at first. In general, a shot indicates continuous presence of characters and a scene contains interactions of characters in a relatively larger social group. Therefore, extracting shots and scenes are considered as effective preprocessing for analyzing character relationships in movies. To identify the same characters in different shots, face detection is performed for each shot. Then, face recognition based [1][3] and face clustering based [2] methods are employed to identify repeatedly appearing characters. Fina...

Citation Context

...l measurement is developed to assess the similarity between two face tracks with different lengths. Then, based on temporal constraints, the character relationship graph of a movie is built. As shown in the experiments, our method can successfully retrieve character relationships from movies in real-time without any prior information or training. 1 Introduction Exploring character relationships from videos provides a cognitive way for content understanding and indexing. To achieve the goal, state-of-the-art methods [1][2][3] usually analyze temporal structures of videos, i.e. shots and scenes [4], at first. In general, a shot indicates continuous presence of characters and a scene contains interactions of characters in a relatively larger social group. Therefore, extracting shots and scenes are considered as effective preprocessing for analyzing character relationships in movies. To identify the same characters in different shots, face detection is performed for each shot. Then, face recognition based [1][3] and face clustering based [2] methods are employed to identify repeatedly appearing characters. Finally, character relationships are obtained by using temporal scene constraints. ...

Citation Context

...nd a scene contains interactions of characters in a relatively larger social group. Therefore, extracting shots and scenes are considered as effective preprocessing for analyzing character relationships in movies. To identify the same characters in different shots, face detection is performed for each shot. Then, face recognition based [1][3] and face clustering based [2] methods are employed to identify repeatedly appearing characters. Finally, character relationships are obtained by using temporal scene constraints. Detailed reviews of character relationship analysis methods can be found in [5]. Besides using face information for character relationship analysis, some researchers combine transcripts [6], and cast lists [7] with face information to identify people. As mentioned above, most of the state-of-the-art methods retrieve character relationships based on temporal video segmentation and face recognition results. However, current scene detection methods such as [8] hardly perform promising results on different styles of movies. Moreover, because of variant head motions and scene illumination conditions in shots, the recognized characters are not accurate enough for character rel...

Citation Context

...sidered as effective preprocessing for analyzing character relationships in movies. To identify the same characters in different shots, face detection is performed for each shot. Then, face recognition based [1][3] and face clustering based [2] methods are employed to identify repeatedly appearing characters. Finally, character relationships are obtained by using temporal scene constraints. Detailed reviews of character relationship analysis methods can be found in [5]. Besides using face information for character relationship analysis, some researchers combine transcripts [6], and cast lists [7] with face information to identify people. As mentioned above, most of the state-of-the-art methods retrieve character relationships based on temporal video segmentation and face recognition results. However, current scene detection methods such as [8] hardly perform promising results on different styles of movies. Moreover, because of variant head motions and scene illumination conditions in shots, the recognized characters are not accurate enough for character relationship analysis. In practice, when watching movies, the audience can understand character relationships without knowing any tem...

Citation Context

...+1 is considered as a candidate tracked face of ft. The appearance prior is proposed to verify if the tracked face belongs to the same person. If a candidate face ft+1 is visually similar to ft, it is then confirmed to be the following face of ft. In general, appearances of faces in different shots of a person vary a lot with different orientations, lightings, and facial expressions. Fortunately, a video contains continuous motions of people. The appearance changes of the faces of a person between adjacent frames will be relatively minor. Here, we use modified contrast context histogram (CCH) [12] descriptors to efficiently represent the appearance prior of face regions. For each detected face ft, an oval mask is firstly applied to the face region. The masked face region is normalized to zero-mean and unit variance. Seven reference keypoints [K1, . . . ,Kp, . . . ,K7] and the surrounding region qKp for each Kp are used to represent MVA2013 IAPR International Conference on Machine Vision Applications, May 20-23, 2013, Kyoto, JAPAN14-18 431 (a) (b) (c) (d) (e) (f) (g) (h) Figure 1. Seven sampled keypoints (red dots) and their surrounding face regions (black rectangles). (a) the original ...