Activity recognition Behavior Recognition

Summer vacation listened to computer vision of a summer School, inside Jason J. Corso said they use the Low-mid-high hierarchy for video understanding and Activity Recognition's approach has benefited quite a bit, and here's a summary of his approach:

-------------------------------------------------------------------------------------------------

1. The hierarchy represents:

• Bottom part Reuse
• Each object is a mixed graph connected by a forward and a no-edge
• The bottom layer allows atomic nodes to form space-time lines, planes, and regions through nonlinear learning.

People's activities, is these objects in the middle and high-level connection of the mixed map

-------------------------------------------------------------------------------------------------

2. Motion perception--sts

Different action stimulate Different subpopulation of cells.

-------------------------------------------------------------------------------------------------

3. Activity Recognition

Corso's Method:

• Low-level: The most effective approach to the bottom is the bag of Features, characterized by the temporal and spatial characteristics of the bottom-up/low level, constantly updated with time and levels. The underlying object is detected by the template;
• Mid-level: The middle layer detects and tracks 2D skeleton pose from the images, and analyzes the dynamic pose through the background content;
• High-level: The high-level activity composition method is to combine the feature of different time points into a time-probability model. Temporal and spatial tracking of feature is carried out on time, and the probability model is based on the composition grammar.
• Another way to express recognition: segmentation

Thought: Build space-time Patch descriptors, compose visual words histogram, build multi-channel classifier.

Find Shikongharris Corner points:

Requires densely sample on feature rather than sparse sample.

Extract action Feature:f, described with Hog/hof

Space-Time patch descriptors form histogram. Each histogram is a histogram of the feature points on the three components of a x,y,t.

However, there are problems in using hog and Hof, which can only be seen from the front and back frames, but not the characteristic changes of the whole ball. Out of this idea, put forward the hog3d, this feature in the BMVC08 has a specific description of the article, not to repeat here, we are interested to read the article it.

-------------------------------------------------------------------------------------------------

4. Behavioral Trajectory

The tracking of feature points using Klt:kanade-lucas-tomasi Feature Tracker can be used as auxiliary means for local feature detection.

• Trajectories by Local KeyPoint Tracking
• Use dense trajectory (dense sampling can improve object recognition and action recognition)
• CVPR Wang et al. "Action recognition by dense trajectories." A method is proposed to track the trajectory with a single density optical flow field.
• Description of locus points using Hog/hof/mbh

-------------------------------------------------------------------------------------------------

5. Action Bank

The basic idea of Action Bank is shown in the figure <CVPR 2012:action bank:a High-level representation of Activity in video>

Action Bank:

• A large action detector set recorded in different scale and viewpoints
• On NS scales, there are NA detector on the action bank, and 1^3+2^3+4^3=73 vector on each group of Action-scale. So the action bank feature vector has a na*ns*73 dimension (about 73 How to get a detailed reference to the Beyond Bags of features:spatial Pyramid Matching for recognizing Natural Sce NE Categories ")
• When actually doing it, use a detector on 1-2 scale.

In the two-dimensional image recognition problem, the bin statistic method of object Bank, the action bank in the video is made into three-dimensional statistics (plus one-dimensional time)

• Classifier using the SVM classifier:

e.g. for playing basketball this action, according to the high-level performance, will jumping, throwing, running joins the positive sample, will biking,fencing,drumming joins the negative sample, uses the SVM to play the basketball two class classification training. PS: The authors used l1-regularization and random forests without significant improvement.

• To achieve the creation of the Action Bank:

1. Select the 6 action on the 50 action,kth on UCF and the digging action on the visint.org to form the 205 templates totally

2. Each action selects 3-6 different views, style or examples of the movement rhythm

3. Average resolution: 50x120pixel;40-50frames/example

To put it simply, a templates is described in terms of different viewing angles, style, and movement rhythm, which makes up 205 templates and describes 57 action.

• About templates

1. As long as a new viewpoint/movement Rhythm/style is found, select the template 2. Each template is manually cropped to a space-time range with the size of the Human Action Action Bank classification result:

-------------------------------------------------------------------------------------------------

6. Activity identification based on the human pose method

• By dividing the person into different parts, the posture estimation of each part can be clearly described by the model.

• 3D Human Pose Estimation:

-------------------------------------------------------------------------------------------------

7. Parts-based human Pose estimation

The model combines local appearance and symmetric structures, and there are several articles involving model estimation:

Pictorial structures (Fischler & Elschlager, Felzenswalb and Huttenlocher 00)
Cardboard people (Yu et al 96)
Body plans (Forsyth & Fleck 97)
Active appearance Models (Cootes & Taylor 98)
Constellation Models (Burl et all 98, Fergus et al 03)

Using the deformable part model

Slide credit:d. Ramanan. Method is from Yang and Ramanan, CVPR 2011.

Result:

• Dynamic Pose based Activity recognition

1. For skeletal pose, we construct a K-means codebook of $ visual words from 24-dimensional skeletal pose data using Eu Clidean distance. For the dynamic pose, we construct codebook using our specific distance function.
2. For classification we use many One-versus-one histogram intersection kernel SVMs.
3. Handling dynamic pose and global context are in motion, with hog3d and dense trajectory can get better effect.

-------------------------------------------------------------------------------------------------

9. Video segmentation: Beyond Points for Video understanding

Standard method:

• Meanshift
• Graph-based
• Hierarchical graph-based
• Nystrom normalized cuts
• Segmentation by Weighted Aggregation

ECCV Xu, Xiong and Corso method: The video as a stream, based on the direction of Markov hypothesis and time to establish an estimation framework, video segmentation.

Segmentation:s = {S1, S2, ..., Sm}

Input video:v = {V1, V2, ..., Vm} (input stream on time series)

Take S*=arg min E (s| V

In the split of a layer, take:

Take the same assumption throughout the hierarchy:

<ECCV Xu, Xiong and corso>

from:http://blog.csdn.net/abcjennifer/article/details/7976145

Activity recognition Behavior Recognition