CN115984966B - Character object interaction action detection method based on feature refining and multiple views - Google Patents

Abstract

The invention provides a character interaction action detection method based on feature refining and multiple views, which comprises the following steps of S1: extracting entity time sequence characteristics of a video frame to be detected, and acquiring the entity time sequence characteristics; s2: refining the entity time sequence characteristics to obtain refined entity time sequence characteristics; s3: based on the refined entity time sequence characteristics, acquiring a human-centered interactive action modeling model; s4: constructing and fusing a plurality of groups of multi-view character interaction action characteristics by adopting a human-centered interaction action modeling model to obtain character interaction action classification characteristics; s5: classifying the character interaction actions through an action classifier. The accuracy of the time sequence feature representation is improved, the robustness of the character interaction motion representation is enhanced, and the problem of character interaction motion detection in the video stream is solved.

Description

A human-character interaction action detection method based on feature refinement and multi-view

Technical field

The invention relates to the field of computer vision technology, and in particular to a method for detecting interactive actions of people and objects based on feature refining and multi-view.

Background technique

The interaction detection task aims to detect and locate interactions between people and objects. Currently, methods for interactive action recognition can be roughly divided into two categories, namely single-frame-based interactive action detection methods and time-series-based interactive action detection methods.

For single-frame based interactive action detection methods, in order to better detect human interactive actions, some existing methods capture visual information or spatial information of people and objects to represent interactive actions. In the graph transfer neural network method, a method that combines graphical models and neural networks is proposed to integrate visual feature information from people and objects. The method iteratively learns the graph structure and infers the message passing weights to output the final parsing graph, which includes the graph structure of people and objects and the labels of interaction actions. In the visual spatial graph network method, a method that combines spatial structure with graph network is proposed to integrate the relative spatial position information between people and objects. This architecture extracts and refines vision from pairs of persons based on their spatial positions. features, and detect human interactions through graph convolutional networks.

However, all these methods do not model the temporal dependence between people and objects, and cannot better understand the interaction between people and objects based on temporal information. Moreover, these methods require action analysis by enumerating all pairs of people and objects, so there is a problem of high computational and reasoning time costs.

For time-series-based interactive action detection methods, in order to better utilize time clues, some existing methods use video time-series features to model character time-series interaction representations, thereby accurately representing interactive actions. In the asynchronous interaction aggregation method, a method is proposed to promote action detection by integrating different interactive behaviors. This method first extracts long-term time series features through an asynchronous memory update algorithm, then models human interaction features, character interaction features and time series features respectively, and finally models interaction features through an interaction aggregation structure. In the advanced relationship modeling method, a method of indirectly modeling high-order interactive relationships is proposed by inferring the interactive relationships between multiple participants and the context. This method first models the first-order relationship between people and context, then builds a high-order relationship inference model, and finally uses the first-order relationship to model the second-order relationship through the inference model.

Although these methods achieve better results than single frame-based interactive action detection methods, their methods of obtaining video features are based on a combination of video clip features and ROI alignment, which will result in fast-moving images. It is impossible to accurately obtain the timing characteristics of actions of a certain nature. Moreover, existing action detection methods cannot accurately and effectively model and characterize human interaction actions, resulting in uninterpretability of the model and poor detection accuracy.

Contents of the invention

In order to solve the above problems, the present invention provides a human interaction action detection method based on feature refinement and multi-view, using YOLO target detection algorithm, ROI alignment algorithm and SlowFast timing feature generation algorithm to capture the timing features of entities, and then using movement The positioning method realizes the tracking and positioning of the entity's motion trajectory, performs feature refining operations, and cascades the temporal features at different time steps based on the positioning results to improve the accuracy of the entity's temporal features and solve the problem of inaccurate temporal features caused by spatial offset. Finally, the action relationship between the person to be detected and multiple entities is explored from multiple views, and the interactive actions of people and objects in the subject view and collaboration view are respectively represented. Multi-view features are constructed by fusing the features in different views, and based on Multiple sets of multi-view features represent human and object interaction actions, solve the problem of character and object interaction action modeling, and enhance the robustness of character and object interaction action representation.

The present invention provides a method for detecting interactive actions of people and objects based on feature refinement and multi-view. The specific technical solutions are as follows:

S1: Extract entity timing features from the video frames to be detected to obtain entity timing features;

S2: Refine the entity timing characteristics and obtain the refined entity timing characteristics; including the following steps:

S201: Track and position the movement trajectory of the entity through mobile positioning operations;

S202: Based on the position of the positioned entity, extract timing features segment by segment and refine the timing features;

S3: Based on refined entity timing characteristics, obtain a human-centered interactive action modeling model;

S4: Use a human-centered interactive action modeling model to construct and fuse multiple sets of multi-view human and character interaction action features to obtain character and character interaction action classification features; including the following steps:

S401: Use a human-centered interactive action modeling model to construct interactive action characteristics of characters in the subject view;

S402: Use a human-centered interactive action modeling model to construct human-character interaction action characteristics under the collaboration view;

S403: Construct character interaction action classification features based on the character interaction action features in the subject view and collaboration view;

S5: Classify human and character interaction actions through action classifiers.

Further, step S1 includes the following steps:

S101: Use the YOLO target detection algorithm to detect the categories and coordinate frames of people and objects in the current frame in real time;

S102: Generate the timing features of the current frame through the SlowFast timing feature extraction algorithm;

S103: Use the ROI alignment algorithm and the maximum pooling algorithm to extract the temporal features of people and objects based on the detected entity coordinates and the temporal features of the current frame;

S104: Store the coordinate frame and temporal features of the entity in the current frame into the feature pool.

Further, in step S201, the mobile positioning operation includes progressive expansion and adaptive positioning. By iteratively executing the two steps of progressive expansion and adaptive positioning, each entity in the current frame is positioned in the previous and subsequent frames. The position of the entity after offset.

Further, the progressive expansion is to copy the positioned entity coordinate frame in the previous iteration to adjacent unpositioned frames.

Further, if it is the first iteration, the coordinate frames of all entities in the current frame are copied to adjacent unpositioned frames.

Further, the specific process of adaptive positioning is as follows:

Obtain the entity coordinate frame stored in the adjacent unpositioned frame from the feature pool, then calculate the entity coordinate frame in the adjacent frame and the center point of the entity coordinate frame copied in the progressive expansion step, and calculate the two sets of coordinate frames distance between center points;

Use Hungarian algorithm to solve the correspondence between center points;

Based on the corresponding relationship, the position of the corresponding stored entity coordinate frame of each copied entity coordinate frame is obtained, and the position is used as the position of the positioned entity.

Further, in step S202, the specific process of refining temporal features is as follows:

Split the temporal features of the current frame into 2D+1 feature blocks, where D represents the total number of iterations;

Use the positioned entity position and ROI alignment method to capture the partition timing features of each entity in each feature block;

All partition timing features of each entity are concatenated, and refined entity timing features are constructed through convolution operations.

Further, the convolution operation consists of two stacked convolution layers, Dropout layers and ReLU layers.

Further, in step S3, the human-centered interactive action modeling model includes a learnable weight block, a long-term temporal feature enhancement block and two atomic action representation blocks;

The atomic action representation block takes the refined timing characteristics of the human body to be detected and the refined timing characteristics of other people/objects participating in the interaction as inputs, and the output of the atomic action representation block is connected to the input of the learnable weight block, so The output of the learnable weight block and the long-term temporal features are used as the input of the long-term temporal feature enhancement block.

Further, in step S401, the specific process of constructing the interactive action characteristics of the characters in the subject view is as follows:

Through the human-centered interactive action modeling method in step S3, the refined timing features of the human body to be detected and the refined timing features of the two objects participating in the interaction are used as input to obtain the interactive action features of people and objects in the subject view.

Further, in step S402, the specific process of constructing the character interaction action characteristics in the collaboration view is as follows:

Through the human-centered interactive action modeling method in step S3, the refined timing features of the human body to be detected, the refined timing features of the human body assisting the interaction, and the refined timing features of the co-operated objects are used as input to obtain the collaborative view. Characteristics of character interaction;

Further, in step S403, the specific process of constructing character interaction action classification features is as follows:

The multi-view character interaction characteristics are obtained by adding the character interaction characteristics in the main view and the character interaction characteristics in the collaboration view;

Construct multiple groups of multi-view character interaction action features;

Add and fuse multiple sets of multi-view human and object interaction action features to obtain the character and object interaction action classification features.

The beneficial effects of the present invention are as follows:

1. After extracting the temporal features of the entity, the temporal features are refined based on mobile positioning, which effectively solves the problem of inaccurate feature description due to spatial displacement in temporal feature extraction and improves the accuracy of the temporal features.

2. A human-centered interactive action modeling model is given. The model is composed of two atomic action representation blocks, a set of learnable weights and long-term temporal feature enhancement blocks. Based on this model, the model is separately analyzed from two different views. Characterize human-character interaction actions. Finally, multi-view features are constructed by merging features under different views, and characterize human-character interaction actions based on multiple sets of multi-view features. This effectively solves the problem that existing methods cannot model human-character interaction features. And based on the multi-view method, the robustness of human-character interaction features is enhanced.

Description of the drawings

Figure 1 is a schematic diagram of the overall process module of the method;

Figure 2 is a schematic structural diagram of the human-centered interactive action modeling model.

Detailed ways

The technical solutions in the embodiments of the present invention are clearly and completely described in the following description. Obviously, the described embodiments are only some of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of the present invention.

In the description of the embodiments of the invention, it should be noted that the indicated orientation or positional relationship is based on the orientation or positional relationship shown in the drawings, or is the orientation or positional relationship that is customarily placed when the product of the invention is used, or is the one in this field. The orientation or positional relationship commonly understood by skilled persons, or the orientation or positional relationship in which the product of the invention is customarily placed when used, is only for the convenience of describing the present invention and simplifying the description, and does not indicate or imply that the device or component referred to must have Specific orientations, construction and operation in specific orientations and therefore are not to be construed as limitations of the invention. In addition, the terms "first" and "second" are only used to differentiate descriptions and cannot be understood as indicating or implying relative importance.

In the description of the embodiments of the present invention, it should also be noted that, unless otherwise clearly stated and limited, the terms "set" and "connection" should be understood in a broad sense. For example, it can be a fixed connection or a detachable connection. , or integrally connected; it can be directly connected or indirectly connected through an intermediary. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood on a case-by-case basis.

Example 1

Embodiment 1 of the present invention discloses a human interaction action detection method based on feature refining and multi-view, as shown in Figure 1, including the following steps:

S1: Extract entity timing features from the video frames to be detected to obtain entity timing features;

In this embodiment, a video to be detected is defined as Among them, v _t represents the t-th frame in the video currently being detected, which is called the current frame, and T is the total number of video frames in the video to be detected.

Specifically, the steps are as follows:

S101: Use the YOLO target detection algorithm to detect the categories and coordinate frames of people and objects in the current frame in real time. The coordinate frames of people and objects are recorded as Bh _t and Bo _t respectively, and the set of Bh _t and Bo _t is recorded as Be _t ;

S102: Generate the timing features of the current frame through the SlowFast timing feature extraction algorithm;

S103: Use the ROI alignment algorithm and the maximum pooling algorithm to extract the temporal features of people and objects based on the detected entity coordinates and the temporal features of the current frame, which are recorded as Fh _t and Fo _t respectively; the set of Fh _t and Fo _t is recorded as _Fet ;

S104: Store the coordinate frame and temporal features of the entity in the current frame into the feature pool. The feature pool is expressed as

S2: Refine the entity timing characteristics and obtain the refined entity timing characteristics;

Since entities often move quickly during interactions, the traditional ROI alignment-based method for extracting temporal features of entities is prone to inaccurate feature extraction due to large offsets in entity movement;

In this embodiment, the accuracy of the entity timing features is improved by refining the entity timing features.

Specific steps are as follows:

S201: Track and position the movement trajectory of the entity through mobile positioning operations;

The purpose of mobile positioning is to locate the offset position of each entity in the key frame in the previous and subsequent frames.

The mobile positioning operation includes progressive expansion and adaptive positioning. In this embodiment, by iteratively executing the two steps of progressive expansion and adaptive positioning, each entity in the current frame is positioned and offset in the previous and subsequent frames. The final entity position. In each iteration, a progressive expansion operation is first performed to provide an entity position reference for displacement positioning, and then an adaptive positioning operation is performed to match the moved entity coordinate frame.

Specifically, the progressive expansion is to copy the entity coordinate frame positioned in the previous iteration to the adjacent unpositioned frames v _t+d*Ns and v _td*Ns . The copied coordinate frame is recorded as Bc _t+d*Ns , Bc _td*Ns , where d represents the current number of iterations, and Ns represents the number of frames within the set displacement positioning interval;

If it is the first iteration, copy the coordinate frames of all entities in the current frame to adjacent unpositioned frames.

The specific process of adaptive positioning is as follows:

Obtain the adjacent unpositioned frames from the feature pool P, such as v _t+d*Ns , the stored entity coordinate frame Be _t+d*Ns , and calculate the stored entity coordinate frame Be _t+d*Ns and The center point of the entity coordinate frame Bc _t+d*Ns copied in the progressive expansion step, and finally the distance between the center points of the two sets of coordinate frames is calculated as follows:

j∈[1,Se],Dis(i,j)=|center(i)-center(j)|;

Among them, Sc and Se are the number of entities in Bc _t+d*Ns and Be _t+d*Ns respectively, center is used to solve the center point of the entity coordinate frame, and Dis(i,j) stores the copied sitting frame Bc _t+ The distance between the i-th coordinate frame in _d*Ns and the j-th coordinate frame in the stored coordinate frame Be _t+d*Ns ;

Then the Hungarian algorithm is used to solve the correspondence between the center points, as follows:

Π ^* =argmin<Π,Dis>

Among them, Π represents the correspondence between center points, Dis is a matrix used to store the distance between center points, Π ^* stores the corresponding results between the center points of the solution;

The calculation of the total number of iterations D in this embodiment is as follows:

D＝[Nc/Ns]/2

Among them, Nc is the number of video frames that need to be input when the SlowFast model extracts the timing features of the current frame. In order to avoid the problem of frame-by-frame offset positioning causing too many iterations and increasing the amount of calculation, in this embodiment, the displacement is performed at intervals of one second. Positioning, that is, Ns represents the number of frames of the video in one second.

Finally, based on the corresponding relationship, each copied coordinate frame Bc _{t+d*Ns is} solved for its corresponding stored entity coordinate frame Be _t+d*Ns , and this position is used as the position of the positioned entity:

Among them, proj is used to project the center point to the corresponding bounding box, Represents the threshold used to constrain the motion displacement distance, and Bt _t+d*Ns stores the mobile positioning results.

S202: Based on the position of the positioned entity, extract timing features segment by segment and refine the timing features;

The specific process is as follows:

Split the temporal features of the current frame into 2D+1 feature blocks, where D represents the total number of iterations;

Use the positioned entity position and ROI alignment method to capture the partition timing features of the entity in each feature block, recorded as Fe;

All partition timing features of each entity are concatenated, and refined entity timing features are constructed through convolution operations.

In this embodiment, the convolution operation consists of two stacked convolution layers, Dropout layers and ReLU layers, which can select and amplify features that are helpful for action recognition from the cascaded features;

The specific processing process is as follows:

Among them, Wr represents the convolution heap operation, Fe is the partition timing feature, and Re represents the refined entity timing feature, including the refined timing features Rh and Ro of people and objects.

S3: Based on refined entity timing characteristics, obtain a human-centered interactive action modeling model.

As shown in Figure 2, the structure of the human-centered interactive action modeling model is as follows:

Includes learnable weight blocks, long-term temporal feature enhancement blocks, and two atomic action representation blocks;

The atomic action representation block takes the refined timing characteristics of the human body to be detected and the refined timing characteristics of other people/objects participating in the interaction as inputs, and the output of the atomic action representation block is connected to the input of the learnable weight block, so The output of the learnable weight block and the long-term temporal features are used as the input of the long-term temporal feature enhancement block.

Specifically, the model includes two atomic action representation blocks, respectively recorded as atomic action representation block 1, atomic action representation block 2, learnable weight block and long-term temporal feature enhancement block;

The atomic action representation block and the long-term temporal feature enhancement block can be implemented in a variety of ways, such as AvgPooling, Transformer and Non-Local Block; because the Non-Local Block can more effectively capture the dependence between features, the selection of the target person's features is highly Activated other features, and they can be combined to enhance the target person features, and it will not consume a lot of computing resources. Therefore, in this embodiment, Non-Local Block is used to extract atomic action features.

Atomic action representation block 1 and atomic action representation block 2 input the refined timing features of the human body to be detected and the refined timing features of other people/objects participating in the interaction, and output the atomic interaction action features;

The learnable weight block is composed of multiple convolution stacks and is used to fuse two atomic interaction features to achieve a combination of atomic interaction actions and output the combined interaction features.

The long-term temporal feature block first extracts the temporal features of all entities with a time span of 5 seconds centered on the current frame from the feature pool P and cascades all entity features within 5 seconds to construct a long-term temporal feature L, and then implements the combined interaction Feature enhancement.

S4: Use a human-centered interactive action modeling model to construct and fuse multiple sets of multi-view human and character interaction action features to obtain character and character interaction action classification features;

As shown in Figure 1, in order to achieve a more robust extraction of interactive action features of people and objects, a human-centered interactive action modeling model is used to extract interactive action features of people and objects in the main view and collaboration view respectively;

Specific steps are as follows:

S401: Use a human-centered interactive action modeling model to construct interactive action characteristics of people and characters in the subject view.

For agent views, primarily models the agents that interact directly. Considering the interaction with two objects respectively during the interaction process, human-character interaction actions can be directly modeled based on the characteristics of the two objects.

In the subject view, the human-object interaction action consists of two atomic actions in which the person to be detected interacts with two objects respectively. Through the human-centered interaction modeling method in step S3, the human-character interaction actions in the subject view are represented as follows:

Isub=Hc_sub(Rh1, Ro1, Ro2, L)

Among them, Hc_sub represents the human-centered interactive action modeling function, Rh1 represents the refined features of the human body to be detected, Ro1 and Ro2 respectively represent the refined features of the two objects interacting, L represents the long-term time series features, and Isub represents the output subject view. Characteristics of character interaction below.

S402: Use a human-centered interactive action modeling model to construct human-character interaction action characteristics under the collaboration view.

For the collaborative view, the human body assisting the interaction and the objects operating together are mainly modeled. Considering that there are other people collaborating during the interaction process, human-character interaction actions can be modeled based on the characteristics of the collaborators and the characteristics of the objects being operated together.

In the collaboration view, the person-object interaction action consists of two atomic actions in which the person to be detected interacts with the collaborator and the co-operated object respectively. Through the human-centered interaction modeling method in step S3, the human-character interaction actions in the collaboration view are represented as follows:

Icol=Hc_col(Rh1, Ro1, Rh2, L)

Among them, Hc_col represents the human-centered interactive action modeling function, Rh1 and Rh2 represent the refined characteristics of the two human bodies participating in the interaction and assisting the interaction, Ro1 represents the refined characteristics of an object operating together, L represents the long-term time series characteristics, Isub Represents the interactive action characteristics of people and objects in the output collaboration view.

S403: Construct character interaction action classification features based on the subject view and collaboration view;

By adding the character-character interaction action features in the subject view and the character-character interaction action features in the collaboration view, the multi-view character-character interaction action features are obtained. And construct multiple sets of multi-view human and object interaction action features, and obtain the character and object interaction action classification features by adding and fusing multiple sets of multi-view human and object interaction action features.

The specific processing process is as follows:

Among them, g is the number of groups of multi-view human-character interaction action features, and F _HOO is the character-character interaction action classification feature.

S5: Classify human and character interaction actions through action classifiers,

Since the refined human body temporal features contain rich interactive semantic information and are suitable for the recognition of interactive actions, in this embodiment, the human-character interactive actions are classified by adding the refined human body temporal features and the human-character interactive action classification features.

The specific expression is as follows:

P＝Wc(F _HOO +Rh)

Among them, Wc represents an action classifier composed of two fully connected layers and a softmax classifier, F _HOO represents human-character interaction action classification features, and Rh represents refined human body timing features.

The present invention is not limited to the specific embodiments described above. The invention extends to any new features or any new combinations disclosed in this specification, as well as to any new method or process steps disclosed or any new combinations.

Claims (8)

1. A human-object interaction action detection method based on feature refining and multi-view, which is characterized by: S1: Extract entity timing features from the video frames to be detected to obtain entity timing features; S2: Refining the entity timing characteristics and obtaining the refined entity timing characteristics; including the following steps: S201: Track and position the movement trajectory of the entity through mobile positioning operations; S202: Based on the position of the positioned entity, extract timing features segment by segment and refine the timing features; The specific process of timing feature refining is as follows: Split the temporal features of the current frame into 2D+1 feature blocks, where D represents the total number of iterations; Use the positioned entity position and ROI alignment method to capture the partition timing features of each entity in each feature block; Concatenate all partition timing features of each entity and construct refined entity timing features through convolution operations; The convolution operation consists of two stacked convolution layers, Dropout layers and ReLU layers; S3: Based on refined entity timing features, build a human-centered interactive action modeling model. The model includes a learnable weight block, a long-term timing feature enhancement block, and two atomic action representation blocks; The atomic action representation block takes the refined timing characteristics of the human body to be detected and the refined timing characteristics of other people/objects participating in the interaction as inputs, and the output of the atomic action representation block is connected to the input of the learnable weight block, so The output of the learnable weight block and the long-term temporal features are used as the input of the long-term temporal feature enhancement block; S4: Use a human-centered interactive action modeling model to construct and fuse multiple sets of multi-view human and character interaction action features to obtain character and character interaction action classification features; including the following steps: S401: Use a human-centered interactive action modeling model to construct interactive action characteristics of characters in the subject view; S402: Use a human-centered interactive action modeling model to construct human-character interaction action characteristics under the collaboration view; S403: Construct character interaction action classification features based on the character interaction action features in the subject view and collaboration view; S5: Classify human and character interaction actions through action classifiers. 2. The human interaction action detection method according to claim 1, characterized in that step S1 includes the following steps: S101: Use the YOLO target detection algorithm to detect the categories and coordinate frames of people and objects in the current frame in real time; S102: Generate the timing features of the current frame through the SlowFast timing feature extraction algorithm; S103: Use the ROI alignment algorithm and the maximum pooling algorithm to extract the temporal features of people and objects based on the detected entity coordinates and the temporal features of the current frame; S104: Store the coordinate frame and temporal features of the entity in the current frame into the feature pool. 3. The human-character interaction action detection method according to claim 1, characterized in that, in step S201, the mobile positioning operation includes progressive expansion and adaptive positioning, and both progressive expansion and adaptive positioning are performed iteratively. This step locates the position of each entity in the current frame after the offset in the previous and subsequent frames. 4. The human-character interaction action detection method according to claim 3, characterized in that the progressive expansion is to copy the entity coordinate frame positioned in the previous iteration to an adjacent unpositioned frame; If it is the first iteration, copy the coordinate frames of all entities in the current frame to adjacent unpositioned frames. 5. The human interaction action detection method according to claim 3, characterized in that the adaptive positioning, the specific process is as follows: Obtain the entity coordinate frame stored in the adjacent unpositioned frame from the feature pool, then calculate the entity coordinate frame in the adjacent frame and the center point of the entity coordinate frame copied in the progressive expansion step, and calculate the two sets of coordinate frames distance between center points; Use Hungarian algorithm to solve the correspondence between center points; Based on the corresponding relationship, the position of the corresponding stored entity coordinate frame of each copied entity coordinate frame is obtained, and the position is used as the position of the positioned entity. 6. The human-character interaction action detection method according to claim 1, characterized in that, in step S401, the specific process of constructing the character-character interaction action characteristics under the subject view is as follows: Through the human-centered interactive action modeling method in step S3, the refined timing features of the human body to be detected and the refined timing features of the two objects participating in the interaction are used as input to obtain the interactive action features of people and objects in the subject view. 7. The human-character interaction action detection method according to claim 1, characterized in that, in step S402, the specific process of constructing the character-character interaction action characteristics under the collaboration view is as follows: Through the human-centered interactive action modeling method in step S3, the refined timing features of the human body to be detected, the refined timing features of the human body assisting the interaction, and the refined timing features of the co-operated objects are used as input to obtain the collaborative view. Characteristics of character interaction. 8. The human-character interaction action detection method according to claim 1, characterized in that, in step S403, the specific process of constructing the character-character interaction action classification feature is as follows: Add the character and object interaction action features in the subject view and the character and object interaction action features in the collaboration view to obtain the multi-view character and object interaction action features; Construct multiple groups of multi-view character interaction action features; Add and fuse multiple sets of multi-view human and object interaction action features to obtain the character and object interaction action classification features.