CN108111818A - Moving target active perception method and apparatus based on multiple-camera collaboration - Google Patents

Abstract

The invention provides an active perception device and method for a moving target based on multi-camera coordination. The method includes: calibrating according to the main camera picture and the secondary camera picture, and establishing a position mapping relationship; monitoring the moving target in the main camera picture in real time, and obtaining a candidate target set; selecting candidate targets according to the importance of the target, and selecting the secondary camera according to the position mapping relationship. Track and shoot; calculate the lens azimuth and zoom factor, adjust the camera to align with the candidate target area, and obtain a high-quality image of the candidate target; analyze the candidate target category according to the high-quality image, and confirm the target that belongs to the predetermined setting type as the target of interest, Put it in the focus target collection. The present invention utilizes the positional relationship between the master and slave cameras, mobilizes the slave cameras to obtain high-quality images of candidate targets in the master camera screen, extracts target image features, confirms target categories, and realizes active confirmation of targets.

Description

Active perception method and device for moving targets based on multi-camera collaboration

technical field

The invention relates to an imaging method and device in a multi-camera monitoring system, in particular to a method and device for active target perception based on multi-camera coordination, belonging to the field of video monitoring.

Background technique

Nowadays, all kinds of video surveillance equipment are widely used in production and living environments. An important task of video surveillance is to discover and record the target in the scene, and further, to discover and record key information about the identity of the target, so as to provide help for the subsequent identification of the identity information of the target. The face, license plate, vehicle annual inspection mark, etc. captured by the camera can be used to determine the identity of the target, which is the part that contains the unique description information of the target. The camera captures more unique description information, which can help identify the target identity.

Existing video monitoring equipment collects video images of a monitoring scene and analyzes the video images to achieve the effect of identifying and discovering objects in the scene. However, due to the different video surveillance application scenarios, the positions and attitudes of the targets are various, and the imaging quality of some targets far away from or facing the camera is poor, resulting in poor results for some targets based on the recognition method based on image features. Existing surveillance devices use passive imaging strategies. The equipment shoots the surveillance scene at a fixed position, and cannot actively adjust the camera position and imaging parameters to improve the imaging effect. Passive imaging equipment cannot actively obtain high-quality target images and unique description information of the target, so it cannot effectively identify the target, and there are problems of false positives and missed negatives in the application. It is necessary to design an active perception device with active imaging function to solve the problem of active target confirmation.

A type of bayonet camera system assumes a monitoring device in an area where the target's passing posture is restricted, avoiding the impact of the target's posture on imaging, and solving the problem of obtaining unique description information for the target in the restricted scene. This type of system can obtain high-quality images by setting up cameras in areas such as entrances and exits, supplemented by flashlights, infrared lights and other equipment. The target image captured by this type of system has high resolution and good quality, and it is easy to extract the key information of the target, and the target recognition accuracy is high. However, due to scenario requirements, this type of system can only be deployed in a few places such as toll checkpoints and building entrances and exits, and its application scenarios are limited.

Through camera coordination, the present invention mobilizes the slave cameras to track and capture the target to be confirmed found in the main camera, and confirms the target category according to the captured high-quality target image, which solves the problem while meeting the requirement of active target confirmation. The problem of limited application scenarios of bayonet cameras.

The present invention designs a method for active perception of moving targets based on multi-camera collaboration. In this method, the moving area in the screen is detected from the main camera screen to obtain a set of candidate targets, and the camera linkage relationship is used to mobilize the camera to obtain each candidate target. High-quality images; use the classifier to analyze the high-quality images of the target, extract the features of the target image, analyze the target category, and confirm the target of interest according to the target of the predetermined type according to the classification result.

Contents of the invention

The problem to be solved by the present invention is that, for the candidate target detected by the main camera, the peripheral cameras are mobilized to obtain high-quality images of the candidate target, the features of the target image are extracted, the target category is analyzed, and whether the candidate target is an interested object.

The camera used in the present invention is divided into two types, a panoramic camera fixed relative to the monitoring scene and a PTZ camera with functions of rotation (Pan), pitch (Tilt) and zoom (Zoom).

The invention discloses an active sensing device for a moving target, which comprises a motionless panoramic camera, a plurality of PTZ cameras and the active sensing device for a moving target. Among them, the panoramic surveillance camera is the main camera, which is used to obtain the panoramic surveillance video, and the PTZ camera is the secondary camera, which is used to track and shoot the target and obtain high-quality images of the target. The moving target active sensing device is used to extract the target to be confirmed in the main camera screen, mobilize the secondary camera to take high-quality target images, analyze the target category, and confirm the target that belongs to the predetermined setting type to be confirmed as the target.

The invention discloses a multi-camera coordination-based autonomous perception imaging method for a moving target, which is characterized in that it includes the following steps:

(1) According to the main camera picture and the secondary camera picture, the automatic calibration between the main camera and the secondary camera is carried out through feature extraction and feature matching, and the position mapping relationship is established.

(2) According to the detection threshold setting, real-time detection is performed on multiple moving areas in the field of view of the main camera to obtain a set of candidate targets,

(3) Select the candidate target with the highest importance in the candidate target set according to the importance evaluation function, and select the secondary camera to track and shoot according to the position mapping relationship.

(4) According to the position of the candidate target and the position mapping relationship between the main camera and the slave camera, the slave camera calculates the lens azimuth and zoom factor, adjusts the slave camera to align with the candidate target area, and obtains a high-quality image of the candidate target,

(5) Extract the features of the high-quality image of the target, analyze and confirm the target category, according to the target classification result, confirm the target that belongs to the predetermined setting type as the attention target, put it into the focus target set, and confirm the target that does not belong to the predetermined setting type as the target Non-focused targets are not included in the set of focused targets.

(6) Confirm whether the confirmation of all candidate targets is completed, if yes, exit, otherwise return to step 3.

The method for active perception of moving objects based on multi-camera coordination as described above is characterized in that the step 1 has the following process:

1.1 Select any uncalibrated slave camera.

1.2 Manually adjust the focal length of the slave camera to the minimum value, and adjust the lens direction of the slave camera until the slave camera and the master camera have the maximum overlapping field of view.

1.3 Extract the main camera picture and the speeded-up robust features (Speeded-Up RobustFeatures) from the camera picture respectively.

1.4 Use the k-nearest neighbor (k-Nearest Neighbor) algorithm and the brute force search algorithm to match the accelerated robust (SURF) feature points, and get the matching result GoodMatches.

1.5 The main camera uses the matching result GoodMatches to calculate the affine matrix between the main camera picture and the slave camera picture using the least squares method, establish a position mapping relationship, and complete the master-slave camera calibration.

1.6 Determine whether all the slave cameras have been registered, if yes, return to step 1.1, otherwise exit.

The method for active perception of moving objects based on multi-camera collaboration as described above is characterized in that: in the step 1, the K nearest neighbor algorithm and the brute force search algorithm are used in the feature matching step to match the acceleration of the main camera picture and the secondary camera picture Robust feature points. For each accelerated robust feature point from the camera image, use the K nearest neighbor (KNN) algorithm to search for the three feature points with the closest Euclidean distance in the main camera accelerated robust feature point set, and record the results into the set Matches; calculate all Accelerate the Euclidean distance of the robust feature point pair, record the minimum distance as d, and take all the min(2d,minDist) point pairs with a distance smaller than Matches to form the set GoodMatches, which is the set of matching feature point pairs. Among them, minDist is a preset threshold, which can be adjusted according to the actual situation, but it should be ensured that the number of point pairs in GoodMatches is not less than 15.

The method for active perception of moving objects based on multi-camera collaboration as described above is characterized in that: in the step 1, the position mapping relationship between the master camera and the slave camera includes two parts: the coordinates of the master camera picture and the corresponding relationship between the slave camera; the master camera and from the coordinate transformation relationship of the camera screen,

The frame coordinates of the master camera and the slave camera are represented by a convex hull surrounding the matching feature points in the master camera frame,

According to the matching feature point pairs in GoodMatches, calculate the convex hull that can enclose all the feature points in the main camera picture, and in step C, assign the candidate targets falling in the convex hull to the slave camera.

The coordinate transformation relationship between the main camera and the slave camera screen is represented by affine transformation,

According to the corresponding relationship between the image coordinate positions of the point pairs in the set GoodMatches, the affine transformation from the main camera picture to the secondary camera picture is calculated by the least square method.

The method for active perception of moving objects based on multi-camera coordination as described above is characterized in that: in the step 2, the frame difference method is used to detect the candidate objects in the main camera picture,

Use continuous adaptive mean shift algorithm to track candidate objects in the main camera view,

And the result of the real-time detection of the candidate target has the following form:

[ObjectID, Time, PosX_Left, PosY_Left, PosX_Right, PosY_Right];

in:

ObjectID represents the number of the candidate target,

Time represents the appearance time of the candidate target,

PosX_Left, PosY_Left, PosX_Right, and PosY_Right represent the time series of the coordinates of the upper left corner and the lower right corner of the bounding box, respectively.

The above-mentioned method for active perception of moving objects based on multi-camera coordination is characterized in that in step 3, the following formula is used to describe the importance of the object:

E＝ _Eleave +α× _Ewait

Among them, E _leave is an evaluation function describing the time when the target leaves the screen, and the shorter the time for the target to leave the screen, the greater the value of this function. E _wait is an evaluation function that describes the waiting time of the target in the target queue. The longer the time without being captured, the greater the value of this function; α is an adjustable parameter, and the larger the α, the more attention is paid to the entry order of the target.

The above-mentioned method for active perception of moving objects based on multi-camera coordination is characterized in that in step 3, the time when the object leaves the frame is estimated by the following function:

Among them, w, h is the width and height of the main camera screen, (x, y) is the current position of the target, [x ₀ , y ₀ ] is the position when the target enters the screen, is the estimated velocity of the target. The time calculated by this function represents the time when the target moves in a straight line at a constant speed along the current motion direction in the main camera image to the edge of the image.

The method for active perception of moving objects based on multi-camera collaboration as described above is characterized in that: in step 4, the coordinates of the candidate target in the main camera are converted to the initial position of the slave camera using the coordinate transformation relationship between the master camera and the slave camera screen The relative coordinates on the screen, according to the fisheye spherical projection rule, convert the relative coordinates of the screen from the initial position of the camera to the angular coordinates from the direction of the camera lens. The focal length of the camera is estimated by this method: if the maximum length and width of the given target is l ^* pixel, the focal length when establishing the position mapping matrix from the camera is f, and the width and height of the candidate target in the image of the camera are w, h, then it can be The adjusted focal length is:

Description of drawings

FIG. 1 is a flow chart of a method for active perception of a moving object based on multi-camera coordination according to an embodiment of the present invention.

Fig. 2 is a system configuration diagram of an active sensing device for moving objects based on multi-camera coordination according to an embodiment of the present invention.

FIG. 3 is a flow chart of master-slave camera calibration in a method for active perception of moving objects based on multi-camera coordination according to an embodiment of the present invention.

Detailed ways

The present invention will be described in further detail below in conjunction with the accompanying drawings and specific embodiments.

The system configuration of the device for active perception of moving objects based on multi-camera coordination according to an embodiment of the present invention is shown in FIG. 2 . The device for implementing the multi-camera coordination-based active perception method for moving objects in the present invention includes: at least two cameras for realizing the master-slave camera working mode; and a set of moving object active perception devices. The working mode of the master-slave camera in the present invention specifically refers to mobilizing the slave cameras for active perception of objects found in the master camera to obtain high-quality video images.

The cameras used in the present invention are divided into two categories, one is a fixed panoramic camera corresponding to a monitoring scene, and the other is a PTZ camera with functions of rotation (Pan), pitch (Tilt) and zoom (Zoom). According to a specific embodiment of the present invention, the main camera and the slave camera include a fixed panoramic camera and multiple PTZ cameras; the panoramic camera as the main camera has a larger field of view, and its picture can cover at least most of the monitoring scene area. In a specific embodiment according to the present invention, the main camera is a fixed bolt. In another specific embodiment according to the present invention, the main camera is a PTZ camera with a fixed lens direction.

According to an embodiment of the present invention, the surveillance scene covered by a secondary camera overlaps with the surveillance scene covered by the main camera, but does not overlap with the scenes covered by other secondary cameras. However, in another embodiment of the present invention, the monitoring scene covered by the screen of one secondary camera partially overlaps with the scenes covered by other secondary cameras.

The moving object active sensing device according to the present invention collects video images of a master camera and a plurality of slave cameras, and processes the collected video images.

The moving object active sensing device according to one embodiment of the present invention is set on a personal computer (PC), embedded processing box or board.

In one embodiment of the present invention, the hardware equipped with the device for active sensing of moving objects according to the present invention is integrated into the hardware of the main camera. In yet another embodiment of the present invention, the device for actively sensing a moving object according to the present invention is set on a computer connected to a master camera and a slave camera through a network.

As shown in FIG. 2 , an active sensing device for moving objects according to an embodiment of the present invention includes an image acquisition unit, a candidate object detection unit, an object selection unit, a position mapping unit, and an object tracking confirmation unit.

The image acquisition unit acquires the images of the master camera and the slave camera. The image acquisition unit transmits the image from the main camera to the candidate target detection unit for detection of candidate targets; the image acquisition unit also transmits the image from the slave camera to the target tracking confirmation unit for extracting target features, analyzing and confirming the target category.

The candidate target detection unit receives the image of the main camera from the image acquisition unit, monitors the moving area in the image in real time according to the detection threshold setting, obtains a set of candidate targets, and transmits the set to the target selection unit. According to an embodiment of the present invention, the candidate target detection unit uses the frame difference method to extract the moving area in the main camera picture, and uses the continuous adaptive mean shift algorithm to track the candidate target.

The target selection unit receives the set of candidate targets from the candidate target detection unit, selects the candidate target with the highest importance at the current moment in the candidate target set according to the target importance evaluation function, and sends the selected candidate target to the target position mapping unit.

The position mapping unit accepts the selected candidate target, selects the slave camera according to the coordinate mapping relationship between the master camera and the slave camera, and sends the coordinates of the target candidate in the main camera image to the selected slave camera. Receive the coordinates of the selected target from the camera, calculate the lens angle and focus adjustment, aim at the candidate target for shooting, and send high-quality target images to the target tracking confirmation unit. In addition, the position mapping unit controls the master camera and the slave camera to complete the automatic calibration process and records the position mapping relationship during the start-up process of the moving object active sensing device according to the present invention.

The target tracking confirmation unit accepts the high-quality target image from the selected camera, extracts the image features in the high-quality target image, classifies the image features of the target through the classifier, obtains the target category, and confirms the target belonging to the predetermined setting type as attention Targets, put them into the focus target set, and confirm the targets that do not belong to the predetermined setting type as non-focus targets, and do not put them into the focus target set.

Figure 1 shows an active perception method for moving objects based on multi-camera coordination according to an embodiment of the present invention, including 5 steps:

Establishment of position mapping relationship based on image feature matching;

Obtain a candidate target set based on moving target detection;

Select the candidate target with the highest importance in the candidate target set according to the importance evaluation function, and select the secondary camera for tracking and confirmation according to the position mapping relationship between the primary camera and the secondary camera;

According to the master-slave camera position mapping relationship, the slave camera calculates the lens orientation and focal length adjustment, adjusts the slave camera lens to align with the candidate target orientation, and obtains high-quality images;

Use target high-quality images to extract target features, analyze and confirm target categories.

The above five steps in one embodiment of the present invention will be described in sequence below.

(1) Establishment of position mapping relationship based on image feature matching

As shown in FIG. 3 , according to the method of the present invention, the position mapping relationship based on image feature matching is established, and the master camera and each slave camera are calibrated to establish the position mapping relationship through feature extraction and feature matching. Calibration in the present invention refers to the process of establishing the coordinate mapping of the same object in the main camera picture and the secondary camera picture.

The position mapping relationship between the master camera and the slave cameras in the present invention includes two parts: the corresponding relationship between the coordinates of the master camera picture and the slave camera; the coordinate conversion relationship between the master camera picture and the slave camera picture.

In the present invention, affine transformation is used to describe the coordinate mapping relationship between the master camera picture and the slave camera picture. By extracting the Speeded-Up Robust Features points of the master-slave camera picture, the master-slave camera is calibrated by using the position correspondence relationship of the similar feature points in the master camera picture and the slave camera picture, and the position mapping relationship is established. In an embodiment according to the present invention, the representation form of the mapping relationship between the main camera frame coordinates and the slave camera target position coordinates is an affine transformation matrix.

Affine transformation is a combination of translation and linear mapping functions. In the field of image processing, affine transformation is suitable for describing image translation, rotation, scaling and inversion (mirroring). Let the affine transformation M, M can be expressed by the following formula:

The coordinate relationship between the main camera picture and the same scene in the secondary camera picture can be described by affine transformation. Given the main camera picture and several matching point pairs known from the secondary camera picture, bring them into Equation (1), use the least square method to solve the parameters a ₁ ~ a ₄ , t _x and t _y , then you can get The affine transformation of the two images is the position mapping matrix in the present invention.

In one embodiment of the present invention, the matching point pairs are obtained by extracting acceleration robust feature points from the initial position images of the main camera and the slave camera, and acquiring them in a matching manner. In the process of feature matching, brute force is used to search the similarity between the main camera picture and the secondary camera picture to accelerate the robust feature feature point. Firstly extract the accelerated robust feature points of the main camera picture and the secondary camera picture, for each accelerated robust feature point of the secondary camera picture, use the K nearest neighbor (KNN) algorithm to search for the Euclidean algorithm in the accelerated robust feature point set of the main camera picture The three closest feature points, the results are recorded in the set Matches; calculate the Euclidean distance of all accelerated robust feature point pairs in Matches, record the minimum distance as d, and take all the distances in Matches that are less than min(2d,minDist) The point pairs constitute the set GoodMatches, which is the set of output matching feature point pairs. Among them, minDist is a preset threshold, which can be adjusted according to the actual situation. In an embodiment according to the present invention, the threshold minDist may be set to 1000.

In a specific embodiment according to the present invention, the position mapping unit selects the target to be confirmed from the camera to track and shoot according to the position correspondence. The position correspondence is calculated by matching feature point pairs in GoodMatches. The position mapping unit calculates the convex hull that can contain the feature points of the main camera picture in the set GoodMatches, and assigns the candidate targets falling in the convex hull to the corresponding secondary camera tracking and shooting in the subsequent steps.

(2) Candidate target extraction and candidate target collection

Moving objects are the most concerned objects in video surveillance, therefore, candidate object extraction focuses on moving objects in the scene. Motion regions in the scene are called object latent regions, which may contain passing candidates. In the present invention, the objects obtained through the motion region detection are referred to as candidate objects. All candidate targets constitute a candidate target set. The candidate target set records the time sequence of each candidate target since entering the frame and the coordinates of the bounding box.

In the present invention, the candidate target detection unit uses the frame difference method to obtain candidate targets on the main camera screen, uses the continuous adaptive mean shift (CamShift) algorithm to track the candidate targets, and records the position sequence of the candidate targets into the candidate target set.

In an embodiment according to the present invention, the set of candidate targets includes but is not limited to:

[ObjectID, Time, PosX_Left, PosY_Left, PosX_Right, PosY_Right];

Among them, ObjectID represents the candidate target number, Time represents the time when the candidate target appears, PosX_Left, PosY_Left, PosX_Right, PosY_Right represent the coordinates of the upper left corner and the lower right corner of the bounding box, respectively.

(3) Candidate target selection based on target importance evaluation function

Target importance evaluation is mainly based on the candidate target's position, motion direction, speed and the length of time it has not been perceived after entering the monitoring scene to comprehensively evaluate the target importance. In one embodiment of the present invention, the evaluation principle is that the faster the moving speed, the closer to the edge of the monitoring screen and the moving direction towards the edge, and the longer the unperceived time after entering the scene, the higher the importance of the target. The target selection unit sorts by importance, and selects the target with the highest importance for perception.

According to an embodiment of the present invention, the form of the importance evaluation function is as follows:

E＝ _Eleave +α× _Ewait

Among them, E _leave is an evaluation function describing the time when the target leaves the screen, and the shorter the time for the target to leave the screen, the greater the value of this function. E _wait is an evaluation function that describes the waiting time of the target in the target queue. The longer the time without being captured, the greater the value of the function; α is an adjustable parameter, and the larger the α, the more attention is paid to the entry order of the target.

According to one embodiment of the present invention, the time when the object leaves the frame is estimated by the following function:

Among them, w, h is the width and height of the main camera image, (x, y) is the current position of the target, [x ₀ , y ₀ ] is the position when the target enters the screen, is the estimated velocity of the target. The time calculated by this function indicates the time when the target moves in a straight line at a uniform speed along the current direction of motion in the screen and reaches the border of the monitoring screen.

For the selected candidate target with the highest importance, the position mapping unit selects the slave camera according to the position mapping relationship, and sends the target coordinate sequence recorded by the candidate target set to the slave camera.

(4) Calculation of the control parameters of the slave camera based on the position mapping relationship between the master and slave cameras

According to one embodiment of the present invention, the coordinate conversion between cameras is performed using the position mapping matrix M generated by the initialization step. Let the candidate target center point (x, y) in the main camera be transformed to the relative coordinate process of the initial picture center of the slave camera through the position mapping matrix M between the master and slave cameras, expressed as:

The relative coordinates (x', y') of the center of the initial image of the slave camera are the two-dimensional pixel coordinates of the slave camera image, the camera cannot be adjusted accordingly, and needs to be converted to the azimuth coordinates of the slave camera. The slave camera converts the relative coordinates of the initial position screen into the angular coordinates of the slave camera lens direction according to the fisheye spherical projection rule.

According to an embodiment of the present invention, if the relative coordinates of the target at the center of the initial picture from the camera are (x, y), the width and height of the picture from the camera are (w, h), and the field of view is Camera Angle Adjustment There are the following forms:

In an embodiment according to the present invention, the size conversion of the bounding box (Bounding Box) between the master camera and the slave camera is estimated by using the upper left and lower right vertex coordinates of the coordinate conversion target. The size of the bounding box enclosed by the upper left and lower right vertices after conversion is the estimated size of the target.

In an embodiment according to the present invention, the change of the target size caused by adjusting the focal length (field of view) of the camera is calculated by an inverse proportional function of the focal length. The method and/or device according to an embodiment of the present invention can collect fixed-size target snapshot images during operation, and can estimate the focal length value of the secondary camera under the given snapshot image size requirements. If the maximum length and width of a given target is l ^* pixels, the focal length when establishing the position mapping relationship from the camera is f, and the candidate target is estimated to be (w, h) in size from the camera screen, then the adjusted focal length can be deduced as:

Adjust the camera according to the camera direction and focal length adjustment calculated by the above method, and perform continuous tracking and shooting for a period of time after aiming at the target to obtain high-quality target images.

(5) Use the high-quality image of the target to extract target features, analyze and confirm the target category

In one embodiment according to the present invention, the target tracking confirmation unit receives the high-quality image of the target captured by the camera, extracts the target features, uses a classifier to analyze the target category, and updates the category of the candidate target according to the target classification result. In an embodiment according to the present invention, the type belonging to the predetermined setting type is confirmed as an interested object, and is put into the interested object set, and the object that does not belong to the predetermined setting type is confirmed as a non-interested object, and is not put into the interested object set.

In one embodiment of the present invention, the extracted features are features that can identify the target type, mainly referring to features such as human face, torso, limbs, or the appearance shape of a motor vehicle, wheels, and license plate area. Classes of candidate objects are confirmed by such distinguishing features. The classifier gives the classification result of the target by analyzing the target features in the picture.

In one embodiment of the present invention, each frame of high-quality images captured by the camera is classified, the classification results of each frame are comprehensively counted, and the classification result with the greatest possibility is selected as the target classification result. Due to the limited rotation speed of the camera, the first few frames of video may appear blurred or fail to capture the target, which will have a negative impact on the classification results. In an embodiment according to the present invention, low-quality images can be discarded in object classification to avoid adverse effects on classification results.

The above disclosures are only specific embodiments of the present invention. On the premise of not departing from the scope of the claims of the present invention, those skilled in the art can make various corresponding changes and modifications according to the basic technical concept provided by the present invention.

Claims (13)

1. A moving target active perception method based on multi-camera cooperation is characterized by comprising the following steps:

A) automatically calibrating the master camera and the slave camera by means of feature extraction and feature matching according to the pictures of the master camera and the slave camera, establishing a position mapping relation,

B) detecting a plurality of motion areas in the field of view of the main camera in real time according to the setting of a detection threshold value to obtain a set of candidate targets,

C) selecting the candidate target with the highest importance in the candidate target set according to the importance evaluation function, selecting the corresponding slave camera for tracking shooting according to the position mapping relation,

D) according to the mapping relation between the candidate target position and the position between the master camera and the slave camera, the slave camera calculates the lens azimuth angle and the zoom multiple, adjusts the slave camera to be aligned with the candidate target area, acquires the high-quality image of the candidate target,

E) extracting the characteristics of high-quality images of the targets, analyzing and confirming the target types, confirming the targets belonging to the preset setting types as attention targets according to the target classification results, putting the attention targets into an attention target set, confirming the targets not belonging to the preset setting types as non-attention targets, and not putting the attention target set.

2. The active perception method for moving objects based on multi-camera collaboration as claimed in claim 1, wherein the step a) includes:

A1) any slave camera that is not calibrated is selected,

A2) adjusting the focal length of the slave camera to the minimum value, adjusting the lens direction of the slave camera until the slave camera and the master camera have the maximized overlapped visual field,

A3) the accelerated robust features of the master and slave camera views are extracted separately,

A4) and matching the accelerated robust feature points by using a K-Nearest Neighbor (K-Nearest Neighbor) algorithm and a brute force search algorithm to obtain a matching result GoodMatches.

A5) And (4) calculating an affine matrix of the pictures of the master camera and the slave camera by using a least square method according to a matching result GoodMatches to finish the calibration of the master camera and the slave camera.

A6) And judging whether all the slave cameras are registered, if so, returning to the step A1), and otherwise, exiting. And the moving target active perception method based on multi-camera cooperation further comprises the following steps:

F) and C), confirming whether the confirmation of all the candidate targets is finished, if so, exiting, otherwise, returning to the step C).

3. The active perception method of moving objects based on multi-camera collaboration as claimed in claim 1, wherein:

the characteristic matching operation in the step A) uses a K nearest neighbor algorithm and a brute force search algorithm to match accelerated robust characteristic points in the pictures of the master camera and the slave camera,

for each acceleration robust feature point of the auxiliary camera picture, searching 3 feature points with the shortest Euclidean distance in the acceleration robust feature point set of the main camera by using a K nearest neighbor algorithm, recording the result into a set of Matches,

calculating Euclidean distances of all accelerated robust feature point pairs in a set of Matches, taking the minimum distance as d, and taking all point pairs with the distances smaller than min (2d, minDist) in the set of Matches to form a set of GoodMatches, wherein the set of GoodMatches is a matching feature point pair set, and minDist is a preset threshold value and can be adjusted according to actual conditions, but the number of the point pairs in the set of GoodMatches is not less than 15.

4. The active perception method of moving objects based on multi-camera collaboration as claimed in claim 1, wherein:

in the step a, the position mapping relationship between the master camera and the slave camera includes two parts: the picture coordinates of the master camera and the slave camera correspond to each other; the coordinate transformation relation between the pictures of the master camera and the slave camera,

the master camera view coordinates and the slave camera are represented by a convex hull surrounding the matching feature points in the master camera view,

from the pairs of matching feature points in the set GoodMatches, a convex hull is computed in the master camera view that can enclose all the feature points, and in step C, a candidate object falling in the convex hull is assigned to the slave camera.

The master camera and slave camera view coordinate conversion relationship is represented by affine transformation,

and (3) according to the corresponding relation of the image coordinate positions of the point pairs in the set GoodMatches, calculating affine transformation from the picture of the master camera to the picture of the slave camera by using a least square method.

5. The active perception method of moving objects based on multi-camera collaboration as claimed in claim 1, wherein:

in the step B), a candidate target in the main camera picture is detected by using a frame difference method,

a candidate target in the main camera view is tracked using a continuous adaptive mean shift algorithm,

and the results of the real-time detection of the candidate targets have the following form:

[ObjectID,Time,PosX_Left,PosY_Left,PosX_Right,PosY_Right]；

wherein:

ObjectID indicates the number of candidate objects,

time denotes the Time of occurrence of the candidate object,

PosX _ Left, PosY _ Left, PosX _ Right, PosY _ Right represent the time series of coordinates of the upper Left and lower Right corners of the bounding box, respectively.

6. The active perception method for moving objects based on multi-camera coordination according to claim 1, characterized in that in the step C),

the importance of the target is characterized using the following formula:

E＝E_leave+α×E_wait

wherein,

E_leaveto describe the evaluation function of the time for the target to leave the picture, the shorter the time for the target to leave the picture, the larger the value of the function,

E_waitto describe the evaluation function of the waiting time of the target in the target queue, the longer the uncaptured time is, the larger the value of the function is,

alpha is an adjustable parameter, the larger alpha is, the more attention is paid to the target entering sequence,

the time that the target leaves the frame is characterized by the following function:

wherein,

w and h are the width and height of the main camera image,

(x, y) is the current position of the target,

[x₀,y₀]is the position of the object when it enters the picture,

as an estimate of the velocity of the movement of the object,

the time characterized by the above equation represents the time for the object to move straight to the boundary of the picture at a uniform velocity in the current motion direction in the main camera picture.

7. The active perception method for moving objects based on multi-camera cooperation according to claim 1, wherein in the step D), the slave camera calculates the angular coordinate and the focal length of the lens direction of the slave camera using the position mapping relationship between the master camera and the slave camera generated in the step B),

the slave camera lens direction is calculated as follows:

converting the coordinates of the candidate target on the master camera into the relative coordinates on the initial position picture of the slave camera according to the coordinate mapping shutdown, and then converting the relative coordinates on the initial position picture of the slave camera into the angular coordinates in the lens direction of the slave camera according to the fisheye spherical projection rule

The camera focal length is calculated as follows:

if the maximum length and width of the given target is l × pixels, the focal length when the position mapping relationship is established from the camera is f, and the width and height of the candidate target in the picture of the slave camera is w, h, then the focal length after adjustment can be deduced as:

。

8. an active target sensing apparatus, comprising:

the image acquisition unit is used for acquiring video images of the master camera and the slave camera;

the candidate target detection unit is used for extracting candidate targets from the video image of the main camera to form a candidate target set;

the target selection unit is used for selecting a candidate target with the highest importance at the current moment in the candidate target set;

the position mapping unit is used for establishing a position mapping relation between the master camera and the slave camera, selecting the candidate target selected by shooting of the slave camera and sending the position information of the candidate target to the slave camera;

and a target tracking confirming unit for analyzing the target category according to the target high-quality image shot from the camera, confirming the target belonging to the preset setting type as the attention target, and putting the attention target set.

9. The active target perception device of claim 8, wherein the location mapping unit matches the accelerated robust feature points in the master and slave camera views using a K-nearest neighbor algorithm and a brute force search algorithm,

for each acceleration robust feature point of the auxiliary camera picture, searching 3 feature points with the shortest Euclidean distance in the acceleration robust feature point set of the main camera by using a K nearest neighbor algorithm, recording the result into a set of Matches,

calculating Euclidean distances of all accelerated robust feature point pairs in a set of Matches, taking the minimum distance as d, and taking all point pairs with the distances smaller than min (2d, minDist) in the set of Matches to form a set of GoodMatches, wherein the set of GoodMatches is a matching feature point pair set, and minDist is a preset threshold value and can be adjusted according to actual conditions, but the number of the point pairs in the set of GoodMatches is not less than 15.

10. The active target perception device of claim 8, wherein the position mapping relationship between the master camera and the slave camera in the position mapping unit includes two parts: the picture coordinates of the master camera and the slave camera correspond to each other; the coordinate transformation relation between the pictures of the master camera and the slave camera,

the master camera view coordinates and the slave camera are represented by a convex hull surrounding the matching feature points in the master camera view,

from the pairs of matching feature points in GoodMatches, a convex hull is computed in the master camera view that can enclose all the feature points, and in step C, the candidate targets that fall within the convex hull are assigned to the slave camera.

The master camera and slave camera view coordinate conversion relationship is represented by affine transformation,

and (3) according to the corresponding relation of the image coordinate positions of the point pairs in the set GoodMatches, calculating affine transformation from the picture of the master camera to the picture of the slave camera by using a least square method.

11. The active target perception device of claim 8, wherein the candidate target detection unit detects the candidate target in the main camera view using a frame difference method,

a candidate target in the main camera view is tracked using a continuous adaptive mean shift algorithm,

and the results of the real-time detection of the candidate targets have the following form:

[ObjectID,Time,PosX_Left,PosY_Left,PosX_Right,PosY_Right]；

wherein:

ObjectID indicates the number of candidate objects,

time denotes the Time of occurrence of the candidate object,

PosX _ Left, PosY _ Left, PosX _ Right, PosY _ Right represent the time series of coordinates of the upper Left and lower Right corners of the bounding box, respectively.

12. The active target perception device of claim 8, wherein the target selection unit characterizes the importance of the target using the following formula:

E＝E_leave+α×E_wait

wherein,

E_leaveto describe the evaluation function of the time for the target to leave the picture, the shorter the time for the target to leave the picture, the larger the value of the function,

E_waitto describe the evaluation function of the waiting time of the target in the target queue, the longer the uncaptured time is, the larger the value of the function is,

alpha is an adjustable parameter, the larger alpha is, the more attention is paid to the target entering sequence,

the time that the target leaves the frame is characterized by the following function:

wherein,

w and h are the width and height of the main camera image,

(x, y) is the current position of the target,

[x₀,y₀]is the position of the object when it enters the picture,

as an estimate of the velocity of the movement of the object,

the time characterized by the above equation represents the time for the object to move straight to the boundary of the picture at a uniform velocity in the current motion direction in the main camera picture.

13. The active sensing apparatus of claim 8 or 11, wherein in the target tracking unit, the slave camera calculates the angular coordinate and the focal length of the lens direction of the slave camera using the position mapping relationship between the master camera and the slave camera in the address mapping unit,

the slave camera lens direction is calculated as follows:

converting the coordinates of the candidate target on the master camera into the relative coordinates on the initial position picture of the slave camera according to the coordinate mapping shutdown, and then converting the relative coordinates on the initial position picture of the slave camera into the angular coordinates in the lens direction of the slave camera according to the fisheye spherical projection rule

The camera focal length is calculated as follows:

if the maximum length and width of the given target is l × pixels, the focal length when the position mapping relationship is established from the camera is f, and the width and height of the candidate target in the picture of the slave camera is w, h, then the focal length after adjustment can be deduced as:

。