JP2019096062A - Object tracking device, object tracking method, and object tracking program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To accurately associate objects tracked by each camera in object tracking using a plurality of cameras. An object tracking device has a first object tracking unit that tracks a first object by detecting a first object image, and a second object tracking unit that tracks a second object by detecting a second object image. The object tracking unit, the second object image selection unit that selects the second object image used for determining the same object from the group of the second object images obtained by tracking the second object, and the tracking of the first object. A first object image selection unit that selects a first object image having characteristics similar to those of the selected second object image from the group of the first object images obtained in the above, and the selected first object image. Using the object image and the selected second object image, the same object determination unit that determines whether the first object and the second object are the same object, and the first object and the second object by the same object determination unit. When it is determined that the objects are the same, the association processing unit that performs the processing of associating the first object with the second object is provided. [Selection diagram] Fig. 1

Description

  The present invention relates to a technique for tracking an object using a plurality of cameras.

  As one of object tracking, there is a technology in which a plurality of cameras installed at different places capture an object in a range of the carrier and track the object using an image captured by each camera. According to this, it is possible to track an object at a long distance. In this object tracking, it is necessary to associate the tracked objects with each camera. As a premise of this correspondence, it is determined whether the objects tracked using each camera are the same object or not.

  As a determination technique of the same object (for example, the same person), for example, Patent Document 1 detects a target person from an input image, obtains an image area of the detected target person, and calculates the moving direction of the target person. A detection unit, a storage unit storing information indicating the correspondence between the image feature and the direction of the person, and a correspondence table of the moving direction of the object person with respect to the optical axis of the camera and the observable range. An observation area range is selected according to the movement direction, and the extracted partial area features are divided to create a feature table, and an image feature extraction unit that stores the partial area features in the observation area range and two features to be compared Disclosed is an image processing apparatus comprising: a feature collating unit which finds a range of partial regions on which a table is superimposed, and collates only the features of the partial regions to be superimposed. doing.

JP, 2015-2547, A

  Even if the object imaged by the camera is the same object, the shape of the image of the object (hereinafter referred to as an object image) differs depending on the position of the object relative to the arrangement position of the camera. In particular, when the angle of view of the camera is large, the shape of the object image is largely different. For example, the shapes of the object images of the central portion, the upper end portion, the lower end portion, the left end portion, and the right end portion of the field of view of the camera greatly differ even if they are the same object. If the inventor uses the object image immediately before the object comes out of the field of view of one camera and the object image immediately after the object enters the field of view of the next camera, it is determined whether or not they are the same object. Although they are the same object, they are determined not to be the same object, and as a result, it has been found that the objects can not be accurately associated with each other.

  An object of the present invention is to provide an object tracking device, an object tracking method, and an object tracking program, which can accurately associate objects tracked with each camera in object tracking using a plurality of cameras. is there.

  An object tracking device according to a first aspect of the present invention is an object tracking device for tracking an object using a plurality of cameras, and an image captured by a first camera which is one of the plurality of cameras is time-series An acquisition unit configured to acquire a first time-series image aligned in time, and a second time-series image aligned in time series, an image captured by a second camera that is another one of the plurality of cameras; A first object tracking unit that tracks the first object by detecting a first object image that is an image from the first time-series image, and a second object image that is an image of a second object at the second time A second object tracking unit for tracking the second object by detecting from the sequence image, and a group of the second object images obtained by tracking the second object by the second object tracking unit according to a predetermined standard Second, which is used to determine the same object from among A second object image selection unit for selecting a body image; and a second object image selection unit selected from a group of the first object images obtained by tracking of the first object by the first object tracking unit. A first object image selecting unit for selecting the first object image having a feature similar to the feature of the second object image; and the first object image and the second selected by the first object image selecting unit An identical object determination unit that determines whether the first object and the second object are the same object using the second object image selected by the object image selection unit; and the same object determination unit And an association processing unit configured to associate the first object with the second object when it is determined that the first object and the second object are the same object.

  An object to be tracked (first object, second object) may move from the field of view of the first camera to the field of view of the second camera, or move from the field of view of the second camera to the field of view of the first camera. May be The field of view of the first camera and the field of view of the second camera may not overlap or may partially overlap.

  The second object image selection unit selects a second object image to be used for determination of the same object in accordance with a predetermined criterion. For example, the second object image selection unit selects a second object image included in an image (frame) in which the second object image first appears among images (frames) constituting the second time-series image.

  The first object image selecting unit selects a first object image having a feature similar to the feature of the second object image selected by the second object image selecting unit. The characteristic similar judgment elements are as follows.

(1) The movement direction of the first object image (the direction of the movement line) and the movement direction of the second object image (the direction of the movement line) are the same.
(2) The position on the image (frame) of the first object image and the position on the image (frame) of the second object image are the same.
(3) The moving speed of the first object image (first object) is the same as the moving speed of the second object image (second object).
(4) The orientation of the first object image is the same as the orientation of the second object image.

  The same here may be completely the same or almost the same. If the first object image and the second object image are the same for any one of (1) to (4) or a combination of two or more of them, the first object image selection unit performs the first It is determined that the feature of the object image and the feature of the second object image are similar. For example, if (1) and (2) are satisfied, the features of the first object image and the features of the second object image are similar.

  The first object image selecting unit selects the first object image from among the plurality of first object images according to a predetermined criterion if there are a plurality of first object images having features similar to the features of the second object image. . For example, the first object image selecting unit is included in an image (frame) captured at a time closest to a time when an image (frame) including the second object image selected by the second object image selecting unit is captured. The first object image to be selected is selected.

  The same object determination unit determines whether the first object and the second object are the same object, using the second object image and the first object image having a feature similar to the feature of the second object image. Therefore, the determination accuracy of the same object can be improved. Therefore, according to the object tracking device according to the first aspect of the present invention, in object tracking using a plurality of cameras, objects tracked using each camera can be accurately associated with each other.

  In the above configuration, the association processing unit performs processing of associating one first object among a plurality of first objects with respect to one second object, and the object tracking device is configured to The first object having the possibility of being the same object as the second object among the plurality of first objects based on the relationship between the position of the second object and the positions of the plurality of first objects Further includes a first object determining unit that determines the first object image selecting unit, for selecting the first object image for the first object determined by the first object determining unit. .

  A case is considered where the first object image selection unit performs processing to select the first object image for all the first objects tracked by the first object tracking unit. In this case, if the number of first objects tracked by the first object tracking unit is large, the processing amount for selecting the first object image increases, which hinders the speeding up of object tracking. According to this configuration, the first object image selection unit does not perform processing for selecting the first object image for all the first objects tracked by the first object tracking unit, but the first object determination unit A process of selecting a first object image is performed on the first object determined by Therefore, even if the number of first objects tracked by the first object tracking unit is large, the processing amount for selecting the first object image can be suppressed, so that the object tracking can be speeded up.

  In the above configuration, there is an overlapping area in which the field of view of the first camera and the field of view of the second camera partially overlap, and the first object determining unit determines that the second object image is detected in the overlapping area. The first object indicated by the first object image detected in the overlapping area is determined as the first object having the possibility of being the same object as the second object.

  This configuration determines the above-mentioned "relationship between the position of one second object and the positions of a plurality of first objects" based on the overlapping area.

  If there is a first object image detected in the overlap region and a first object image detected outside the overlap region at the time when the second object image is detected in the overlap region, the first object detected in the other region than the overlap region The first object indicated by the object image is not the same as the second object indicated by the second object image. In addition, at the time when the second object image is detected in the overlapping area, if the number of first object images detected in the overlapping area is two or more, one of the first objects indicated by the two or more first object images is , The same as the second object. As described above, in this configuration, at the time when the second object image is detected in the overlap area, the first object indicated by the first object image detected in the overlap area is identical to the second object indicated by the second object image. It is determined that the first object has the possibility of being

  In the above configuration, the same object determination unit determines whether the first object and the second object are the same object using machine learning.

  This configuration is an example of the determination method of the same object.

  An object tracking method according to a second aspect of the present invention is an object tracking method for tracking an object using a plurality of cameras, wherein an image captured by a first camera, which is one of the plurality of cameras, is time-series Acquiring a first time-series image aligned in time, and a second time-series image aligned in time series with an image captured by a second camera that is one of the plurality of cameras; A first object tracking step of tracking the first object by detecting a first object image which is an image from the first time-series image; and a second object image which is an image of a second object at the second time A second object tracking step of tracking the second object by detecting from the sequence images, and a group of the second object images obtained by tracking of the second object by the second object tracking step according to a predetermined criterion Of the same object from among The second object image selecting step of selecting the second object image used for the constant setting, and the second object image from the group of the first object images obtained by the tracking of the first object by the first object tracking step A first object image selecting step of selecting the first object image having a feature similar to the feature of the second object image selected by the object image selecting step; and the second selected by the first object image selecting step A same object determining step of determining whether the first object and the second object are the same object using the one object image and the second object image selected in the second object image selecting step; An associating process step of associating the first object with the second object when the first object and the second object are determined to be the same object in the identical object determining step

  The object tracking method according to the second aspect of the present invention defines the object tracking device according to the first aspect of the present invention from the viewpoint of the method, and the same effects as the object tracking device according to the first aspect of the present invention Have.

  An object tracking program according to a third aspect of the present invention is an object tracking program for tracking an object using a plurality of cameras, and an image captured by a first camera which is one of the plurality of cameras is time-series Acquiring a first time-series image aligned in time, and a second time-series image aligned in time series with an image captured by a second camera that is one of the plurality of cameras; A first object tracking step of tracking the first object by detecting a first object image which is an image from the first time-series image; and a second object image which is an image of a second object at the second time A second object tracking step of tracking the second object by detecting from the sequence images, and a group of the second object images obtained by tracking of the second object by the second object tracking step according to a predetermined criterion From within From the group of the first object image obtained by the tracking of the first object by the second object image selecting step of selecting the second object image used for the determination of the same object, and the first object tracking step A first object image selecting step of selecting the first object image having features similar to the features of the second object image selected by the second object image selecting step; and selected by the first object image selecting step The same object determination to determine whether the first object and the second object are the same object using the first object image and the second object image selected in the second object image selection step An associating process step of associating the first object with the second object when the first object and the second object are determined to be the same object in the same object determining step; , Cause the computer to execute an.

  The object tracking program according to the third aspect of the present invention defines the object tracking device according to the first aspect of the present invention from the viewpoint of the program, and the same effects as the object tracking device according to the first aspect of the present invention Have.

  According to the present invention, in object tracking using a plurality of cameras, objects tracked using each camera can be accurately associated with each other.

It is a functional block diagram of an object tracking device concerning an embodiment. It is a schematic diagram which shows an example of arrangement | positioning of a 1st camera and a 2nd camera. It is a top view of the visual field of the 1st camera seen from the ceiling. It is a flow chart explaining operation of an object tracking device concerning an embodiment. It is a schematic diagram which shows a part of moving image V1 imaged by the 1st camera, and a part of moving image V2 imaged by the 2nd camera, when two objects (persons) are walking in the passage. FIG. 6 is a schematic view showing a state in which a rectangular area surrounding an object image is set in FIG. 5; In FIG. 6, it is a schematic diagram which shows the state in which the flow line was set. It is a functional block diagram of the object tracking device concerning the 1st modification. It is a functional block diagram of the object tracking device concerning the 2nd modification. It is a schematic diagram which shows an example of two flames | frames whose imaging time is the same. It is a schematic diagram which shows the other example of two flames | frames whose imaging time is the same. FIG. 12 is a schematic view showing an image on which two areas corresponding to the overlapping area are overlapped in FIG.

  Hereinafter, an embodiment of the present invention will be described in detail based on the drawings. In each figure, the structure which attached | subjected the same code | symbol shows that it is the same structure, and the description is abbreviate | omitted about the content already demonstrated about the structure. In the present specification, suffixes are collectively denoted by reference numerals without suffixes (for example, object 4), and suffixes are denoted by suffixes when referring to individual configurations (for example, object 4-1) , Object 4-2).

  FIG. 1 is a functional block diagram of an object tracking device 3-1 according to the embodiment. The object tracking device 3-1 determines the same object as the communication unit 31, the first object tracking unit 32, the second object tracking unit 33, the first object image selecting unit 34, and the second object image selecting unit 35. A section 36 and an association processing section 37 are provided. A person is taken as an example and described as the object 4 tracked by the object tracking device 3-1. However, the object 4 is not limited to a person, and may be a moving body (for example, a car) that can be imaged by a camera.

  Referring to FIG. 1, the communication unit 31 is a communication interface that communicates with the first camera 1 and the second camera 2 by wire or wirelessly. The first camera 1 and the second camera 2 are used as surveillance cameras. The communication unit 31 receives the moving image V1 captured by the first camera 1 and the moving image V2 captured by the second camera 2. The moving image V1 is an example of a first time-series image in which images captured by the first camera 1 are arranged in time series. The moving image V2 is an example of a first time-series image in which images captured by the second camera 2 are arranged in time series. The moving image V <b> 1 in a period in which the object 4 is present in the field of view 11 of the first camera 1 includes an object image indicating the object 4. The moving image V2 in a period in which the object 4 is present in the field of view 21 of the second camera 2 includes an object image indicating the object 4. The frame rates of the moving pictures V1 and V2 are arbitrary (for example, 24 fps, 15 fps, 10 fps, 1 fps).

  As described above, the communication unit 31 functions as an acquisition unit that acquires the moving image V1 and the moving image V2.

  FIG. 2 is a schematic view showing an example of the arrangement of the first camera 1 and the second camera 2. Referring to FIG. 2, this figure shows the passage 12 viewed from the side. An object 4-1 (person) is walking along the passage 12 in the arrow D direction. An object 4-2 (person) is walking along the passage 12 in the direction opposite to the arrow D direction. A ceiling 13 is located above the passage 12. The first camera 1 and the second camera 2 are installed on the ceiling 13 at an interval along the direction in which the passage 12 extends. The field of view 11 of the first camera 1 and the field of view 21 of the second camera 2 partially overlap, and an overlapping area 22 of the field of view is formed. The first camera 1 captures an object 4 located in the field of view 11. The second camera 2 captures an object 4 located within the field of view 21.

  The field of view of the camera viewed from the ceiling 13 will be described by taking the first camera 1 as an example. FIG. 3 is a plan view of the visual field 11 of the first camera 1 viewed from the ceiling 13. The optical axis center 14 of the first camera 1 is located at the center of the field of view 11. Depending on the orientation of the object 4 and the position of the object 4 on the field of view 11, the appearance shape of the object image showing the object 4 greatly differs even for the same object.

  Referring to FIGS. 1 and 2, the object 4 tracked by the first object tracking unit 32 is taken as a first object. In other words, the object 4 tracked using the first camera 1 is the first object. The first object tracking unit 32 tracks the first object by detecting a first object image that is an image of the first object from the moving image V1 received by the communication unit 31. For this tracking, known object tracking techniques (eg, particle filter, optical flow) can be used.

  The first object tracking unit 32 includes, for example, hardware such as a central processing unit (CPU), a random access memory (RAM), a read only memory (ROM), and a hard disk drive (HDD), and a first object tracking. The program is realized by a program and data for executing the function of the unit 32. Some or all of the functions of the first object tracking unit 32 may be realized by processing by a DSP (Digital Signal Processor) instead of or together with processing by the CPU. Also, some or all of the functions of the first object tracking unit 32 may be realized by processing by a dedicated hardware circuit instead of or in addition to processing by software. The same applies to the second object tracking unit 33, the first object image selection unit 34, the second object image selection unit 35, the identical object determination unit 36, and the association processing unit 37. The same applies to the first object determination unit 38 provided in the first modification described later and the first object determination unit 39 provided in the second modification.

  The object tracking device 3-1 is composed of a plurality of elements shown in FIG. Therefore, programs for realizing these elements are stored in the HDD. That is, in the HDD, each of the first object tracking unit 32, the second object tracking unit 33, the first object image selecting unit 34, the second object image selecting unit 35, the same object judging unit 36, and the association processing unit 37. A program for realizing the program is stored (in the first modified embodiment, a program for realizing the first object determining unit 38 is further stored in the HDD, and in the second modified embodiment, the program for realizing the first object determining unit 38 is stored). A program for realizing the object determination unit 39 is further stored in the HDD). These programs are expressed as a first object tracking program, a second object tracking program, a first object image selection program, a second object image selection program, an identical object determination program, and an association processing program.

  These programs are expressed using element definitions. The first object tracking unit 32 and the first object tracking program will be described as an example. The first object tracking unit 32 tracks the first object by detecting the first object image from the moving image V1 (first time-series image). The first object tracking program is a program for tracking a first object by detecting a first object image from a moving image V1 (first time-series image).

  A flowchart of these programs (a first object tracking program, a second object tracking program, a first object image selection program, a second object image selection program, a same object determination program, an association processing program) executed by the CPU It is FIG. 4 demonstrated later.

  Although these programs are stored in advance in the HDD, the present invention is not limited to this. For example, a recording medium (for example, an external recording medium such as a magnetic disk or an optical disk) recording these programs may be prepared, and the program stored in the recording medium may be stored in the HDD. . Also, these programs may be stored in a server connected to the object tracking device 3-1 via a network, and these programs may be sent to the HDD via the network and stored in the HDD. These programs may be stored in the ROM instead of the HDD. The object tracking device 3-1 may include a flash memory instead of the HDD, and these programs may be stored in the flash memory.

  An object 4 tracked by the second object tracking unit 33 is taken as a second object. In other words, the object 4 tracked using the second camera 2 is the second object. The second object tracking unit 33 tracks the second object by detecting a second object image which is an image of the second object from the moving image V2 received by the communication unit 31. For this tracking, known object tracking techniques (eg, particle filter, optical flow) can be used.

  The first object image selection unit 34, the second object image selection unit 35, the same object determination unit 36, and the association processing unit 37 will be described later.

  The operation of the object tracking device 3-1 according to the embodiment will be described. FIG. 4 is a flowchart explaining this. Referring to FIGS. 1 and 2, the first camera 1 and the second camera 2 operate continuously for a predetermined time (for example, continuous operation for 24 hours), and the communication unit 31 (an example of an acquisition unit) The moving image V1 captured by the camera 1 and the moving image V2 captured by the second camera 2 are received in real time (step S1 in FIG. 4).

  The first object tracking unit 32 tracks the object 4 (first object) by detecting an object image from the moving image V1 received by the communication unit 31, and the second object tracking unit 33 is received by the communication unit 31. The object 4 (second object) is tracked by detecting the object image from the moving image V2 (step S2 in FIG. 4). The tracking of the object 4 will be described in detail. FIG. 5 shows a part of the moving image V1 captured by the first camera 1 and an image captured by the second camera 2 when the object 4-1 (person) and the object 4-2 (person) walk in the passage 12 It is a schematic diagram which shows a part of moving image V2. A part of the moving image V1 is constituted by a frame F1 from an imaging time T1 to an imaging time T6. A part of the moving image V2 is composed of a frame F2 from an imaging time T1 to an imaging time T6. The time advances in order of imaging time T1, imaging time T2, imaging time T3, imaging time T4, imaging time T5, and imaging time T6. FIG. 6 is a schematic view showing a state in which rectangular regions 411, 421 and 431 surrounding the object images 41, 42 and 43 are set in FIG. FIG. 7 is a schematic view showing a state in which flow lines L1, L2 and L3 are set in FIG.

  Referring to FIGS. 1, 2 and 5, object image 41 (first object image) is an image of object 4-1 (first object) included in moving image V1 captured by first camera 1. . The object image 43 (first object image) is an image of an object 4-2 (first object) included in the moving image V1 captured by the first camera 1. The object image 42 (second object image) is an image of the object 4-1 (second object) included in the moving image V2 captured by the second camera 2.

  Referring to FIGS. 1, 2 and 6, the first object tracking unit 32 detects an object image included in the moving image V1, and sets a rectangular area surrounding the object image as the moving image V1. Here, the first object tracking unit 32 detects the object image 41 and the object image 43, and sets a rectangular area 411 surrounding the object image 41 and a rectangular area 431 surrounding the object image 43 as the moving image V1. Although the object image 41 is included in the frame F1 at the imaging time T4, the object image 41 is not detected because it is an image of a part of the object 4-1.

  The second object tracking unit 33 detects an object image included in the moving image V2, and sets a rectangular area surrounding the object image as the moving image V2. Here, the second object tracking unit 33 detects the object image 42 and sets the rectangular area 421 surrounding the object image 42 as the moving image V2. Although the object image 42 is included in the frame F2 at the imaging time T3, the object image 42 is not detected because it is an image of a part of the object 4-1.

  The field of view 11 of the first camera 1 and the field of view 21 of the second camera 2 partially overlap. The overlapping area 22 is three-dimensional, and the height of the overlapping area 22 is lower than that of the object 4. For this reason, since only a part of the object 4 is included in the overlapping area 22, the object 4 is not detected in the overlapping area 22. Therefore, the object image 41 of the object 4-1 and the object image 42 of the object 4-1 are not simultaneously detected. That is, at the imaging time T3, the object image 41 of the object 4-1 is detected, but the object image 42 of the object 4-1 is not detected. At the imaging time T4, the object image 42 of the object 4-1 is detected, but the object image 41 of the object 4-1 is not detected.

  The above is the tracking of the object 4 (step S2 in FIG. 4). The object tracking device 3-1 uses the object image 41 (first object image) and the object image 42 (second object image) to track the object 4-1 to display an object 4-1 indicated by the object image 41. It must be determined that 1 (first object) and the object 4-1 (second object) indicated by the object image 42 are the same object. Referring to FIG. 6, for example, among three frames F1 in which object image 41 is detected and three frames F2 in which object image 42 is detected, object images 41 included in frames F1 and F2 having the closest imaging time. , 42 are used to determine the same object. Here, the object image 41 included in the frame F1 at the imaging time T3 and the object image 42 included in the frame F2 at the imaging time T4. The object image 41 included in the frame F1 at the imaging time T3 is an image on the back side of the object 4-1 (person). The object image 42 included in the frame F2 at the imaging time T4 is an image on the front side of the object 4-1 (person). In terms of appearance, the object image 41 and the object image 42 are not similar, and the object 4-1 indicated by the object image 41 and the object 4-1 indicated by the object image 42 are the same although they are the same object. It may be determined not to be an object.

  Therefore, in the embodiment, the object image 41 (first object image) and the object image 42 (second object image) to be used for determination of the same object are selected by the following steps S3 and S4. Referring to FIGS. 1, 2 and 7, second object image selection unit 35 selects object image 42 used for determination of the same object (step S3 in FIG. 4). Specifically, the second object image selection unit 35 sets a flow line to the object image of the moving image V2. The movement line is a line indicating the direction in which the object 4 walks and the trajectory of the walk. The second camera 2 captures an image of an object 4-1 walking in the direction of arrow D in the passage 12. The second object image selection unit 35 generates a flow line L2 of the object image 42 indicating the object 4-1, and a frame F2 (a frame F2 of the imaging time T4) including the object image 42 detected by the second object tracking unit 33. The flow line L2 is set to the frame F2 at the imaging time T5 and the frame F2 at the imaging time T6. The flow line L2 of the object image 42 is a straight line in the same direction as the arrow D direction.

  The second object image selection unit 35 selects an object image 42 used for the determination of the same object from the frame F2 in which the flow line L2 is set. In the embodiment, the object image 42 detected first, that is, the object image 42 included in the frame F2 at the imaging time T4 is selected.

  As described above, the second object image selection unit 35 determines the group of the second object images obtained by tracking the second object (object 4-1) by the second object tracking unit 33 according to a predetermined criterion (imaging time) From the object images 42) respectively included in the frame F2 of T4, T5, and T6, the second object image (the object image 42 included in the frame F2 of the imaging time T4) used to determine the same object is selected.

  Referring to FIGS. 1, 2 and 7, next, the first object image selection unit 34 selects the object image 41 used for the determination of the same object (step S4 in FIG. 4). Specifically, the first object image selection unit 34 sets a flow line to the object image of the moving image V1. The first camera 1 captures an object 4-1 that walks in the passage 12 in the arrow D direction, and an object 4-2 that walks in the passage 12 in the direction opposite to the arrow D direction. The first object image selection unit 34 generates a flow line L1 of the object image 41 indicating the object 4-1, and a frame F1 (a frame F1 at an imaging time T1) including the object image 41 detected by the first object tracking unit 32. Then, the flow line L1 is set to the frame F1 of the imaging time T2 and the frame F1 of the imaging time T3. The flow line L1 of the object image 41 is a straight line in the same direction as the arrow D direction.

  The first object image selection unit 34 generates a flow line L3 of the object image 43 indicating the object 4-2, and a frame F1 including the object image 43 detected by the first object tracking unit 32 (frame F1 of imaging time T1 The flow line L3 is set to the frame F1 at the imaging time T2 and the frame F1 at the imaging time T3. The flow line L3 of the object image 43 is a straight line in the direction opposite to the arrow D direction.

  The first object image selection unit 34 selects, among the objects 4 tracked by the first object tracking unit 32, the object 4 indicated by the object image in which the movement line in the same direction as the movement line L2 is set. Here, the first object tracking unit 32 tracks the object 4-1 and the object 4-2. An object 4 indicated by an object image in which a movement line in the same direction as the movement line L2 is set is an object 4-1 indicated by an object image 41 in which the movement line L1 is set.

  The first object image selection unit 34 selects an object image 41 having the same position on the frame as the object image 42 selected by the second object image selection unit 35 in step S3 among the object images 41 for which the flow line L1 is set. Choose This is because the shape of the image (object image) of the object 4 is different depending on the position of the object 4 on the field of view 11 even in the same object as described in FIG. For example, a frame virtually divided into a total of six regions (vertical upper region, central lower region, upper left region, lower left region, upper right region, lower right region) of 2 vertical × 3 horizontal In F1 and frame F2, if the object image is located in the same area, the position on the frame of the object image is considered to be the same.

  The object image 42 selected by the second object image selection unit 35 is located in the lower right area of the frame F2. Therefore, the first object image selection unit 34 selects the object image 41 (the object image 41 included in the frame F1 at the imaging time T1) located in the lower right area of the frame F1. Here, among the group of object images 41, the object image 41 located in the lower right region of the frame F1 is only the object image 41 included in the frame F1 at the imaging time T1. However, if the object image 41 is positioned in the lower right region in any of the continuous frames F1 including the frame F1 at the imaging time T1, the first object image selecting unit 34 determines a predetermined one of these object images 41. The object image 41 is selected according to the criteria of. For example, the first object image selection unit 34 selects the object image 41 included in the frame F1 at the imaging time closest to the imaging time T4 from the continuous frames F1.

  In the embodiment, if the area where the object image 41 is located and the area where the object image 42 is located are the same, it is determined that they are at the same position. However, the present invention is not limited to this. For example, when the distance between the gravity center of the object image 41 and the gravity center of the object image 42 is less than a predetermined threshold, the first object image selection unit 34 You may judge.

  As described above, the first object image selection unit 34 generates a group of first object images obtained by tracking the first object (object 4-1) by the first object tracking unit 32 (imaging times T1, T2, and T3). Features similar to the features of the second object image (the object image 42 included in the frame F2 at the imaging time T4) selected by the second object image selection unit 35 from the object images 41) respectively included in the frame F1 of A first object image (object image 41 included in frame F1 at imaging time T1) is selected. The characteristic similar judgment elements are as follows.

(1) The movement direction of the first object image (the direction of the movement line) and the movement direction of the second object image (the direction of the movement line) are the same.
(2) The position on the image (frame) of the first object image and the position on the image (frame) of the second object image are the same.
(3) The moving speed of the first object image (first object) is the same as the moving speed of the second object image (second object).
(4) The orientation of the first object image is the same as the orientation of the second object image.

  The same here may be completely the same or almost the same. The “moving speed” of (3) is calculated based on, for example, the amount of movement of the object image on the frame in a predetermined time. The “direction” of (4) is obtained, for example, by template matching. If the first object image and the second object image are the same for any one of (1) to (4) or a combination of two or more of them, the first object image selection unit 34 It is determined that the feature of one object image and the feature of the second object image are similar. In the embodiment, if (1) and (2) are satisfied, it is determined that the feature of the first object image and the feature of the second object image are similar.

  Referring to FIGS. 1, 2 and 7, the same object determination unit 36 selects the object image 41 (first object image) selected by the first object image selection unit 34 in step S4, and the second one in step S3. The object 4-1 (first object) indicated by the object image 41 and the object 4-1 indicated by the object image 42 are identical using the object image 42 (second object image) selected by the object image selection unit 35 It is determined whether or not it is an object (step S5 in FIG. 4).

  For this determination, it is possible to use a known determination technique for the same object. For example, the same object determination unit 36 determines the color information of the object image 41 selected by the first object image selection unit 34 in step S4 and the color of the object image 42 selected by the second object image selection unit 35 in step S3. If the degree of coincidence with the information is higher than a predetermined threshold, it is determined that the same object is present (histogram intersection). Instead of color information, the matching degree of shape information may be used. For example, the same object determination unit 36 may determine whether the object is the same object using machine learning (for example, deep learning, support vector machine).

  When the same object determination unit 36 determines that the object is the same (Yes in step S5 in FIG. 4), the association processing unit 37 sets the same ID as the ID of the object 4-1 (first object) indicated by the object image 41. , And is applied to the object 4-1 (second object) indicated by the object image 42 (step S6 in FIG. 4). That is, in the association processing unit 37, the same object determination unit 36 determines that the object 4-1 (first object) indicated by the object image 41 and the object 4-1 (second object) indicated by the object image 42 are the same object. When it is determined, processing is performed to associate the first object with the second object.

  When the same object determination unit 36 determines that the object is not the same (No in step S5 of FIG. 4), the association processing unit 37 determines an ID different from the ID of the object 4-1 indicated by the object image 41. Is given to the object 4-1 indicated by (step S 7 in FIG. 4).

  As described above, the association processing unit 37 causes the first object tracking unit 32 to track two objects 4-1 (one second object) tracked by the second object tracking unit 33. Among the objects 4-1, 4-2 (a plurality of first objects), processing of associating one object 4-1 (a first object) is performed.

  In the embodiment, among the objects 4-1 and 4-2 (first object) tracked by the first object tracking unit 32, an object 4 indicated by an object image in which a movement line in the same direction as the movement line L2 is set The case where the number of first objects) is one (object 4-1) has been described as an example. However, when the number is two or more, the first object image selection unit 34 performs the process of step S4 for each of two or more objects 4 (first objects), and the same object determination unit 36 performs the step Process S5. Then, the object 4-1 shows the same ID as the ID of the object 4 (first object) having the largest determination score of the same object among the two or more objects 4 (first objects). Apply to (second object).

  The main effects of the embodiment will be described. Referring to FIGS. 1, 2 and 7, first object image selection unit 34 is an object image having a feature similar to the feature of object image 42 (second object image) included in frame F2 at imaging time T4. The object image 41 included in the frame F1 at the imaging time T1 is selected as 41 (first object image). The identical object determination unit 36 uses the two object images 41 and 42 to generate an object 4-1 (first object) indicated by the object image 41 and an object 4-1 (second object) indicated by the object image 42. It is determined whether it is the same object or not. Thereby, the determination accuracy of the same object can be improved. Therefore, according to the object tracking device 3-1 according to the embodiment, in the object tracking using a plurality of cameras, objects tracked using each camera can be accurately associated with each other.

  Although the embodiment has been described in the case where the number of cameras is two, the number of cameras may be three or more. When the number of cameras is three, two cameras whose fields of view are adjacent become the first camera 1 and the second camera 2. For example, when three cameras, camera A, camera B and camera C, are arranged, and the field of view of camera A, the field of view of camera B, and the field of view of camera C are arranged in order, camera A and camera B, and camera B and camera C Is the relationship between the first camera 1 and the second camera 2.

  When the number of cameras is three or more, the object tracking device 3-1 includes a storage unit (not shown) that previously stores data in which two cameras with adjacent fields of view are linked. Two cameras whose fields of view are adjacent to each other are the first camera 1 and the second camera 2, and a moving image V1 (first time-series image) captured by the first camera 1 and a moving image V2 captured by the second camera 2 (second It is because it is determined whether or not the first object and the second object are the same object by using the time-series image).

  A modification of the embodiment will be described. In the embodiment, a process in which the first object image selection unit 34 selects an object image (step S4 of FIG. 4: setting of flow line, the direction of the flow line is the same as the object image 42 selected by the second object image selection section 35). And the object 4 used as the object of specification of the object image with the same position on a flame | frame is all the objects 4 which the 1st object tracking part 32 tracked (object 4-1, object 4-2). When the number of the objects 4 tracked by the first object tracking unit 32 is large, the amount of computation of the above process (step S4 in FIG. 4) increases, which prevents the object tracking from being speeded up. Therefore, in the first modification, processing is performed that can reduce the number of objects 4 to be targets of the above processing (step S4 in FIG. 4). Details will be described below.

  FIG. 8 is a functional block diagram of an object tracking device 3-2 according to a first modification. The differences from the object tracking device 3-1 shown in FIG. 1 will be mainly described. The object tracking device 3-2 further includes a first object determination unit 38 in addition to the components of the object tracking device 3-1 illustrated in FIG.

  Referring to FIGS. 2 and 6, object 4-1 walks in the direction of arrow D (leftward in the plane of FIG. 2). Therefore, the object image 41 of the object 4-1 disappears from the left end (left side) of the frame F1, and after a predetermined time, the object image 42 of the object 4-1 appears at the right end (right side) of the frame F2. In other words, the object image 41 of the object 4-1 is located at the left end of the frame F1 at the imaging time T3, and the object image 42 of the object 4-1 is located at the right end of the frame F2 at the imaging time T4. This predetermined time is an estimated time (several seconds) determined from the walking speed. Therefore, among the plurality of objects 4 tracked by the first object tracking unit 32, the objects 4 satisfying the following conditions (i) and (ii) are one object 4 tracked by the second object tracking unit 33 ( There is a possibility of the object 4-1).

(I) The frame F1 is a frame F1 which is before the imaging time (imaging time T4) of the frame F2 in which the object image 42 of the object 4-1 is positioned at the right end and which is imaged within a predetermined time from this imaging time. (Frame F1 at imaging time T3).
(Ii) The object 4 is the object 4 (object 4-1) in which the object image (object image 41) is located at the left end of the frame F1 of (i).

  The object image 43 is not located at the left end of the frame F1 at the imaging time T3 for the object 4-2 which is not the same object as the one object 4 (object 4-1) tracked by the second object tracking unit 33. Therefore, the object 4-2 does not satisfy the conditions of (i) and (ii).

  As described above, when the object 4 (first object) tracked by the first object tracking unit 32 satisfies the conditions of (i) and (ii), the first object determining unit 38 tracks the second object tracking unit 33 It is determined that the object 4 (first object) has the possibility of being the same object as the one object 4 (second object). Thus, the first object determination unit 38 selects one of the plurality of first objects based on the relationship between the position of one second object and the positions of the plurality of first objects, and the same object as the second object. Determine the first object that has the possibility of being

  A process in which the first object image selection unit 34 selects an object image for the object 4 (first object) determined by the first object determination unit 38 (step S4 in FIG. 4). )do.

  The association processing unit 37 is configured to track two objects 4-1, 4 tracked by the first object tracking unit 32 with respect to one object 4-1 (one second object) tracked by the second object tracking unit 33. -2 A process of correlating one object 4-1 (one first object) among -2 (a plurality of first objects) is performed. The first modification is based on this, and by providing the first object determination unit 38, the process described above (FIG. 4) even when the number of objects 4 (first objects) tracked by the first object tracking unit 32 is large. Since the amount of calculation in step S4) can be suppressed, the speed of object tracking can be increased.

  A second modified example of the embodiment will be described. Among the plurality of objects 4 (first objects) tracked by the first object tracking unit 32, objects having the possibility of being identical to one object 4 (second object) tracked by the second object tracking unit 33 The second modification is different from the first modification in the determination method of 4 (first object). FIG. 9 is a functional block diagram of an object tracking device 3-3 according to a second modification. The differences from the object tracking device 3-1 shown in FIG. 1 will be mainly described. The object tracking device 3-3 further includes a first object determination unit 39 and an address storage unit 40 in addition to the components of the object tracking device 3-1 shown in FIG.

  In the second modification, the angle of view of the first camera 1 and the second camera 2 is set so that the object 4 can be detected in the overlapping area 22 shown in FIG. FIG. 10 is a schematic view showing an example of the frame F1 and the frame F2 having the same imaging time. In the frame F1, an area 221 corresponding to the overlapping area 22 is located at the left end of the frame F1. In the frame F2, an area 222 corresponding to the overlapping area 22 is located at the right end of the frame F2. In the second modification, “the relationship between the position of one second object and the positions of a plurality of first objects” is determined based on the overlapping area 22.

  The first object tracking unit 32 detects an object image 44, an object image 45 and an object image 46. In the frame F1, a rectangular area 441 surrounding the object image 44 is set, a rectangular area 451 surrounding the object image 45 is set, and a rectangular area 461 surrounding the object image 46 is set. The object 4 indicated by the object image 44, the object 4 indicated by the object image 45, and the object 4 indicated by the object image 46 are different from each other.

  The second object tracking unit 33 detects an object image 47 and an object image 48. In the frame F2, a rectangular area 471 surrounding the object image 47 is set, and a rectangular area 481 surrounding the object image 48 is set. The object 4 shown by the object image 47 and the object 4 shown by the object image 48 are different.

  The address storage unit 40 stores in advance the address of the area 221 on the frame F1 and the address of the area 222 on the frame F2. The address storage unit 40 is realized by an HDD, a flash memory or the like. The first object determination unit 39 determines whether the object image (the object image 47 and the object image 48) on the frame F2 is positioned in the region 222 based on the address of the region 222 on the frame F2. The first object determination unit 39 determines whether the object image (the object image 44, the object image 45, the object image 46) on the frame F1 is located in the region 221 based on the address of the region 221 on the frame F1. to decide.

  The object image 44 and the object image 45 are located in the area 221, and the object image 46 is located outside the area 221. The object image 47 is located in the area 222. Therefore, among the three object images 44, 45 and 46, the object 4 shown by the object image 44 and the object 4 shown by the object image 45 have the possibility of the same object as the object 4 shown by the object image 47. Therefore, the first object determination unit 39 causes each of the object images 44 and 45 detected in the overlapping area 22 at the time when the object image 47 (second object image) is detected in the overlapping area 22 (first A first object shown by the object image is determined as an object 4 (first object) having the possibility of being the same object as the object 4 (second object) shown by the object image 47.

  In the description of step S4 in FIG. 4, the first object image selection unit 34 determines that there are a plurality of first object images (object images 41) having features similar to the features of the second object image (object image 42). Included in the frame F1 captured at a time closest to the time (imaging time T4) at which the image frame F2 including the second object image (object image 42) selected by the second object image selection unit 35 is captured The first object image is selected. For example, the first object image selection unit 34 may select the first object image (object image 41) having a feature most similar to the feature of the second object image (object image 42). The first object image selection unit 34 may select the first object image based on the respective distances between the second object image and the plurality of first object images. The latter will be described in detail.

  FIG. 11 is a schematic view showing another example of the frame F1 and the frame F2 having the same imaging time. The differences between FIG. 11 and FIG. 10 will be described. The first object tracking unit 32 detects an object image 49 and an object image 50. In the frame F1, a rectangular area 491 surrounding the object image 49 is set, and a rectangular area 501 surrounding the object image 50 is set. The object 4 shown by the object image 49 and the object 4 shown by the object image 50 are different. The second object tracking unit 33 detects the object image 51. In the frame F2, a rectangular area 511 surrounding the object image 51 is set.

  The first object image selection unit 34 superimposes the image of the area 221 corresponding to the overlapping area 22 and the image of the area 222 corresponding to the overlapping area 22. FIG. 12 is a schematic view showing an image subjected to this processing. The first object image selection unit 34 calculates the center of gravity 49a, the center of gravity 50a, and the center of gravity 51a for each of the object image 49, the object image 50, and the object image 51.

  The first object image selection unit 34 calculates the distance d1 between the center of gravity 51a of the object image 51 (second object image) and the center of gravity 49a of the object image 49 (first object image). The distance d2 between the center of gravity 51a) and the center of gravity 50a of the object image 50 (first object image) is calculated. Among these distances, the first object image selection unit 34 identifies the shortest distance (here, the distance d1), and an object image (here, the object image 49) having a center of gravity that defines this distance. It is selected as the first object image.

DESCRIPTION OF SYMBOLS 1 1st camera 2 2nd camera 3-1, 3-2, 3-3 Object tracking device 4, 4, 4- 2, 2 Object 11 Field of view 12 Passage 13 Ceiling 14 Optical axis center 21 Field of view 22 Overlap area 41-51 Object image 49a, 50a center of gravity 411, 421, 431, 441, 451, 461, 471, 481, 491, 501, 511 rectangular area 221, 222 area d1, d2 corresponding to overlapping area distance F1, F2 frame L1, L2, frame L3 flow line V1 movie (an example of the first time series image)
V2 movie (example of second time series image)

Claims (7)

An object tracking apparatus for tracking an object using a plurality of cameras, comprising:
A first time-series image in which images taken by a first camera, which is one of the plurality of cameras, are arranged in time series, and an image taken by a second camera, which is another one of the plurality of cameras An acquisition unit that acquires second time-series images arranged in time series;
A first object tracking unit configured to track the first object by detecting a first object image which is an image of the first object from the first time-series image;
A second object tracking unit that tracks the second object by detecting a second object image that is an image of a second object from the second time-series image;
A second object for selecting the second object image to be used for determination of the same object from among a group of the second object images obtained by tracking of the second object by the second object tracking unit according to a predetermined standard An image selection unit,
Among the group of the first object images obtained by the first object tracking unit by the first object tracking unit, the feature similar to the feature of the second object image selected by the second object image selecting unit is A first object image selection unit that selects the first object image that it has;
The first object and the second object are the same using the first object image selected by the first object image selection unit and the second object image selected by the second object image selection unit. An identical object determination unit that determines whether the object is an object;
An object tracking unit that includes an association processing unit that performs processing to associate the first object with the second object when the first object and the second object are determined to be the same object by the same object determination unit apparatus. The association processing unit performs processing to associate one first object among a plurality of first objects with respect to one second object,
The object tracking device is the same object as the second object among the plurality of first objects based on the relationship between the position of one second object and the positions of the plurality of first objects. The apparatus further comprises a first object determination unit that determines the first object having the possibility
The object tracking device according to claim 1, wherein the first object image selection unit performs a process of selecting the first object image for the first object determined by the first object determination unit. There is an overlapping area where the field of view of the first camera and the field of view of the second camera partially overlap,
The first object determination unit determines the first object indicated by the first object image detected in the overlapping area at the time when the second object image is detected in the overlapping area, the same object as the second object The object tracking device according to claim 2, wherein the first object having the possibility of being determined is determined.   The object tracking device according to any one of claims 1 to 3, wherein the same object determination unit determines whether the first object and the second object are the same object by using machine learning. The first object image selecting unit is configured such that the first object image and the second object image are the same for any one of the following (1) to (4) or a combination of two or more of them: The object tracking device according to any one of claims 1 to 4, wherein it is determined that the features of the first object image and the features of the second object image are similar, if any.
(1) The moving direction of the first object image is the same as the moving direction of the second object image.
(2) The position on the image of the first object image and the position on the image of the second object image are the same.
(3) The moving speed of the first object image and the moving speed of the second object image are the same.
(4) The orientation of the first object image and the orientation of the second object image are the same. An object tracking method for tracking an object using a plurality of cameras, comprising:
A first time-series image in which images taken by a first camera, which is one of the plurality of cameras, are arranged in time series, and an image taken by a second camera, which is another one of the plurality of cameras An acquisition step of acquiring a second time-series image arranged in time series;
A first object tracking step of tracking the first object by detecting a first object image which is an image of the first object from the first time-series image;
A second object tracking step of tracking the second object by detecting a second object image which is an image of a second object from the second time-series image;
A second object for selecting the second object image used to determine the same object from among the group of the second object images obtained by the tracking of the second object by the second object tracking step according to a predetermined criterion An image selection step,
Among the group of the first object image obtained by the tracking of the first object by the first object tracking step, a feature similar to the feature of the second object image selected by the second object image selecting step Selecting a first object image having the first object image;
The first object and the second object are the same using the first object image selected in the first object image selection step and the second object image selected in the second object image selection step An identical object determination step of determining whether the object is an object;
An object tracking process including a process of correlating the first object with the second object when the first object and the second object are determined to be the same object in the same object determining step Method. An object tracking program for tracking an object using a plurality of cameras, comprising:
A first time-series image in which images taken by a first camera, which is one of the plurality of cameras, are arranged in time series, and an image taken by a second camera, which is another one of the plurality of cameras An acquisition step of acquiring a second time-series image arranged in time series;
A first object tracking step of tracking the first object by detecting a first object image which is an image of the first object from the first time-series image;
A second object tracking step of tracking the second object by detecting a second object image which is an image of a second object from the second time-series image;
A second object for selecting the second object image used to determine the same object from among the group of the second object images obtained by the tracking of the second object by the second object tracking step according to a predetermined criterion An image selection step,
Among the group of the first object image obtained by the tracking of the first object by the first object tracking step, a feature similar to the feature of the second object image selected by the second object image selecting step Selecting a first object image having the first object image;
The first object and the second object are the same using the first object image selected in the first object image selection step and the second object image selected in the second object image selection step An identical object determination step of determining whether the object is an object;
When the first object and the second object are determined to be the same object in the same object determining step, the computer executes an associating processing step of associating the first object with the second object. Object tracking program.

Images