TWI879635B - Six-dimensional object posture tracking method and device, computer-readable recording medium - Google Patents

Abstract

A six-dimensional object posture tracking method, wherein a two-dimensional object tracking module in a six-dimensional object posture tracking device detects, tracks and classifies an object on an input RGB image, and generates a two-dimensional object tracking result corresponding to each object in the RGB image. Then, when a six-dimensional object posture estimation module in the six-dimensional object posture tracking device determines that an identification code included in one of the two-dimensional object tracking results does not appear in an object list, the six-dimensional object posture estimation module According to the two-dimensional object tracking results of the identification codes that do not appear in the object list, six-dimensional posture estimation is performed on the objects in the RGB image that correspond to the identification codes that do not appear in the object list, and the corresponding six-dimensional posture estimation results are recorded in the object list; then the six-dimensional object posture tracking module in the six-dimensional object posture tracking device performs six-dimensional object posture tracking on the objects in the RGB image according to all six-dimensional posture estimation results contained in the object list.

Description

Six-dimensional object posture tracking method and device, and computer-readable recording medium

The present invention relates to a method for tracking an object in an image, and more particularly to a six-dimensional object posture tracking method for tracking the six-dimensional posture of an object in an image.

The existing 6D object posture tracking technology can track the 6D posture of objects in the image (including three translational degrees of freedom (displacement in the x, y, and z axes) and three rotational degrees of freedom (rotation angles around the x, y, and z axes)). The 6D object posture tracking technology mainly includes two parts: object posture estimation and object posture tracking. Object posture estimation is relatively time-consuming. Therefore, when there are more objects in the image, more time will be spent in the object posture estimation stage.

In addition, the existing six-dimensional object posture tracking technology will perform six-dimensional posture estimation and tracking on all objects in each input image. Therefore, even if the objects in an input image are completely unchanged from the previous image or only one object is added, the six-dimensional object posture tracking technology will still perform posture estimation and tracking on all objects in the input image again, resulting in the performance of the existing six-dimensional object posture tracking technology cannot be effectively improved.

Therefore, the purpose of the present invention is to provide a six-dimensional object posture tracking method and a six-dimensional object posture tracking device and a computer-readable recording medium for implementing the method, which can shorten the time of estimating the six-dimensional object posture in the six-dimensional object posture tracking technology to improve its tracking performance.

Therefore, the present invention provides a six-dimensional object posture tracking method, which is applied to a six-dimensional object posture tracking device; the method includes the following steps.

A two-dimensional object tracking module of the six-dimensional object posture tracking device performs object detection, tracking and classification on an input RGB image, and generates a two-dimensional object tracking result corresponding to each object in the RGB image, each of which includes an identification code and a bounding box of the corresponding object.

A six-dimensional object posture estimation module of the six-dimensional object posture tracking device compares the two-dimensional object tracking results with an object list, and when it is determined that the identification code included in one of the two-dimensional object tracking results does not appear in the object list, the six-dimensional object posture estimation module performs six-dimensional posture estimation on the object corresponding to the identification code that does not appear in the object list in the RGB image based on the two-dimensional object tracking result to which the identification code that does not appear in the object list belongs and a depth map corresponding to the RGB image, and records the corresponding six-dimensional posture estimation result in the object list.

A six-dimensional object posture tracking module of the six-dimensional object posture tracking device performs six-dimensional object posture tracking on the object in the RGB image according to all six-dimensional posture estimation results included in the object list and the depth map corresponding to the RGB image.

Furthermore, the present invention implements a six-dimensional object posture tracking device for the above method, including a storage unit and a processing unit; the storage unit stores an object list; the processing unit includes a two-dimensional object tracking module, a six-dimensional object posture estimation module and a six-dimensional object posture tracking module.

The processing unit executes the two-dimensional object tracking module to detect, track and classify objects on an input RGB image, and generates a two-dimensional object tracking result corresponding to each object in the RGB image. Each of the two-dimensional object tracking results includes an identification code and a bounding box of the corresponding object.

The processing unit executes the six-dimensional object posture estimation module to compare the two-dimensional object tracking results with the object list stored in the storage unit. When it is determined that the identification code included in one of the two-dimensional object tracking results does not appear in the object list, the six-dimensional object posture estimation module performs six-dimensional posture estimation on the object corresponding to the identification code that does not appear in the object list in the RGB image based on the two-dimensional object tracking result to which the identification code does not appear in the object list and a depth map corresponding to the RGB image, and records the corresponding six-dimensional posture estimation result in the object list.

The processing unit executes the six-dimensional object posture tracking module so that it performs six-dimensional object posture tracking on the object in the RGB image according to all six-dimensional posture estimation results contained in the object list and the depth map corresponding to the RGB image.

In some implementations of the present invention, when the six-dimensional object pose estimation module determines that the identification code included in one of the six-dimensional pose estimation results in the object list does not appear in one of the two-dimensional object tracking results, the six-dimensional object pose estimation module will remove the six-dimensional pose estimation result to which the identification code that does not appear in one of the two-dimensional object tracking results belongs from the object list.

In some embodiments of the present invention, the 2D object tracking module and the 6D object posture estimation module perform asynchronous operations via a multi-thread mechanism.

In some implementations of the present invention, the six-dimensional object pose estimation module performs six-dimensional pose estimation on the object in the RGB image corresponding to the identification code that does not appear in the object list based on a CAD model corresponding to the object in the RGB image corresponding to the identification code that does not appear in the object list.

In some embodiments of the present invention, the 6D object pose tracking module simultaneously tracks multiple objects in the RGB image through a dynamic batch inference mechanism.

In some embodiments of the present invention, each of the two-dimensional object tracking results also includes the corresponding class name and mask of each of the objects, and each of the six-dimensional posture estimation results includes the corresponding identification code, class name, two-dimensional posture information, six-dimensional posture information, mesh, mesh tensor, mesh diameter and model center of the object.

In addition, the present invention implements a computer-readable recording medium for implementing the above-mentioned method, in which a software program including a two-dimensional object tracking module, a six-dimensional object posture estimation module and a six-dimensional object posture tracking module is stored. When the software program is loaded and executed by a computer device, the computer device can complete the six-dimensional object posture tracking method as described above.

The effect of the present invention is that: the two-dimensional object tracking module first performs object detection, tracking and classification on the input RGB image to generate two-dimensional object tracking results corresponding to each object in the RGB image, and the six-dimensional object posture estimation module compares the two-dimensional object tracking results with the object list to find out the identification codes corresponding to the objects in the RGB image that exist in the two-dimensional object tracking results but not in the object list, and only performs six-dimensional posture estimation on the objects in the RGB image that correspond to the identification codes that do not appear in the object list, thereby saving software and hardware computing resources, shortening the computing time of six-dimensional posture estimation, and accelerating the speed and performance of six-dimensional object posture tracking.

Before the present invention is described in detail, it should be noted that similar components are represented by the same reference numerals in the following description.

Referring to FIG. 1 , there is shown a flow chart of an embodiment of a six-dimensional object posture tracking method of the present invention, which is implemented by a six-dimensional object posture tracking device 100 shown in FIG. 2 . The six-dimensional object posture tracking device 100 is a computer device, which mainly includes a storage unit 1 and a processing unit 2. The storage unit 1 (e.g., a memory module) stores an object list 11.

The processing unit 2 is, for example but not limited to, a central processing unit (CPU), a microprocessor (MPU), a graphics processing unit (GPU), a tensor processing unit (TPU), etc., or any combination thereof, and the processing unit 2 loads and executes a software program stored in a computer-readable recording medium (for example but not limited to the storage unit 1), and the software program includes a two-dimensional object tracking module 21, a six-dimensional object posture estimation module 22, and a six-dimensional object posture tracking module 23.

The two-dimensional object tracking module 21 includes an object detection model for performing object detection, such as but not limited to YoloV8, an object tracking module for performing object tracking, such as but not limited to BoT-SORT, and an object classification model for performing object classification, such as but not limited to DINO-V2. The combination of YOLOv8 and BoT-SORT realizes multi-object tracking (MOT). Under changing environmental conditions, such as light changes, occlusion, etc., YOLOv8 can accurately detect target objects in the image, while BoT-SORT can effectively track these objects. In addition, the two-dimensional object tracking module 21 also includes a filtering model for filtering tiny target objects in the image. In this embodiment, the two-dimensional object tracking module 21 is a model that has been pre-trained with sample data.

The six-dimensional object attitude estimation module 22 may be, but is not limited to, including (using) the SAM-6D algorithm. SAM-6D is a zero-sample six-dimensional (6D) attitude estimation framework that does not require prior training with sample data. By given a CAD model of any target object, SAM-6D can perform instance segmentation and attitude estimation of the target object from the RGB image and its corresponding depth map. Therefore, in this embodiment, the six-dimensional object attitude estimation module 22 will pre-obtain the CAD model corresponding to each object in the RGB image to be used for six-dimensional object attitude estimation. In addition, the processing unit 2 of this embodiment allows the two-dimensional object tracking module 21 and the six-dimensional object attitude estimation module 22 to perform asynchronous operations through a multithreading mechanism to improve overall processing performance.

The six-dimensional object pose tracking module 23 in this embodiment may be, but is not limited to, including (using) the object tracking technology in the Foundation Pose algorithm published by NVIDIA. And the six-dimensional object posture tracking module 23 applies a dynamic batch inference mechanism (dynamic batch inference) to simultaneously track multiple objects in the image. Dynamic batch inference is a technique used in the online inference phase of deep learning models. The main purpose is to improve performance and allow hardware resources such as GPU and TPU in the processing unit 2 to fully utilize the hardware parallel computing capabilities to achieve a better balance between throughput and latency, so that the processing unit 2 can be used more efficiently.

In addition, in order to illustrate the effectiveness of this embodiment, before performing the process of FIG. 1 , it is assumed that the object list 11 is empty at the beginning, and at this time the processing unit 2 receives the input of the first RGB image P1 and its corresponding depth map D1, as shown in FIG. 3 . As shown in FIG. 4 , the first RGB image P1 contains three kinds of objects, including two rectangular objects 41, 42, a cube object 43, and two cylindrical objects 44, 45. Then, as shown in step S1 of FIG. 1 , the processing unit 2 executes the two-dimensional object tracking module 21 to perform object detection, tracking, and classification on the first RGB image P1, thereby generating a first tracking result image T1 (which can be temporarily stored in the storage unit 1) as shown in FIG. 3 and a two-dimensional object tracking result T11 (which can be temporarily stored in the storage unit 1) corresponding to each of the objects 41-45 in the first RGB image P1.

As shown in FIG4 , the first tracking result image T1 indicates an identification code (e.g., ID1, ID2, ID3, ID4, ID5) of each of the objects 41 to 45 detected in the first RGB image P1, a classification name (e.g., the classification name of the rectangular objects 41 and 42 is a, the classification name of the cube object 43 is b, and the classification name of the cylindrical objects 44 and 45 is c), a bounding box (a dotted box defining the boundary of the object as shown in FIG4 ) and a mask (a translucent layer defining the surface range of the object, not shown), and each of the two-dimensional object tracking results T11 includes the corresponding identification code, the category name, the bounding box and the mask of the object.

Next, as shown in step S2 of FIG. 1 , the processing unit 2 executes the six-dimensional object posture estimation module 22, compares the object list 11 read from the storage unit 1 with the two-dimensional object tracking results T11, and determines whether the identification code included in one of the two-dimensional object tracking results T11 (for example, at least one or more of them) does not appear in the object list 11. If not, it means that the identification codes included in the two-dimensional object tracking results T11 have all appeared in the object list 11, and then directly enters step S4; if yes, it means that at least one of the two-dimensional object tracking results T11 has appeared in the object list 11. If the identification code included in one of the two-dimensional object tracking results T11 to which the identification code not appearing in the object list 11 belongs and the first depth map D1 corresponding to the first RGB image P1, the six-dimensional object posture estimation module 22 performs six-dimensional posture estimation on the object corresponding to the identification code not appearing in the object list 11 in the first RGB image P1 according to the two-dimensional object tracking result T11 to which the identification code not appearing in the object list 11 belongs, and records the corresponding six-dimensional posture estimation result including the identification code in the object list 11, and then enters step S4.

At this time, since the object list 11 is empty, in step S3, the 6D object pose estimation module 22 performs 6D pose estimation on all objects 41-45 in the first RGB image P1, and records each of the 6D pose estimation results T12 corresponding to each of the objects in the object list 11; therefore, when this step S3 is completed, the object list 11 records the 6D pose estimation results T12 of all objects 41-45 in the first RGB image P1, as shown in FIG5. Each of the 6D pose estimation results T12 includes the corresponding identification code (e.g., ID, ID2, ...), category name (e.g., a, b, c), 2D pose information, 6D pose information, grid, grid tensor, grid diameter, and model center of the object.

Next, as in step S4, the 6D object pose estimation module 22 determines whether the identification code included in one of the 6D pose estimation results T12 (e.g., at least one or more of them) included in the object list 11 does not correspond to one of the 2D object tracking results T11. If so, it means that the object list 11 contains a 6D pose estimation result of an object that does not exist in the first RGB image P1. Then, as in step S5, the 6D object pose estimation module 22 determines whether the identification code included in one of the 6D pose estimation results T12 (e.g., at least one or more of them) included in the object list 11 does not correspond to one of the 2D object tracking results T11. If so, it means that the object list 11 contains a 6D pose estimation result of an object that does not exist in the first RGB image P1. The estimation module 22 removes the six-dimensional posture estimation result to which the identification code does not appear in any of the two-dimensional object tracking results T11 from the object list 11, and then proceeds to step S6; and at this stage, because the identification codes of the six-dimensional posture estimation results T12 recorded in the object list 11 all correspond to the identification codes of the two-dimensional object tracking results T11, the judgment result of step S4 is no, and the process directly proceeds to step S6.

In step S6, the processing unit 2 executes the six-dimensional object posture tracking module 23, so that the six-dimensional object posture tracking module 23 performs six-dimensional posture tracking on the object in the first RGB image P1 according to all six-dimensional posture estimation results T12 included in the object list 11 and the first depth map D1 corresponding to the first RGB image P1.

Next, as shown in FIG3, when the second RGB image P2 and its corresponding second depth map D2 are input to the processing unit 2, step S1 is repeated, and the two-dimensional object tracking module 21 performs object detection, tracking and classification on the input second RGB image P2, and accordingly generates a second tracking result image T2 and a two-dimensional object tracking result T21 corresponding to each object in the second RGB image P2, as shown in FIG3 and FIG6. And as shown in FIG4 and FIG6, in this embodiment, the second RGB image P2 has a newly added cube object 46 compared to the first RGB image P1, so the two-dimensional object tracking results T21 will include the two-dimensional object tracking results of the newly added cube object 46 (including the identification code ID6 and the classification name b).

Then, step S2 is performed, and the six-dimensional object posture estimation module 22 compares the two-dimensional object tracking results T21 with the object list 11. At this time, the six-dimensional object posture estimation module 22 determines that the identification code (ID6) included in one of the two-dimensional object tracking results T21 (i.e., the two-dimensional object tracking result of the newly added cube object 46) does not appear in the six-dimensional posture estimation results T12 in the object list 11. Therefore, step S3 is performed. The two-dimensional object tracking result of the cube object 46 and the second depth map D2 corresponding to the second RGB image P2 are used to perform six-dimensional posture estimation on the newly added cube object 46 in the second RGB image P2, and a six-dimensional posture estimation result T22 corresponding to the newly added cube object 46 and including the identification code (ID6) of the cube object 46 is generated, and the six-dimensional posture estimation result T22 is recorded (added) in the object list 11, as shown in FIG7 .

Then, the six-dimensional object posture estimation module 22 executes step S4, and after determining that the identification codes included in the six-dimensional posture estimation results T12 and T22 included in the object list 11 all appear in the two-dimensional object tracking results T21, step S6 is performed, and the six-dimensional object posture tracking module 23 performs six-dimensional posture tracking on the objects 41~46 in the second RGB image P2 according to the six-dimensional posture estimation results T12 and T22 included in the object list 11 and the second depth map D2 corresponding to the second RGB image P2.

It can be seen from this that the present embodiment will refer to the two-dimensional object tracking results of the RGB image, and only perform six-dimensional posture estimation on the object or objects corresponding to the one or more identification codes included in the two-dimensional object tracking results but not appearing in the object list 11 (i.e., the objects newly added to the RGB image compared to the previous RGB image), and will not perform posture estimation again on the one or more objects corresponding to the one or more identification codes included in both the two-dimensional object tracking results and in the object list 11. In this way, in the process of six-dimensional object posture tracking of the RGB image, the time for estimating the six-dimensional object posture can be shortened, hardware computing resources can be saved, and the performance of six-dimensional object posture tracking can be improved.

Furthermore, as shown in FIG3 , when the third RGB image P3 and its corresponding third depth map D3 are input to the processing unit 2, step S1 will be repeated again, and the two-dimensional object tracking module 21 will detect, track and classify the input third RGB image P3, and generate a third tracking result image T3 and a two-dimensional object tracking result T31 corresponding to each object in the third RGB image P3. And as shown in FIG8 , in this embodiment, the third RGB image P3 lacks one rectangular object 42 compared to the second RGB image P2 shown in FIG6 . Therefore, the two-dimensional object tracking results T31 will lack the two-dimensional object tracking results of the rectangular object 42.

Then, step S2 is performed, the six-dimensional object posture estimation module 22 compares the two-dimensional object tracking results T31 with the object list 11 shown in FIG. 7, and determines that the identification codes included in the two-dimensional object tracking results T31 all appear in the six-dimensional posture estimation results T12 and T22 in the object list 11, and then step S4 is performed. In step S4, the six-dimensional object posture estimation module 22 determines that among the six-dimensional posture estimation results T12 and T22 included in the object list 11 shown in FIG. 7, the six-dimensional posture estimation corresponding to the rectangular object 42 When the identification code (ID2) included in the result does not appear in the two-dimensional object tracking results T31, step S5 is performed to remove the six-dimensional posture estimation result corresponding to the rectangular object 42 from the object list 11, as shown in Figure 9, and then step S6 is performed, and the six-dimensional object posture tracking module 23 performs six-dimensional posture tracking on the objects 41, 43~46 in the third RGB image P3 according to the six-dimensional posture estimation results T12, T22 included in the object list 11 of Figure 9 and the third depth map D3 corresponding to the third RGB image P3.

It can be seen that each RGB image input to the processing unit 2 must go through the above step S1, and after being judged by step S2, proceed to step S3, S4 or directly proceed to step S4, and then proceed to step S5, S6 or directly proceed to step S6 after being judged by step S4. In addition, as shown in FIG. 10, steps S4 and S5 can also be swapped with steps S2 and S3, that is, after step S1, steps S4 and S5 can be executed first and then steps S2 and S3 can be executed.

In addition, since the six-dimensional object posture estimation module 22 generates the six-dimensional posture estimation results T12, T22 in the object list 11 based on (with reference to) the two-dimensional object tracking results T11, T21, T31 pre-generated by the two-dimensional object tracking module 21, the six-dimensional object posture tracking module 23 will be more accurate when performing object posture tracking on the RGB image based on the six-dimensional posture estimation results T12, T22 in the object list 11.

In summary, the above embodiment uses the two-dimensional object tracking module 21 to detect, track and classify objects on the input RGB image, and generates two-dimensional object tracking results corresponding to each object in the RGB image. The six-dimensional object posture estimation module 22 compares the two-dimensional object tracking results with the object list to find out the objects that exist in the two-dimensional object tracking results but not in the object list. The object in the RGB image corresponds to an identification code, and only the object in the RGB image that corresponds to the identification code that does not appear in the object list is estimated in six dimensions. Compared with the existing six-dimensional object posture tracking technology, in addition to saving software and hardware computing resources, shortening the computing time of six-dimensional posture estimation and accelerating the speed and performance of six-dimensional object posture tracking, the effect and purpose of the present invention are indeed achieved.

However, the above is only an example of the implementation of the present invention, and it should not be used to limit the scope of the implementation of the present invention. All simple equivalent changes and modifications made according to the scope of the patent application of the present invention and the content of the patent specification are still within the scope of the patent of the present invention.

S1~S6: Steps 100: Six-dimensional object posture tracking device 1: Storage unit 11: Object list 2: Processing unit 21: Two-dimensional object tracking module 22: Six-dimensional object posture estimation module 23: Six-dimensional object posture tracking module P1: First RGB image D1: First depth map P2: Second RGB image D2: Second depth map P3: Third RGB image D3: Third depth map T1: First tracking result image T2: Second tracking result image T3: Third tracking result image T11, T21, T31: Two-dimensional object tracking results T12, T22: Six-dimensional posture estimation results 41, 42: (rectangular) object 43, 46: (Cube) Objects 44, 45: (Cylinder) Objects

Other features and functions of the present invention will be clearly shown in the implementation method with reference to the drawings, in which: FIG. 1 is the main process steps of an embodiment of the six-dimensional object posture tracking method of the present invention; FIG. 2 is a schematic diagram of the software and hardware blocks included in an embodiment of the six-dimensional object posture tracking device of the present invention; FIG. 3 illustrates the tracking results generated by the two-dimensional object tracking module of the present embodiment tracking the RGB image; FIG. 4 illustrates the first RGB image of the present embodiment and its corresponding first tracking result image; FIG. 5 illustrates a schematic diagram of updating the object list according to the two-dimensional object tracking results of the first RGB image of the present embodiment; FIG. 6 illustrates the second RGB image of the present embodiment and its corresponding second tracking result image; FIG. 7 is a schematic diagram illustrating the updating of the object list according to the two-dimensional object tracking results of the second RGB image in the present embodiment; FIG. 8 is a schematic diagram illustrating the third RGB image and its corresponding third tracking result image in the present embodiment; FIG. 9 is a schematic diagram illustrating the updating of the object list according to the two-dimensional object tracking results of the third RGB image in the present embodiment; and FIG. 10 is another process step of the present embodiment.

S1~S6: Steps

Claims (13)

A six-dimensional object posture tracking method is applied to a six-dimensional object posture tracking device; the method comprises: A two-dimensional object tracking module of the six-dimensional object posture tracking device detects, tracks and classifies an input RGB image, and generates a two-dimensional object tracking result corresponding to each object in the RGB image, each of which includes an identification code and a bounding box of the corresponding object; A six-dimensional object posture estimation module of the six-dimensional object posture tracking device compares the two-dimensional object tracking results with an object list, and when it is determined that the identification code included in one of the two-dimensional object tracking results does not appear in the object list, the six-dimensional object posture estimation module performs six-dimensional posture estimation on the object corresponding to the identification code that does not appear in the object list in the RGB image based on the two-dimensional object tracking result to which the identification code that does not appear in the object list belongs and a depth map corresponding to the RGB image, and records the corresponding six-dimensional posture estimation result in the object list; A six-dimensional object posture tracking module of the six-dimensional object posture tracking device performs six-dimensional object posture tracking on the object in the RGB image according to all six-dimensional posture estimation results contained in the object list and the depth map corresponding to the RGB image. A six-dimensional object posture tracking method as described in claim 1, wherein, when the six-dimensional object posture estimation module determines that the identification code included in one of the six-dimensional posture estimation results in the object list does not appear in one of the two-dimensional object tracking results, the six-dimensional object posture estimation module removes the six-dimensional posture estimation result to which the identification code that does not appear in one of the two-dimensional object tracking results belongs from the object list. As described in claim 1, the six-dimensional object posture tracking method, wherein the two-dimensional object tracking module and the six-dimensional object posture estimation module perform asynchronous operations through a multi-thread mechanism. A six-dimensional object pose tracking method as described in claim 1, wherein the six-dimensional object pose estimation module performs six-dimensional pose estimation on the object in the RGB image corresponding to the identification code that does not appear in the object list based on a CAD model corresponding to the object in the RGB image corresponding to the identification code that does not appear in the object list. A six-dimensional object posture tracking method as described in claim 1, wherein the six-dimensional object posture tracking module simultaneously tracks multiple objects in the RGB image through a dynamic batch inference mechanism. A six-dimensional object posture tracking method as described in claim 1, wherein each of the two-dimensional object tracking results also includes the corresponding class name and mask of each of the objects, and each of the six-dimensional posture estimation results includes the corresponding identification code, class name, two-dimensional posture information, six-dimensional posture information, grid, grid tensor, grid diameter and model center of the object. A six-dimensional object posture tracking device comprises: a storage unit storing an object list; and a processing unit comprising a two-dimensional object tracking module, a six-dimensional object posture estimation module and a six-dimensional object posture tracking module; wherein the processing unit executes the two-dimensional object tracking module to detect, track and classify an input RGB image, and generates a two-dimensional object tracking result corresponding to each object in the RGB image, each of the two-dimensional object tracking results comprising an identification code and a bounding box of the corresponding object; The processing unit executes the six-dimensional object posture estimation module to compare the two-dimensional object tracking results with the object list stored in the storage unit, and when it is determined that the identification code included in one of the two-dimensional object tracking results does not appear in the object list, the six-dimensional object posture estimation module performs six-dimensional posture estimation on the object corresponding to the identification code that does not appear in the object list in the RGB image based on the two-dimensional object tracking result to which the identification code does not appear in the object list belongs and a depth map corresponding to the RGB image, and records a corresponding six-dimensional posture estimation result in the object list; The processing unit executes the six-dimensional object posture tracking module so that it performs six-dimensional object posture tracking on the object in the RGB image according to all six-dimensional posture estimation results contained in the object list and the depth map corresponding to the RGB image. A six-dimensional object posture tracking device as described in claim 7, wherein, when the six-dimensional object posture estimation module determines that the identification code included in one of the six-dimensional posture estimation results in the object list does not appear in one of the two-dimensional object tracking results, the six-dimensional object posture estimation module removes the six-dimensional posture estimation result to which the identification code that does not appear in one of the two-dimensional object tracking results belongs from the object list. The six-dimensional object posture tracking device as described in claim 7, wherein the two-dimensional object tracking module and the six-dimensional object posture estimation module perform asynchronous operations through a multi-thread mechanism. A six-dimensional object posture tracking device as described in claim 7, wherein the six-dimensional object posture estimation module performs six-dimensional posture estimation on the object in the RGB image corresponding to the identification code that does not appear in the object list based on a CAD model corresponding to the object in the RGB image corresponding to the identification code that does not appear in the object list. A six-dimensional object posture tracking device as described in claim 7, wherein the six-dimensional object posture tracking module simultaneously tracks multiple objects in the RGB image through a dynamic batch inference mechanism. A six-dimensional object posture tracking device as described in claim 7, wherein each of the two-dimensional object tracking results also includes the corresponding class name and mask of each of the objects, and each of the six-dimensional posture estimation results includes the corresponding identification code, class name, two-dimensional posture information, six-dimensional posture information, grid, grid tensor, grid diameter and model center of the object. A computer-readable recording medium stores a software program including a two-dimensional object tracking module, a six-dimensional object posture estimation module and a six-dimensional object posture tracking module. When the software program is loaded and executed by a computer device, the computer device can complete the six-dimensional object posture tracking method described in any one of claims 1 to 6.

Images