TWI756996B - Automatic bio-specimen inspection system and inspection method thereof as well as non-volatile computer readable storage media - Google Patents

Abstract

An automatic bio-specimen inspection system includes an inspection device, an image processing module, a spatial learning module, a path generation module, and a motion device. The inspection device is used to approach an inspection area to perform an inspection operation. The image processing module is used to capture and process a plurality of two-dimensional images of the inspection area. The spatial learning module generates three-dimensional spatial information of the inspection area according to the two-dimensional images. The path generation module generates inspection path information based on the three-dimensional space information. The motion device moves the inspection device to the inspection area according to the inspection path information for inspection operation.

Description

Biological sample automatic collection and inspection system, collection and inspection method and non-volatile electricity Brain-readable recording medium

This disclosure is about a bio-specimen collection and inspection system, its collection and inspection method and application, especially an automated bio-specimen collection and inspection system and its collection and inspection method and application.

With the spread of novel coronavirus pneumonia (COVID-19) and other respiratory diseases, in order to find out the confirmed infected person, medical staff need to first take the suspected infected person for inspection. Reporting, treatment and tracking. For respiratory diseases, the examination methods are generally conducted by medical staff wearing protective clothing to examine the subjects with bare hands.

However, the face-to-face operation between medical staff and subjects easily increases the risk of infection for medical staff during testing. At present, safety testing kiosks or transparent acrylic baffles have been used to isolate medical staff from subjects to reduce the risk of medical staff sampling. However, when a large number of suspected cases appear, the inspection operation is carried out manually. Manpower and time are still a big waste. Automating the collection and inspection operation and avoiding the use of manpower has become a hot topic in this technical field.

However, the sampling location is usually located in the body chamber of the living being. To automate the sampling operation, it is necessary to know the three-dimensional spatial information of the sampling site, so that the sampling device, such as the swab, can be inserted into the Inspection site. Although the existing technology has been able to use, for example, Computed Tomography (CT) scanning, Magnetic Resonance Imaging (MRI) scanning and X-ray (X-Ray Imaging) imaging and other equipment, to construct the detection site three-dimensional space. But these instruments are not only expensive, bulky, and complicated to operate, but also carry the risk of radiation exposure.

Although the current stereo optical photography technology is quite mature, it can be used to directly capture the stereo image of the inspection site. However, the lens size of stereo optical photography is large, and it is not suitable for extending into any body cavity, and the lens with too large size is very likely to hinder the inspection operation.

In addition, in order to effectively apply medical human resources, all walks of life are striving for automatic inspection solutions. The difficulties of the automatic inspection plan include at least (1) the lack of depth information in the 2D image, which affects the accuracy and safety of the automatic inspection system; (2) the narrow oral area makes it difficult to use commercially available 3D imaging devices to create three-dimensional oral images; (3) The collection and inspection method is too simple, and it is difficult to apply to various situations.

Therefore, there is a need to provide an advanced automatic collection and inspection system for biological specimens, a collection and inspection method and application thereof, to solve the problems faced by the prior art.

In response to the above problems, this disclosure is in response to the need for automatic collection and inspection in order to effectively prevent the epidemic under the influence of the new crown pneumonia. At present, all walks of life are trying their best to develop automatic sampling and inspection solutions. However, automatic sampling and inspection mostly use two-dimensional images combined with force feedback devices to identify the position of the subject's mouth and then use a unified robotic arm for sampling and inspection. It is easy to have safety concerns in the collection and inspection; and the collection and inspection method is too simple, and its accuracy is also questionable. The present disclosure is based on the technical means of visual rapid reconstruction of oral 3D point cloud combined with deep learning of the robot arm's inspection posture to achieve the effect of rapid inspection by the robot arm's humanoid method.

This disclosure solves the problem of sampling safety by generating deep images from two-dimensional images, and applies the neural network learning method to learn the manual sampling and inspection methods, so that the automatic inspection and inspection system can generate a robotic arm sampling and inspection path imitating human methods, which is suitable for various Sampling situational needs.

An embodiment of the present disclosure provides an automatic collection and inspection system for biological specimens, including a collection and inspection device, an image processing module, a spatial learning module, a path generation module, and a mobile device. The collection and inspection device is used to approach the collection and inspection area of the living body to perform the collection and inspection operation. The image processing module is used for capturing and processing a plurality of two-dimensional images of the sampling area. The spatial learning module generates three-dimensional spatial information of the sampling area according to the plurality of two-dimensional images. The path generation module generates inspection path information according to the three-dimensional space information. The mobile device moves the collection and inspection device to the collection and inspection area to perform the collection and inspection operation according to the collection and inspection path information.

Another embodiment of the present disclosure provides an automatic biological sample collection and inspection method, including the following steps: using a collection and inspection device to approach a collection and inspection area of the biological sample. Use the image processing module to capture and process a plurality of 2D images of the sampling area. use empty The inter-learning module generates three-dimensional spatial information of the sampling area according to a plurality of two-dimensional images. Use the path generation module to generate inspection path information according to the three-dimensional space information. Use the mobile device to move the collection and inspection device to the collection and inspection area according to the collection and inspection path information to perform the collection and inspection operation.

Yet another embodiment of the present disclosure provides a non-volatile computer-readable recording medium, the non-volatile computer-readable recording medium stores a program code, and the program code can be executed by a processor to execute the above-mentioned method for automatic sampling of biological specimens .

Still another embodiment of the present disclosure provides a non-volatile computer-readable recording medium, wherein the non-volatile computer-readable recording medium stores a set of program codes, and the program codes can be controlled by a processor to be executed by an automatic biological specimen collection and inspection system Take the following steps: command the sampling device to approach the sampling area of the biological specimen. The image processing module is instructed to capture and process a plurality of two-dimensional images of the sampling area. Command the spatial learning module to generate three-dimensional spatial information of the inspection area according to a plurality of two-dimensional images. The command path generation module generates inspection path information according to the three-dimensional space information. Command the mobile device to move the sampling and inspection device to the sampling and inspection area to perform the sampling and inspection operation according to the sampling and inspection path information.

For example, the present disclosure can be combined with a robotic arm and a two-dimensional vision lens to perform automatic inspection on the mouth, nose, and throat of a human body or an organism; in addition, the present disclosure can use a plurality of two-dimensional images to generate a three-dimensional image, thereby collecting manual samples. Inspection methods and learning to generate automatic inspection paths for robotic arms. Furthermore, the present disclosure can use two-dimensional images of the oral cavity as a data source for training a three-dimensional learning network, thereby generating three-dimensional depth image information of the oral cavity and throat cavity. For example, loading multiple 2D images of the mouth, nose and throat into the trained stereo learning network and generating oral stereo image data to train the robotic arm The collection and inspection path generation network, the output results of which are combined with the manual collection and inspection path to train the identification neural network. When the identification results are not good, the robotic arm collection and inspection path generation network is retrained, and the operation is repeated until the two neural networks reach the same level. Zero-sum equilibrium, which can obtain a robotic arm automatic inspection path generation network that can produce human-like inspection methods. Finally, through the three-dimensional learning network and the robot arm collection and inspection path generation network, the automatic collection and inspection path of the robot arm can be generated, which can replace manual labor and apply to various collection and inspection situation requirements for automatic collection and inspection.

10: Automatic collection and inspection system

11: Collection and inspection device

13: Mobile Devices

14: Collection area

112: Swab

113: Articulator

121: Image processing module

121a: Light source

121b: Image capture unit

121c: Image Processing Unit

121d: Pressure sensor

122: Spatial Learning Module

123: Path Generation Module

123a: Manual inspection path

123b: Virtual collection and inspection path

123c: Identification feedback

124: 2D Image

124’: 2D image

124b: 2D projected mesh feature map

125: Stereoscopic space information

125’: Stereoscopic space information

126: Collection and inspection path information

127: Stereo Learning Generative Network

127a: Image encoder

127b: Back Projection Module

127c: Recurrent Fusion Neural Networks

127d: Three-dimensional grid prediction module

127c: Projection Module

128: Identify the network

129: Acquisition and inspection path generation network

301: Training Phase

302: Automatic collection and inspection stage

G ^p : 3D mesh feature map

G ^o : 3D mesh feature map

:三維網格特徵圖

: 3D mesh feature map

In order to have a better understanding of the above-mentioned and other aspects of the present disclosure, the following specific embodiments are given and described in detail in conjunction with the accompanying drawings as follows: FIG. 1 is an automatic collection of biological samples according to an embodiment of the present disclosure. Check the block diagram of the system.

FIG. 2 is a schematic diagram illustrating a flow chart of converting a plurality of 2D images into stereoscopic space information by a stereoscopic learning generation network according to an embodiment of the present disclosure.

FIG. 3 is a schematic flowchart of generating a virtual sampling and inspection route by using the sampling and inspection route generation network of the route generation module according to an embodiment of the present disclosure.

FIG. 4 is a schematic flowchart of generating an automatic sampling and inspection path by using a three-dimensional learning generation network and a collection and inspection path generation network according to an embodiment of the present disclosure.

The present disclosure provides an automatic collection and inspection system for biological specimens, a collection and inspection method, and applications thereof, which can achieve the automation of biological specimen collection and inspection to save medical manpower 's consumption. In order to make the above-mentioned embodiments and other objects and features of this specification more obvious and easy to understand, several embodiments are given below and described in detail with the accompanying drawings.

However, it must be noted that these specific implementation cases and methods are not intended to limit the present invention. The present invention may still be practiced with other features, elements, methods and parameters. The preferred embodiments are provided only to illustrate the technical features of the present invention, and not to limit the scope of the present invention. Those with ordinary knowledge in the technical field will be able to make equivalent modifications and changes based on the description of the following specification without departing from the spirit and scope of the present invention. In different embodiments and drawings, the same elements will be represented by the same element symbols.

Please refer to FIG. 1. FIG. 1 is a block diagram of an automatic biological specimen collection and inspection system 10 according to an embodiment of the present disclosure. The automatic biological specimen collection and inspection system 10 includes: a collection and inspection device 11 , an image processing module 121 , a spatial learning module 122 , a path generation module 123 , and a mobile device 13 .

The sampling device 11 is used to approach the sampling area 14 of the living body to perform the sampling operation. For example, in some embodiments of the present disclosure, the sampling device 11 may include a swab 112 (or a swab 112 (or swab) used to enter the oral, nose and throat cavity of the living being, and used to collect the specimen in the nose, throat, or throat. For example, sample collection rods, sample collectors, etc., sample collection and inspection tools with the same function). In order to facilitate the passage of the swab 112 (or, for example, a sampling stick, a sample collector, etc.) through the oral, nose and throat cavity of the subject, the sampling device 11 may additionally include an articulator 113 for attaching the subject's The mandible and maxilla are propped apart to enlarge the visual field of the oral space, and provide a stable channel for the swab 112, the sampling stick, and the entrance and exit of the nasal and throat cavity of the sample collector.

However, the sampling device 11 is not limited to this. Any device, component, element, instrument or consumable for sampling other chambers, lumens, channels or tissue parts in the body, or any position outside the body of the biological specimen are included in the sampling and testing device 11. in the spiritual realm.

The mobile device 13 is electrically connected with the image processing module 121, the spatial learning module 122 and the path generation module 123, and the mobile device 13 accepts an instruction (for example, moving path information) from the path generation module 123, and will collect and detect The device 11 moves to the sampling area 14 . In one embodiment, the moving device 13 may be a robotic arm. The swab 112 of the testing device 11 can be directly disposed at the end of the robotic arm, and the robotic arm inserts the swab 112 into the oral, nose and throat cavity of the subject.

Please refer to FIGS. 1 , 2 and 3 simultaneously, the image processing module 121 is used for capturing and processing a plurality of two-dimensional images 124 of the sampling area 14 . The spatial learning module 122 generates the three-dimensional spatial information 125 of the inspection area 14 according to the captured two-dimensional images 124 . The path generation module 123 generates the inspection path information 126 according to the three-dimensional space information 125 .

In some embodiments of the present disclosure, the image processing module 121 includes a light source 121a, an image capture unit 121b and an image processing unit 121c. The light source 121 a may include (but not limited to) a light-emitting diode (LED) element, which may be used to provide light illumination in the detection area 14 . The image capture unit 121b may be a light detection unit, which at least includes (but is not limited to) a photoelectric conversion element, such as a photodiode (photodiode), a charge-coupled device (CCD, charge-coupled device) or an intensified charge coupling device (ICCD, Intensified CCD) can be used to capture the optical information such as brightness, gray scale, color (RGB) of the detection area 14, and the optical information is converted into a two-dimensional image 124 by the image processing unit 121c. The image processing unit 121c is not limited, and any software, hardware, firmware or any combination of the three for processing images does not exceed the spiritual scope of the image processing unit 121c.

In this embodiment, the image capturing unit 121b is adjacent to one side of the swab 112 fixed to the sampling device 11, and can move in three-dimensional space with the swab 112 in the oral, nose and throat cavity. In one embodiment, the image processing module 121 may further include a lens driver (not shown) for driving the lens of the image capturing unit 121b to capture a plurality of different two-dimensional images 124 from different viewing angles.

Please refer to FIG. 2 . FIG. 2 is a schematic flowchart of converting a plurality of 2D images 124 into stereoscopic space information 125 by a stereoscopic learning generation network 127 according to an embodiment of the present disclosure. In this embodiment, the spatial learning module 122 includes a three-dimensional learning generation network 127 for generating the three-dimensional spatial information 125 of the sampling area 14 according to the plurality of two-dimensional images 124 .

(n=1-f)。 Specifically, first, a plurality of 2D projected mesh feature maps 124b are generated from the plurality of 2D images 124 via the image encoder 127a. After that, a feature matching stage is performed, and the features of these two-dimensional projected grid feature maps are respectively projected into a three-dimensional grid by using an unprojection module 127b to form a plurality of three-dimensional grid feature maps

(n=1-f).

(n=1-f)融合形成三維網格特徵圖G^p。再以立體網格預測模組(3D grid reasoning)127d，使用立體捲積神經網路(3D convolutional neural networks)來計算相匹配成本積(matching cost volume)，將三維網格特徵圖G^p的匹配成本積解碼成為一個具有體積/表面/視差(3D volume/surface/disparity maps)等立體空間資訊的三維網格特徵圖G^o。 Continuing, the recursive neural network (recurrent neural network) 127c is used to compare the features of different three-dimensional grid feature maps, and these three-dimensional grid feature maps are combined.

(n=1-f) is fused to form a three-dimensional mesh feature map ^Gp . Then, the 3D grid reasoning module (3D grid reasoning) ^127d is used to calculate the matching cost volume using a 3D convolutional neural network, and the matching cost volume of the 3D grid feature map Gp is calculated. The cost product is decoded into a 3D mesh feature map G ^o with stereo spatial information such as volume/surface/disparity maps (3D volume/surface/disparity maps).

Then, according to the three-dimensional space information of the three-dimensional grid feature map G ^o , the projection module 127e is used to simulate a three-dimensional voxel occupancy grids of the sampling area 14. The three-dimensional image includes a three-dimensional space Information 125. In this embodiment, the three-dimensional spatial information 125 includes a plurality of depth information and a plurality of color information in the form of RGBD.

Next, as shown in FIG. 4 , the path generation module 123 can generate the inspection path information 126 according to the three-dimensional space information 125 obtained through the FIG. 2 .

As shown in FIG. 1 , FIG. 3 and FIG. 4 , in this embodiment, the path generation module 123 includes a collection and inspection path generation network 129 , which can generate collection and inspection path information 126 and provide it to the mobile device 13 , so that The moving device 13 moves the swab 112 of the sampling device 11 to the sampling area 14 in the oral, nose and throat cavity according to the sampling path information 126 .

Please refer to FIG. 3 first. FIG. 3 is a schematic flowchart of generating a virtual inspection path using the inspection path generation network 129 of the path generation module 123 according to an embodiment of the present disclosure. The process of generating the virtual inspection path may be referred to as "training stage 301".

In the training stage 301, firstly, a plurality of physical inspection operations are performed on a single or multiple organisms to obtain a plurality of two-dimensional images 124', and a plurality of three-dimensional spatial information 125' are generated by the spatial learning module 122; record During the physical collection and inspection operation, the plurality of manual collection and inspection paths 123a of the collection and inspection device 11 record the three-dimensional space information 125' corresponding to each manual collection and inspection path 123a. According to the plurality of manual inspection paths 123a and the plurality of three-dimensional spatial information 125', a neural network-like network is used to construct a collection and inspection path generation network 129. Specifically, each physical inspection operation will obtain a set of two-dimensional images 124', and each set of two-dimensional images 124' will correspond to a three-dimensional spatial information 125'; that is, each manual inspection path 123a will Corresponding to a three-dimensional space information 125'.

In this embodiment, for example, qualified medical care or testing personnel can actually use the swab 112 of the testing device 11 (including the light-emitting diode light source 121a and the image capturing unit 121b ) for the same or a plurality of different mouths, noses and throats The inspection operation is performed in the cavity, and the image processing module 121 collects and records the manual inspection path 123a and the two-dimensional image 124' of the swab 112 of the inspection device 11 in multiple physical inspection operations.

Then, the spatial module 122 generates different three-dimensional spatial information 125' according to the plurality of two-dimensional images 124' of the mouth, nose and throat cavity collected and recorded by the image processing module 121; network 129, and generate a virtual sampling path 123b.

In one embodiment, a pressure sensor 121d may be further included, disposed on the swab 112 of the sampling device 11 to sense the contact stress between the swab 112 of the sampling device 11 and the sampling region 14, thereby It is completed by confirming whether it is actually in contact with the living body The magnitude of the contact stress is detected or sensed, and the contact stress can be fed back to the path generation module 123 .

Please continue to refer to FIG. 3, continue to use the identification network 128 to identify the virtual sampling and inspection path 123b and the manual sampling and inspection path 123a, and provide an identification feedback 123c to the sampling and inspection path generating network 129 according to the identification result, so as to adjust and optimize the sampling and inspection path. The detection path generation network 129 enables the subsequently generated virtual detection path to be closer to the manual detection path. The above is the training stage, and the adjusted and optimized sampling and inspection path generation network can be obtained for the use of the automatic sampling and inspection stage.

Please refer to FIG. 4 . FIG. 4 is a schematic flowchart of generating an automatic sampling and inspection path using the stereo learning generation network and the sampling and inspection path generation network 129 according to an embodiment of the present disclosure. The process of generating the automatic inspection path may be referred to as the "automatic inspection phase 302".

In the automatic inspection stage 302, the image processing module 121 acquires a plurality of two-dimensional images 124 of the oral, nose and throat cavity of the current subject, and the spatial learning module 122 generates inspections according to the captured plurality of two-dimensional images 124 Stereoscopic spatial information 125 of area 14 . Next, the acquisition path generation network 129 of the path generation module 123 generates acquisition path information 126 according to the three-dimensional space information 125 , and the acquisition path information 126 includes at least one acquisition path. The route generation module 123 provides the inspection route information 126 to the mobile device 13 . It should be noted that, in the automatic sampling and inspection stage 302 , the sampling and inspection path generation network 129 is the adjusted and optimized sampling and inspection path generation network obtained in the training stage 301 .

Then, the moving device 13 moves the swab 112 of the sampling device 11 to the sampling area 14 according to the sampling and inspection path to perform the automatic sampling and inspection operation, and the automatic sampling and inspection stage is completed. In one embodiment, the mobile device 13 may be a robotic arm, that is, a robotic arm is used to perform the automatic inspection phase.

Various software, application programs, data or computing logic used by the above-mentioned biological specimen automatic collection and inspection system 10 can be integrated to form a non-volatile computer-readable recording medium and stored in a non-volatile memory storage device (eg, a magnetic disk, an optical disk) , flash memory or other suitable integrated circuits) or computer networks. And the non-volatile computer-readable recording medium has a set of program codes, through which a processor (such as a central processing unit of a computer) can control the aforementioned automatic biological specimen collection and inspection system 10, and execute the following steps: for example , instruct the image processing module 121 to capture and process a plurality of two-dimensional images 124 of the inspection area 14 ; instruct the spatial learning module 122 to generate three-dimensional spatial information of the inspection area 14 according to the captured plurality of two-dimensional images 124 125; instructing the path generation module 123 to generate collection and inspection path information 126 according to the three-dimensional space information 125; and instructing the mobile device 13 to move the swab 112 of the collection and inspection device 11 to the collection and inspection position 14 according to the collection and inspection path information 126 for automatic collection and inspection operate. In detail, the execution of the above collection and inspection system does not require manual operation.

According to the above-mentioned embodiments, the present disclosure provides an automatic collection and inspection system for biological samples, a collection and inspection method, and applications thereof. The biological specimen automatic collection and inspection system includes a collection and inspection device, an image processing module, a space learning module, a path generation module and a mobile device. The image processing module is used for capturing and processing a plurality of two-dimensional images of the sampling area. The spatial learning module generates a three-dimensional space of the sampling area according to a plurality of two-dimensional images. time information. The path generation module generates inspection path information according to the three-dimensional space information. The mobile device moves the collection and inspection device to the collection and inspection area to perform the collection and inspection operation according to the collection and inspection path information. Without additional expensive instruments and equipment, the purpose of automating the collection and inspection of biological samples can be achieved at a lower cost, effectively saving the cost of medical manpower.

Although the present invention has been disclosed above with various embodiments, it is not intended to limit the present invention. Anyone with ordinary knowledge in the technical field can make some changes and modifications without departing from the spirit and scope of the present invention. Therefore, the protection scope of the present invention shall be determined by the scope of the appended patent application.

10: Automatic collection and inspection system

11: Collection and inspection device

13: Mobile Devices

14: Collection area

112: Swab

113: Articulator

121: Image processing module

121a: Light source

121b: Image capture unit

121c: Image Processing Unit

121d: Pressure sensor

122: Spatial Learning Module

123: Path Generation Module

129: Acquisition and inspection path generation network

Claims (13)

An automatic collection and inspection system for biological specimens, comprising: a collection and inspection device for approaching a collection and inspection area of a living body to perform a collection and inspection operation; an image processing module for capturing and processing images of the collection and inspection area a plurality of two-dimensional images; a space learning module for generating a three-dimensional space information of the sampling and inspection area according to the plurality of two-dimensional images; a path generating module for generating a collection and inspection path information according to the three-dimensional space information; and a moving device, which moves the sampling device to the sampling and inspection area to perform the sampling and inspection operation according to the sampling and inspection path information. The automatic biological specimen collection and inspection system according to claim 1, wherein the spatial learning module includes a three-dimensional learning generation network for generating the three-dimensional spatial information according to the plurality of two-dimensional images. The automatic biological specimen collection and inspection system according to claim 1, wherein the path generation module includes a collection and inspection path generation network, and in an automatic collection and inspection stage, the collection and inspection path information is generated according to the three-dimensional space information. The automatic biological specimen collection and inspection system according to claim 1, wherein the collection and inspection area includes a mouth, nose and throat cavity, and the collection and inspection device includes an bite device and a collection and inspection rod. The automatic biological specimen collection and inspection system as claimed in claim 1, further comprising a pressure sensor disposed on the collection and inspection device for sensing a contact stress between the collection and inspection device and the collection and inspection area, and The contact stress is fed back to the path generation module. The automatic biological specimen collection and inspection system according to claim 1, wherein the image processing module includes an image capture unit adjacent to the collection and inspection device for capturing a plurality of two-dimensional images of the collection and inspection area; The plurality of two-dimensional images are captured by the image capturing unit from a plurality of different angles; the three-dimensional spatial information includes a plurality of depth information and a plurality of color information represented by an RGBD format. An automatic collection and inspection method for biological samples, comprising: using a collection and inspection device to approach a collection and inspection area of a living body; using an image processing module to capture and process a plurality of two-dimensional images of the collection and inspection area; a spatial learning module, which generates a three-dimensional spatial information of the sampling area according to the plurality of two-dimensional images; A path generation module is used to generate a collection and inspection path information according to the three-dimensional space information; and a moving device is used to move the collection and inspection device to the collection and inspection area to perform the collection and inspection operation according to the collection and inspection path information. The method for automatically collecting and inspecting biological specimens according to claim 7, wherein the plurality of two-dimensional images are obtained by shooting from a plurality of different angles by the image processing module. The method for automatically collecting and inspecting biological specimens as claimed in claim 7, wherein the step of generating the three-dimensional spatial information includes using a spatial learning generation network of the spatial learning module, and the three-dimensional spatial information includes an RGBD representation. a plurality of depth information and a plurality of color information for . The method for automatic collection and inspection of biological specimens according to claim 7, wherein the step of generating the collection and inspection path information includes using a collection and inspection path generation network of the path generation module to generate the collection and inspection path in an automatic collection and inspection stage. Collection path information. The automatic biological specimen collection and inspection method according to claim 7, wherein the generation of the collection and inspection path information includes using a collection and inspection path generation network, and the generation of the collection and inspection path information includes: manually The organism performs a plurality of entity collection and inspection operations to obtain the two-dimensional image of the plurality of groups; Generate the plurality of three-dimensional space information with the space learning module; when recording the plurality of physical inspection operations, the plurality of manual inspection paths of the inspection device, and record the corresponding three-dimensional space of each of the manual inspection paths information; according to the plurality of manual inspection paths and the plurality of three-dimensional space information, construct a collection and inspection path generation network in the path generation module; use the collection and inspection path generation network to generate a network according to the three-dimensional space information a virtual sampling and inspection path; and using an identification network to identify the virtual sampling and inspection path and the manual sampling and inspection path, and provide an identification feedback to the sampling and inspection path generating network according to the identification result. A non-volatile computer-readable recording medium, wherein the non-volatile computer-readable recording medium stores a program code, and the program code is executed by a processor to automatically request the biological sample described in any one of items 7 to 11. Collection method. A non-volatile computer-readable recording medium, wherein the non-volatile computer-readable recording medium stores a program code, and the program code is executed by a processor with the following steps: instructing a collection and inspection device to approach a collection and inspection of an organism an area; instructing an image processing module to capture and process a plurality of two-dimensional images of the sampling area; Command a spatial learning module to generate a three-dimensional space information of the inspection area according to the plurality of two-dimensional images; instruct a path generation module to generate a collection and inspection path information according to the three-dimensional space information; and instruct a mobile device , and move the collection and inspection device to the collection and inspection area to perform the collection and inspection operation according to the collection and inspection path information.

Images