WO2018120027A1 - Method and apparatus for detecting obstacles - Google Patents

Abstract

A method and apparatus for detecting obstacles, which relate to the technical field of artificial intelligence. The present invention aims to solve the problem in the existing technology wherein accuracy in detecting obstacles is limited. Said method for detecting obstacles comprises: acquiring a pixel image and a depth image respectively, which correspond to an area to be detected (101); determining a road surface area within the area to be detected and a first class obstacle within the road surface area according to the pixel classification of each pixel within the pixel image; and determining a second class obstacle within the area to be detected according to the depth image (102); determining a spatial location of the first class obstacle according to the pixel image and the depth image respectively, and determining a spatial location of the second class obstacle according to the depth image (103). Said method is applicable in a process of detecting obstacles.

Description

Obstacle detection method and device Technical field

The present application relates to the field of computer vision technology, and in particular, to an obstacle detection method and apparatus.

Background technique

In mobile robots and blind guide systems, obstacle avoidance is one of the basic functions necessary, and how to effectively detect obstacles on the road surface is a key problem that the obstacle avoidance system needs to solve.

At present, commonly used obstacle detection methods include non-visual detection methods based on infrared rays, ultrasonic waves, and non-stereoscopic vision detection methods based on a single camera. However, there are corresponding defects and deficiencies in these two methods. Among them, non-visual detection methods such as infrared rays and ultrasonic waves have limited detection accuracy, and can only detect large obstacles and cannot detect small obstacles, so safety is achieved. Poor; based on the obstacle detection method of a single camera, it is often necessary to specify the area of interest. For complex environments, it cannot automatically detect the road surface and obstacles.

Summary of the invention

The embodiment of the present application provides an obstacle detection method and device, which mainly solves the problem that the detection accuracy of obstacles existing in the prior art is limited.

To achieve the above objective, the embodiment of the present application adopts the following technical solutions:

In a first aspect, the present application provides an obstacle detection method, including:

Obtaining, respectively, a pixel image and a depth image corresponding to the to-be-detected area, where each pixel point corresponds to a pixel category, and each pixel point in the depth image corresponds to a depth value; according to each pixel point in the pixel image a pixel class determining a road surface area in the area to be detected and a first type of obstacle in the road surface area; and determining a second type of obstacle in the area to be detected according to the depth image; according to the pixel image And the depth image determines a spatial location of the first type of obstacle and determines a spatial location of the second type of obstacle based on the depth image.

In a second aspect, the present application provides an obstacle detecting apparatus, including: an acquiring unit, configured to respectively acquire a pixel image and a depth image corresponding to an area to be detected, where each pixel point corresponds to a pixel category, the depth Each of the pixel points in the image corresponds to the depth value; the obstacle determining unit is configured to determine a road surface area in the to-be-detected area and the road surface according to a pixel category of each pixel point in the pixel image acquired by the acquiring unit a first type of obstacle in the area; and determining, according to the depth image acquired by the acquiring unit, a second type of obstacle in the area to be detected; a position determining unit, configured to determine according to the pixel image and the depth image The spatial position of the first type of obstacle determined by the obstacle determination unit and the spatial position of the second type of obstacle determined by the obstacle determination unit according to the depth image.

In a third aspect, the present application provides a computer storage medium for storing computer software instructions comprising program code designed to perform the obstacle detection method of the first aspect.

In a fourth aspect, the present application provides a computer program product that can be directly loaded into an internal memory of a computer and includes software code, and the computer program can be loaded and executed by a computer to implement the obstacle detection described in the first aspect. method.

In a fifth aspect, the present application provides an electronic device, including: a memory, a communication interface, and a processor, the memory is configured to store computer executable code, and the processor is configured to execute the computer executable code control to perform the first aspect The obstacle detecting method, the communication interface is used for data transmission between the electronic device and an external device.

In a sixth aspect, the present application provides a robot, including the electronic device of the fifth aspect.

The solution provided by the present application obtains a pixel image and a depth image corresponding to the to-be-detected area, where each pixel point corresponds to a pixel category, and each pixel point in the depth image corresponds to a depth value; a pixel class of each pixel determines a road surface area in the area to be detected and a first type of obstacle in the road surface area; and determines a second type of obstacle in the area to be detected according to the depth image; and finally determines according to the pixel image and the depth image The spatial location of the first type of obstacle and the spatial location of the second type of obstacle based on the depth image, and various obstacles in the prior art Compared with the object detection method, the detection accuracy of the obstacle is limited. Compared with the obstacle image in the road surface area, such as a small obstacle in the road surface, the volume image can detect the volume in the area to be detected through the depth image. Large obstacles, therefore, can obtain a more comprehensive obstacle in the area to be detected; in addition, after detecting the obstacle, the present application can also output the spatial position of the obstacle, which is helpful for obstacle avoidance decision.

DRAWINGS

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings to be used in the embodiments or the prior art description will be briefly described below. Obviously, the drawings in the following description are only It is a certain embodiment of the present application, and other drawings can be obtained according to the drawings without any creative work for those skilled in the art.

1 is a schematic structural diagram of an obstacle avoidance system according to an embodiment of the present application;

2 is a schematic diagram of a depth image corresponding to an area to be detected and an area to be detected according to an embodiment of the present disclosure;

FIG. 3 is a schematic flowchart diagram of an obstacle detection method according to an embodiment of the present application;

4 is a schematic diagram of a pixel image and a depth image corresponding to a to-be-detected area and a to-be-detected area according to an embodiment of the present disclosure;

5 to FIG. 7 are schematic flowcharts of a method for detecting an obstacle provided by an embodiment of the present application;

8 to FIG. 10 are schematic structural diagrams of an obstacle detecting device according to an embodiment of the present application;

FIG. 11 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

detailed description

The system architecture and the service scenario described in the embodiments of the present application are for the purpose of more clearly explaining the technical solutions of the embodiments of the present application, and do not constitute a limitation of the technical solutions provided by the embodiments of the present application. The technical solutions provided by the embodiments of the present application are equally applicable to similar technical problems.

It should be noted that, in the embodiments of the present application, the words "exemplary" or "such as" are used to mean an example, an illustration, or a description. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be construed as An example or design is more preferred or advantageous. Rather, the use of the words "exemplary" or "such as" is intended to present the concepts in a particular manner.

It should be noted that, in the embodiment of the present application, "(English: of)", "corresponding (relevant)" and "corresponding" may sometimes be mixed. It should be noted that When the difference is not emphasized, the meaning to be expressed is the same.

The obstacle avoidance system is used to detect whether there is an obstacle in a specific area, and can be applied to a mobile robot or a guide blind system. Obstacle detection is the basic function of the obstacle avoidance system. The embodiment of the present application provides an obstacle avoidance system. As shown in FIG. 1 , the obstacle avoidance system 10 includes: an image collection device 11 , an obstacle detection device 12 , and a decision device 13 . Wherein, the image capturing device 11 is configured to collect image information of a specific region for performing obstacle detection, such as an image in front of the mobile robot, and transmit the collected image information to the obstacle detecting device 12; exemplarily, The image acquisition device includes one or more cameras that capture planar images, a binocular camera that collects stereoscopic images, and the like. The obstacle detecting device 12 is configured to process the image information transmitted by the image capturing device 11 to obtain information on whether or not the obstacle and the outline, size, position, type, and the like of the obstacle are included, and transmit the processed information to the decision device 13. The decision device 13 is configured to make a decision on how to avoid obstacles according to the information sent by the obstacle detecting device 12, and the decision device may be a device having a processing operation function, such as a server. The specific implementation of the image capturing device 11 and the determining device 13 can refer to the prior art, and details are not described herein again.

A depth image, also referred to as a range image, is an image that takes a distance (or depth) from an image collector, such as a binocular camera, to a point in a region to be detected as a pixel value. It can directly reflect the geometry of the visible surface of the object, that is, it can directly determine the contour of each object. The depth image can be calculated as point cloud data through coordinate conversion, and the point cloud data with rules and necessary information can also be inversely counted as depth image data. In the depth image, each pixel represents the distance from the object to the camera plane at the particular (x,y) coordinate in the field of view of the image collector. The purpose of detecting an obstacle can be achieved according to the distance in the depth image and the contour of each object. As shown in FIG. 2, the image to be detected shown in the left figure is photographed and processed to obtain a depth image shown in the right image. In the depth image, different brightness values indicate different distances, and the brighter the color, the more the target distance is. near. Combined with the depth image shown in Figure 2, Identify chairs, trash cans, and people as obstacles. The specific implementation principle of the depth image and the specific implementation process of the obstacle detection according to the depth image may refer to the prior art, and details are not described herein again. Although depth images can be used to detect obstacles, this method is less accurate and can only detect larger obstacles, and it is powerless for small obstacles on the road surface.

In order to detect small obstacles on the road surface, the present application determines the road surface area from the area to be detected by acquiring the pixel information corresponding to the area to be detected, and then detects the small obstacles in the road surface area.

Based on this, an embodiment of the present invention provides an obstacle detection method, which can detect not only a large obstacle but also a small obstacle, and can further obtain a three-dimensional spatial position of the obstacle according to the depth image. The method is applicable to the system shown in FIG. 1. When applied to the system shown in FIG. 1, the subject of the method is the obstacle detecting device 12.

As shown in FIG. 3, the method includes:

Step 101: Acquire a pixel image and a depth image corresponding to the to-be-detected area, respectively.

Wherein each pixel point in the pixel image corresponds to a pixel category. Specifically, one implementation manner of the pixel image is: capturing a detection area, obtaining a two-dimensional color image corresponding to the to-be-detected area; and combining a preset correspondence relationship between a pixel value including a pixel point and a pixel category, A pixel value of each pixel in the two-dimensional color image is analyzed, a category of each pixel is determined, and the pixel image is generated, in which pixels of the same pixel category have the same value, or The colors are the same.

Optionally, the preset correspondence relationship includes a corresponding relationship between a pixel value and a road surface pixel, and the generated corresponding pixel image is a binary image, and the pixel category corresponding to each pixel point in the binary image is a road surface pixel. Or non-road pixels, the area composed of all road pixels is the road surface area, and the area composed of all non-road pixels is the non-road area.

Each pixel point in the depth image corresponds to a depth value. The specific implementation of the depth image can refer to the prior art, and details are not described herein again.

As shown in FIG. 4, an embodiment of the present invention provides two-dimensional color corresponding to an area to be detected. The image and its corresponding pixel image and depth image. The pixel type corresponding to each pixel of the white area in the pixel image is a road surface pixel, and thus the white area may be referred to as a road surface area. The pixel type corresponding to each pixel of the black area is a non-road pixel, and therefore, the black area may be referred to as a non-road area.

Step 102: Determine a road surface area in a to-be-detected area and a first type of obstacle in the road surface area according to a pixel category of each pixel point in the pixel image; and determine a second type of obstacle in the area to be detected according to the depth image.

As described above, according to the pixel class of each pixel in the pixel image, an area composed of pixels of all the pixel types of the road surface pixels is determined as the road surface area, and an area composed of the pixel points of all the pixel types of the non-road pixels is determined as Non-road area. After determining the road surface area, by detecting the target obstacle in the road surface area or by detecting the contour lines of the road surface area and the non-road surface area, it is further detected whether there is a non-road area in which the contour line of the non-road surface area is surrounded by the contour line of the road surface area. The obstacles in the road surface area are detected in an equal manner. Such an obstacle obtained by analyzing a pixel image in the embodiment of the present application is referred to as a first type of obstacle.

Additionally, the first type of obstacle may not contain all of the obstacles in the area to be detected. Therefore, in the embodiment of the present application, the contour of each object in the depth image is determined according to the depth image, thereby determining an obstacle in the area to be detected, and after obtaining the depth image, the depth image is segmented into different according to the set depth threshold. A sub-image of depth, each sub-image contains only objects within a certain depth range, and the contour information of each sub-image (ie, the contour of the object) is detected to determine an obstacle in the area to be detected. For a specific implementation of determining an obstacle according to the depth image, reference may be made to the prior art, and details are not described herein again. The obstacle obtained by analyzing the depth image in the embodiment of the present application is referred to as a second type of obstacle.

Step 103: Determine a spatial location of the first type of obstacle according to the pixel image and the depth image, and determine a spatial location of the second type of obstacle according to the depth image.

For the first type of obstacle, determining, according to the coordinates of each obstacle in the pixel image of the first type of obstacle, each obstacle in the first type of obstacle in the to-be-detected area Two-dimensional position; then according to each of the first type of obstacles Determining, in a two-dimensional position of the obstacle to be detected, a coordinate of each obstacle in the first type of obstacle in the depth image; and then each obstacle according to the first type of obstacle Determining, in the pixel corresponding to the coordinates in the depth image, a depth value of each obstacle in the first type of obstacle; according to each obstacle in the first type of obstacle, in the to-be-detected area The two-dimensional position and the depth value in the middle obtain the spatial position of each obstacle in the first type of obstacle.

For the second type of obstacle, determining, according to the coordinates of each obstacle in the depth image in the second type of obstacle, each obstacle in the second type of obstacle is in the area to be detected The two-dimensional position and depth values give the spatial position of each obstacle in the second type of obstacle.

The solution provided by the present application obtains a pixel image and a depth image corresponding to the to-be-detected area, where each pixel point corresponds to a pixel category, and each pixel point in the depth image corresponds to a depth value; a pixel class of each pixel determines a road surface area in the area to be detected and a first type of obstacle in the road surface area; and determines a second type of obstacle in the area to be detected according to the depth image; and finally determines according to the pixel image and the depth image The spatial position of the first type of obstacle and the spatial position of the second type of obstacle according to the depth image are compared with the obstacle detection accuracy of various obstacle detection methods in the prior art, and the invention can detect by the pixel image A type of obstacle to the road surface area, such as a small obstacle in the road surface, can detect a large volume obstacle in the area to be detected through the depth image, and thus can comprehensively detect the obstacle in the area to be detected. In addition, after detecting an obstacle, the present application is also capable of outputting an obstacle space. Home help obstacle avoidance decisions.

In practical applications, some obstacles are located on both sides of the road, such as road signs and trees on both sides of the road, which may not affect the travel. Therefore, the embodiment of the present application also provides an obstacle detection method for the second type. Obstructions that do not affect travel are excluded from the obstacle. After the second type of obstacle is determined in step 102, the second type of obstacle may be determined after the determining the road surface area in the area to be detected according to the pixel type of each pixel in the pixel image. As shown in FIG. 5, the method further includes:

Step 201: Determine a two-dimensional position of the road surface area in the area to be detected.

After the road surface area in the area to be detected is determined from the pixel image, the two-dimensional position of the road surface area in the area to be detected can be determined.

Step 202: Determine a road surface area in the depth image according to a two-dimensional position of the road surface area in the to-be-detected area.

Since the depth image and the pixel image correspond to the same area to be detected, after determining the two-dimensional position of the road surface area in the area to be detected according to the pixel image, the road surface represented by the depth image may be determined according to the two-dimensional position. region.

It should be noted that when the road surface area is determined in the depth image, since the entire road surface is generally located at the same horizontal plane, that is, the depth values of the respective pixels corresponding to the entire road surface are the same, the two-dimensional position in the area to be detected can be determined according to the road surface area. Determining the preliminary range of the road surface area. If the pixel values of other pixel points are the same as the pixel values of the initially determined road surface area, the pixel points are also determined as the pixel points of the road surface area, thereby obtaining the final road surface area. .

Step 203: Screen an obstacle located in the road surface area from the second type of obstacle.

Through the implementation process of the above steps 201 to 203, the obstacles outside the road surface area of the second type of obstacles, that is, the obstacles that do not affect the advancement, can be deleted from the second type of obstacles.

Considering that after the first type of obstacle and the second type of obstacle are determined by the above method, there may be cases where the obstacle is repeatedly determined in the first type of obstacle and the second type of obstacle. Therefore, in the embodiment of the present application, in step 102, the road surface area in the to-be-detected area and the first type of obstacle in the road surface area are determined in the pixel category according to each pixel point in the pixel image. And determining the second type of obstacle in the area to be detected according to the depth image, as shown in FIG. 6, the method further includes:

Step 301: Determine, according to coordinates of each obstacle in the pixel image of the first type of obstacle, a two-dimensional position of each obstacle in the to-be-detected area in the first type of obstacle.

Step 302: According to each obstacle in the second type of obstacle, sitting in the depth image The two-dimensional positions of each obstacle in the second type of obstacle in the area to be detected are respectively determined.

Step 303: When there is a target obstacle that belongs to the first type of obstacle and belongs to the second type of obstacle and has the same two-dimensional position in the area to be detected, from the first type of obstacle or the second type of obstacle Delete the target obstacle.

Through the implementation process of the above steps 301 to 303, after determining the first type of obstacle and the second type of obstacle, respectively, each obstacle in the first type of obstacle and the second type of obstacle is determined in the area to be detected. In the two-dimensional position, when an obstacle is determined by the pixel image and the depth image respectively, the two-dimensional position is the same, the obstacle can be regarded as belonging to the first type of obstacle and the second type of obstacle and repeated calculation Obstacle.

Optionally, in the present application, after the obstacle is detected and the spatial position of the obstacle is determined, the contour information, the position information, and the category of the obstacle may be separately determined, so that the obstacle avoidance decision can be made. Provide more complete decision information.

After determining the spatial position of the first type of obstacle according to the pixel image and the depth image and determining the spatial position of the second type of obstacle according to the depth image, in step 103, as shown in FIG. The method also includes:

Step 401: Output spatial position information and contour information of the first type of obstacle whose depth value is within a preset range, and spatial position information and contour information of the second type of obstacle.

Among them, the preset range can be set and adjusted according to actual needs. Illustratively, spatial position information and contour information of an obstacle within a depth of two meters can be output, ignoring further obstacles. This is because the obstacle detection method provided by the present application is a real-time detection process, which can be detected all the time during the traveling, so that the obstacles outside the preset range may have no influence on the current travel, and the depth values at the obstacles may be After the preset range, continue to detect.

When the method is applied to the system shown in FIG. 1, the obstacle detecting device 12 can transmit the information obtained in step 401 to the decision device so that the decision device 13 makes an obstacle avoidance decision based on the information.

Those skilled in the art will readily appreciate that the present application can be implemented in a combination of hardware or hardware and computer software in combination with the elements and algorithm steps of the various examples described in the embodiments disclosed herein. Whether a function is implemented in hardware or computer software to drive hardware depends on the specific application and design constraints of the solution. A person skilled in the art can use different methods to implement the described functions for each particular application, but such implementation should not be considered to be beyond the scope of the present application.

The embodiment of the present application may perform the division of the function module on the obstacle detection device according to the above method example. For example, each function module may be divided according to each function, or two or more functions may be integrated into one processing module. The above integrated modules can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of the module in the embodiment of the present application is schematic, and is only a logical function division, and the actual implementation may have another division manner.

FIG. 8 is a schematic diagram showing a possible structure of the obstacle detecting device involved in the above embodiment. The obstacle detecting device includes: an acquiring unit 501, and an obstacle determination. Unit 502 and location determining unit 503. The obtaining unit 501 is configured to support the obstacle detecting device to execute the process 101 in FIG. 3; the obstacle determining unit 502 is configured to support the obstacle detecting device to perform step 102 in FIG. 3, step 202, step 203 in FIG. 5, FIG. Step 303; The position determining unit 503 is configured to support the obstacle detecting device to perform step 103 in FIG. 3, step 201 in FIG. 5, step 301 and step 302 in FIG. All the related content of the steps involved in the foregoing method embodiments may be referred to the functional descriptions of the corresponding functional modules, and details are not described herein again.

Optionally, as shown in FIG. 9, the obstacle detecting device further includes an output unit 601 for supporting the obstacle detecting device to execute the process 401 in FIG. 7.

In the case of employing an integrated unit, FIG. 10 shows a possible structural diagram of the obstacle detecting apparatus involved in the above embodiment. The obstacle detecting device includes a processing module 701 and a communication module 702. The processing module 701 is configured to perform control management on the action of the obstacle detecting device. For example, the processing module 701 is configured to support the obstacle detecting device to perform the processes 102 and 103 in FIG. 3, and the processes 201, 202, and 203 in FIG. Processes 301, 302, 303 in 6, and/or other processes for the techniques described herein. The communication module 702 is configured to support communication between the obstacle detecting device and other network entities, for example, with the function module or network shown in FIG. Communication between the bodies. The obstacle detecting device may further include a storage module 703 for storing program codes and data of the obstacle detecting device.

The processing module 701 can be a processor or a controller, for example, a central processing unit (CPU), a general-purpose processor, a digital signal processor (DSP), and an application-specific integrated circuit (Application-Specific Integrated Circuit (ASIC), Field Programmable Gate Array (FPGA) or other programmable logic device, transistor logic device, hardware component, or any combination thereof. It is possible to implement or carry out the various illustrative logical blocks, modules and circuits described in connection with the present disclosure. The processor may also be a combination of computing functions, for example, including one or more microprocessor combinations, a combination of a DSP and a microprocessor, and the like. The communication module 702 can be a transceiver, a transceiver circuit, a communication interface, or the like. The storage module 703 can be a memory.

When the processing module 701 is a processor, the communication module 702 is a transceiver, and the storage module 703 is a memory, the obstacle detecting device according to the embodiment of the present application may be the electronic device shown in FIG.

Referring to FIG. 11, the electronic device includes a processor 801, a communication interface 802 memory 803, and a bus 804. The processor 801, the communication interface 802, and the memory 803 are connected to each other through a bus 804; the bus 804 may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA). Bus, etc. The bus can be divided into an address bus, a data bus, a control bus, and the like. For ease of representation, only one thick line is shown in Figure 11, but it does not mean that there is only one bus or one type of bus.

The steps of a method or algorithm described in connection with the present disclosure may be implemented in a hardware or may be implemented by a processor executing software instructions. The software instructions may be composed of corresponding software modules, which may be stored in a random access memory (RAM), a flash memory, a read only memory (ROM), an erasable programmable read only memory ( Erasable Programmable ROM (EPROM), electrically erasable programmable read only memory (EEPROM), registers, hard disk, removable hard disk, compact disk read only (CD-ROM) or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor to enable the processor to read information from, and write information to, the storage medium. Of course, the storage medium can also be The components of the processor. The processor and the storage medium can be located in an ASIC.

Those skilled in the art will appreciate that in one or more examples described above, the functions described herein can be implemented in hardware, software, firmware, or any combination thereof. When implemented in software, the functions may be stored in a computer readable medium or transmitted as one or more instructions or code on a computer readable medium. Computer readable media includes both computer storage media and communication media including any medium that facilitates transfer of a computer program from one location to another. A storage medium may be any available media that can be accessed by a general purpose or special purpose computer.

The specific embodiments of the present invention have been described in detail with reference to the specific embodiments of the present application. It is to be understood that the foregoing description is only The scope of protection, any modifications, equivalent substitutions, improvements, etc. made on the basis of the technical solutions of the present application are included in the scope of protection of the present application.

Claims (16)

An obstacle detection method, comprising: Obtaining, respectively, a pixel image and a depth image corresponding to the to-be-detected area, where each pixel point corresponds to a pixel category, and each pixel point in the depth image corresponds to a depth value; Determining a road surface region in the to-be-detected region and a first type of obstacle in the road surface region according to a pixel class of each pixel point in the pixel image; and determining, in the to-be-detected region, according to the depth image The second type of obstacle; Determining a spatial location of the first type of obstacle based on the pixel image and the depth image and determining a spatial location of the second type of obstacle based on the depth image. The method according to claim 1, wherein after the determining the second type of obstacle in the area to be detected according to the depth image, the method further comprises: Determining a two-dimensional position of the pavement area in the area to be detected; Determining a road surface area in the depth image according to a two-dimensional position of the road surface area in the to-be-detected area; An obstacle located in the road surface area is screened from the second type of obstacle. The method according to claim 1, wherein the road surface region in the to-be-detected region and the first type of obstacle in the road surface region are determined according to a pixel class of each pixel point in the pixel image After determining the second type of obstacle in the area to be detected according to the depth image, the method further includes: Determining, according to coordinates of each obstacle in the pixel image in the first type of obstacle, a two-dimensional position of each obstacle in the first type of obstacle in the to-be-detected area; Determining, according to coordinates of each obstacle in the depth image, the two-dimensional position of each obstacle in the second type of obstacle in the to-be-detected area; When there is a target obstacle that belongs to both the first type of obstacle and the second type of obstacle and has the same two-dimensional position in the area to be detected, from the first type of obstacle or the first The target obstacle is deleted in the second type of obstacle. The method according to any one of claims 1 to 3, wherein the determining the spatial position of the first type of obstacle according to the pixel image and the depth image respectively comprises: Determining, according to coordinates of each obstacle in the pixel image in the first type of obstacle, a two-dimensional position of each obstacle in the first type of obstacle in the to-be-detected area; Determining coordinates of each obstacle in the first type of obstacle in the depth image according to a two-dimensional position of each obstacle in the first type of obstacle in the to-be-detected area; Determining, respectively, a depth value of each obstacle in the first type of obstacle according to a pixel point corresponding to a coordinate of each obstacle in the depth image in the first type of obstacle; A spatial position of each obstacle in the first type of obstacle is obtained according to a two-dimensional position and a depth value of each obstacle in the first type of obstacle in the to-be-detected area. The method according to any one of claims 1 to 3, wherein the determining the spatial location of the second type of obstacle according to the depth image comprises: Determining a two-dimensional position and depth value of each obstacle in the second type of obstacle in the to-be-detected area according to coordinates of each obstacle in the depth image in the second type of obstacle Obtaining a spatial location of each obstacle in the second type of obstacle. The method according to any one of claims 1 to 5, wherein the spatial position of the first type of obstacle is determined according to the pixel image and the depth image, respectively, and the second is determined according to the depth image After the spatial location of the obstacle, the method further includes: Spatial position information and contour information of the first type of obstacle having a depth value within a preset range and spatial position information and contour information of the second type of obstacle are output. An obstacle detecting device, comprising: An acquiring unit, configured to respectively acquire a pixel image and a depth map corresponding to the area to be detected For example, each pixel point in the pixel image corresponds to a pixel category, and each pixel point in the depth image corresponds to a depth value; An obstacle determining unit, configured to determine, according to a pixel category of each pixel point in the pixel image acquired by the acquiring unit, a road surface area in the to-be-detected area and a first type of obstacle in the road surface area; Determining, according to the depth image acquired by the acquiring unit, a second type of obstacle in the to-be-detected area; a position determining unit, configured to respectively determine a spatial position of the first type of obstacle determined by the obstacle determining unit according to the pixel image and the depth image, and determine a second type determined by the obstacle determining unit according to the depth image The spatial location of the obstacle. The device of claim 7 wherein: The position determining unit is further configured to determine a two-dimensional position of the road surface area in the area to be detected; Determining a road surface area in the depth image according to a two-dimensional position of the road surface area in the to-be-detected area; The obstacle determining unit is further configured to screen an obstacle located in the road surface area from the second type of obstacle. The device of claim 7 wherein: The position determining unit is further configured to determine, according to coordinates of each obstacle in the pixel image in the first type of obstacle, each obstacle in the first type of obstacle to be detected a two-dimensional position in the area; Determining, according to coordinates of each obstacle in the depth image, the two-dimensional position of each obstacle in the second type of obstacle in the to-be-detected area; The obstacle determining unit is further configured to: when there is a target obstacle that belongs to both the first type of obstacle and the second type of obstacle and has the same two-dimensional position in the to-be-detected area, from the The target obstacle is deleted in a type of obstacle or a second type of obstacle. The device according to any one of claims 7 to 9, wherein the location determining unit is specifically configured to: According to coordinates of each obstacle in the first type of obstacle in the pixel image, Determining, respectively, a two-dimensional position of each obstacle in the first type of obstacle in the area to be detected; Determining coordinates of each obstacle in the first type of obstacle in the depth image according to a two-dimensional position of each obstacle in the first type of obstacle in the to-be-detected area; Determining, respectively, a depth value of each obstacle in the first type of obstacle according to a pixel point corresponding to a coordinate of each obstacle in the depth image in the first type of obstacle; A spatial position of each obstacle in the first type of obstacle is obtained according to a two-dimensional position and a depth value of each obstacle in the first type of obstacle in the to-be-detected area. The device according to any one of claims 7 to 9, wherein the location determining unit is specifically configured to: Determining a two-dimensional position and depth value of each obstacle in the second type of obstacle in the to-be-detected area according to coordinates of each obstacle in the depth image in the second type of obstacle Obtaining a spatial location of each obstacle in the second type of obstacle. The device according to any one of claims 7 to 11, characterized in that the device further comprises an output unit for: Spatial position information and contour information of the first type of obstacle having a depth value within a preset range and spatial position information and contour information of the second type of obstacle are output. A computer storage medium, characterized by storing computer software instructions comprising program code designed to perform the obstacle detection method of any of claims 1-6. A computer program product characterized by being directly loadable into an internal memory of a computer and containing software code, the computer program being loaded and executed by a computer capable of implementing the obstacle of any of claims 1-6 Object detection method. An electronic device, comprising: a memory, a communication interface for storing computer executable code, and a processor for executing the computer executable code control to perform any of claims 1-6 One of the obstacle detection The method is used for data transmission between the electronic device and an external device. A robot comprising the electronic device of claim 15.

Images