DE102017106931B4 - Method for detecting and locating a non-line-of-sight object - Google Patents

Abstract

A method for detecting and locating a non-line-of-sight object (120b), the method comprising:
Receiving reflections at a detection system (110) of a movable platform (10), the reflections including direct and multi-part reflections;
Obtaining information from sensors (105) other than the sensing system (110), obtaining a location of the platform (10) and a map of the location;
Identify the reflections associated with static targets in order to retain the reflections associated with moving targets;
Identifying static targets on the map;
Distinguishing between line-of-sight objects (120a) and non-line-of-sight objects (120b) among the moving targets;
Localizing the non-line-of-sight objects (120b) relative to the platform (10); and
Displaying approaching non-line-of-sight objects (120b) among the non-line-of-sight objects (120b), wherein the approaching non-line-of-sight objects (120b) move toward the platform (10) on a path that intersects the platform (10); characterized in that
the non-line-of-sight objects (120b) are localized by determining the location of the non-line-of-sight objects (120b) on the map; and that
determining on the basis of the map whether the non-line-of-sight objects (120b) are approaching non-line-of-sight objects (120b);
wherein obtaining information comprises receiving information about moving objects from a camera,
wherein distinguishing between the line-of-sight objects (120a) and the non-line-of-sight objects (120b) comprises translating heading and elevation angles of pixels associated with moving objects in the field of view of the camera into heading and elevation angles associated with the field of view of the detection system (110).

Description

FIELD OF THE INVENTION

The present invention relates to obstacle detection and in particular to methods for detecting and locating a non-line-of-sight object according to the preamble of claim 1, as it essentially consists of the DE 10 2015 210 069 A1 is known.

Further state of the art can also be found in the publications DE 10 2004 058 844 A1 , DE 103 11 959 B4 , DE 10 2014 002 435 A1 and DE 10 2011 119 767 A1 .

BACKGROUND

Obstacle detection in various forms is part of a number of systems. For example, in automated manufacturing facilities, machines that transport equipment and components to different areas of the facility must detect and avoid obstacles. As another example, automated vacuum cleaners must detect and avoid obstacles such as stairs. As yet another example, obstacle detection is one of the tasks that must be accomplished by increasingly automated vehicles. Currently, obstacle detection refers to the detection of obstacles in the line of sight. Accordingly, it is desirable to provide non-line of sight obstacle detection and localization.

SUMMARY OF THE INVENTION

According to the invention, a method for detecting and localizing a non-line-of-sight object is presented, which is characterized by the features of claim 1.

Further described is a non-line-of-sight obstacle detection and localization system. The system is disposed on a moving platform and includes a transmitter portion configured to transmit radio frequency signals from a plurality of transmitting elements; a receiver portion configured to receive reflections at a plurality of receiving antenna elements, wherein the reflections include direct and multi-directional reflections; and a processing system configured to identify the reflections associated with static targets, retain the reflections associated with moving targets, distinguish between line-of-sight objects and non-line-of-sight objects among the moving targets, locate the non-line-of-sight objects relative to the platform, and approach non-line-of-sight objects among the non-line-of-sight objects, wherein approaching non-line-of-sight objects move toward the platform on a path that intersects the platform.

The above-mentioned features and advantages as well as other features and advantages of the invention are readily apparent from the following detailed description of the invention taken in conjunction with the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

• 1 ist eine Darstellung der Nicht-Sichtlinien Hinderniserkennung gemäß Ausführungsformen;
• 2 ist ein Blockdiagramm des Erfassungssystems gemäß Ausführungsformen; und
• 3 ist ein Prozessablauf eines Verfahrens zum Durchführen der Nicht-Sichtlinien Hinderniserkennung gemäß Ausführungsformen.

Other features, advantages and details appear, by way of example only, in the following detailed description of the embodiments and the detailed description, which refers to the following drawings:

• 1 is an illustration of non-line-of-sight obstacle detection according to embodiments;
• 2 is a block diagram of the detection system according to embodiments; and
• 3 is a process flow of a method for performing non-line-of-sight obstacle detection according to embodiments.

DESCRIPTION OF THE EMBODIMENTS

As mentioned above, obstacle detection is part of the operation of many systems that involve automated path control. Depending on location and distance, different types of obstacle detection may be used. For example, automated vacuum cleaners that need to detect obstacles that are on the order of inches away may use infrared transmissions and reflections. In other applications, such as vehicle and aircraft applications, where obstacle detection at longer ranges is of interest, radio detection and ranging (radar) is generally used. Generally, radar systems transmit radio waves and determine the range, angle (azimuth and elevation), and speed of an object based on the reflection of the radio waves from the object. As such, typical radar detection relies on line of sight to the object (target) being detected. However, in vehicle collision avoidance systems, for example, there is an interest in detecting objects that are not yet in the line of sight of the vehicle sensors. In accordance with exemplary embodiments of the invention, non-line of sight obstacle detection and localization is facilitated. As described in more detail below, radar data is combined with a given or learned model of the current topology to derive information about objects that are beyond the line of sight. While the exemplary case of vehicle-based radar systems is described for illustrative purposes, the embodiments described herein are not limited to use in a vehicle system. Other vehicles (e.g., construction equipment, agricultural equipment) and other types of platforms are also contemplated. Additionally, while a Doppler radar system is discussed as an exemplary embodiment of a non-line of sight sensing system herein, any sensor system that provides range, azimuth, altitude, and velocity information may be used in accordance with the detailed embodiments.

1 is an illustration of non-line-of-sight obstacle detection according to embodiments. An exemplary intersection is shown, and the exemplary platform 10 for the non-line-of-sight obstacle detection system 110 is a vehicle. A host vehicle 100, which includes the detection system 110 (2) according to the embodiments described below, is shown at the exemplary intersection. The host vehicle 100 may include other sensors 105 (e.g., camera, lidar system). Other vehicles 120a, 120b (which may also include the detection system 110) and other objects 130 (which are buildings in the exemplary illustration) are also shown in 1 shown. The other objects 130a and one of the other vehicles 120a are within the line of sight of the detection system 110 of the host vehicle 100. One of the other objects 130b and the other vehicle 120b are not within the line of sight of the detection system 110 of the host vehicle 100. That is, the transmissions 215 from the detection system 110 of the host vehicle 100 cannot directly detect the vehicle 120b or the object 130b based on the relative positions of the host vehicle 100 and the vehicle 120b or the 1 shown object 130b. As such, reflections 225 directly from the vehicle 120b or object 130b may not be obtained with the detection system 110 of the host vehicle 100.

However, as the dashed lines in 1 indicate, the transmissions 215 from the detection system 110 of the host vehicle 100 may bounce off the other vehicle 120a and other objects 130a and reach the vehicle 120b or the object 130b that are outside the line of sight of the detection system 110 of the host vehicle 100. The reflections 225 from the vehicle 120b or the object 130b may also bounce off the other vehicle 120a and other objects 130 to reach the host vehicle 100. These bounced signals are referred to as multipath signals because reflections 225 may result from multiple paths based on one transmission 215 by the detection system 110. To be clear, only one of the direct transmissions 215x and the resulting reflections 225x are shown within the line of sight of the host vehicle 100 (to and from the vehicle 120a and the other objects 130) in order to obscure the multi-path impact signals (215/225) of interest according to the embodiments. For example, a transmission 215x may reflect off the other object 130a and result in a reflection 225x directly back to the detection system 110 and, in addition, a transmission 215 may bounce off the other object 130a and result in a reflection 225 that actually originates from the non-line of sight object of the vehicle 120b.

In a typical radar system, these bounced transmissions 215 and resulting reflected reflections 225 or multipath signals are an undesirable effect, while the direct transmissions 215 and reflections 225 from targets within the line of sight (vehicle 120, objects 130) are of interest. However, according to embodiments of the invention, these multipath reflections 225 are isolated and processed to perform non-line of sight obstacle detection and localization. The detection system 110 is described with reference to 2 executed.

2 12 is a block diagram of the sensing system 110 according to embodiments. The sensing system 110, which is a Doppler radar system, is an exemplary embodiment, but any sensor system that provides similar information (e.g., speed, range, azimuth, elevation) may be used in alternative embodiments. Vehicle sensing systems 110 used in vehicle platforms 10, such as the host vehicle 100, may generally operate in continuous linear wave frequency modulation (CW-LFM) and may operate frequency ranges of 21 to 27 gigahertz or 77 to 81 gigahertz. Power is transmitted and received over a number of cycles to perform non-line-of-sight obstacle detection and localization according to the embodiments discussed herein. Information obtained from the sensing system 110 may be provided to the vehicle control systems 270, the display systems 280. The sensing system 110 may additionally communicate with other sensor systems 105 of the host vehicle 100. The vehicle control systems 270 may include, for example, automatic braking or steering control systems. The display systems 280 may display a non-line-of-sight object (e.g., a other vehicle 120b, 1 ). Other sensor systems 105 of the host vehicle 100 may include imaging systems (e.g., a global positioning system (GPS)), vision systems (e.g., mono or stereo camera systems), and ranging systems (e.g., LIDAR). These other sensor systems 105 may facilitate determining the location of the host vehicle 100 and a model of the topology at the location.

The detection system 110 includes a transmitter section 210, a receive section 220, and a processing system 230. The detection system 110 may be a multiple input multiple output (MIMO) array radar, as shown. Thus, the transmitter section 210 may include multiple antenna elements 214 that emit multiple transmissions 215, and the receiver section 220 may include multiple antenna elements 224 to collect the reflections 225. Such reflections are obtained over an azimuth range and an elevation range with the arrays of elements 214, 224. The detection system 110 may use known techniques such as beamforming on the antenna elements 224 of the receive section 220, and specifically using the Doppler effect to determine the speed of the detected objects. Range and power (intensity) is also obtained from the reflections 225. Thus, the array of antenna elements 214, 224 facilitates obtaining an image in which each pixel is assumed to be associated with an azimuth, altitude, range and velocity value, as well as an intensity. Additionally, the detection system 110 may use a model of the topology (indicating the targets in the line of sight) to facilitate identification of non-line of sight objects.

The transmitter section 210 and the receiver section 220 are known and are not described in detail herein. As shown in the expanded view in 2 As shown, the transmitter section 210 generally includes an oscillator 211, a buffer 212, and a power amplifier 213, and the receive section 220 generally includes a preamplifier 221, a mixer 222, and an analog-to-digital (A/D) converter 223. The processing system 230 includes one or more memory devices 240 and one or more processors 250 to perform non-line-of-sight obstacle detection. For example, while the processing system 230 is shown as part of the sensing system 110 and separate from other vehicle control systems 270, the processing system 230 that processes the reflections 225 to perform non-line-of-sight obstacle detection may be shared among one or more systems in the host vehicle 100. The communication between the various systems (110, 270, 280, 105) may be based on hardwired or wireless communication or on a combination of known communication schemes, including, for example, via a common bus. The processing of the reflections 225 performed by the processing system 230 to identify and locate non-line-of-sight objects approaching the host vehicle 100 is described with reference to 3 described.

3 is a process flow of a method for performing non-line-of-sight obstacle detection and localization in accordance with embodiments. The exemplary embodiments presented below refer to the reflections 225 received by the detection system 110 for illustrative purposes. As mentioned above, information (range, velocity, azimuth, altitude) used to perform non-line-of-sight detection may instead be obtained from other known sensor systems or other configurations of radar systems. At block 310, obtaining reflections 225 includes obtaining both the direct and multipath reflections 225 based on the objects present. Obtaining reflections 225 also includes performing multiple transmissions 215 from each transmit antenna element 214 and obtaining multiple reflections 225 at each receive antenna element 224. Processing the reflections 225, at block 320, includes multiple processes 330, 340, 350, as shown. In addition to the reflections 225 obtained at block 310, other information (obtained at blocks 325 and 335) is used to detect non-line-of-sight objects (e.g., another vehicle 120b, 1 ) to detect and locate. Obtaining other information at block 325 refers to obtaining information from other sensors 105 or other processing systems of the platform 10 (e.g., host vehicle 100). The other information may include landscape information about the topology of the current location of the platform 10 (host vehicle 100). For example, the landscape information may include the location of objects 130 such as buildings. This information may be provided as input to the host vehicle 100 or may be learned by the host vehicle 100 from previous visits to the same location according to a known dynamic learning algorithm. Monitoring host vehicle 100 motion parameters may include monitoring other sensors 105, such as a GPS receiver of the host vehicle 100, to determine position and motion at block 335. How 3 displayed, the location information, obtained by monitoring the host vehicle 100 (at block 335) is needed to obtain other information, such as landscape information (at block 325).

Identifying reflections 225 from static environments, at block 330, refers to identifying pixels with zero velocity (zero Doppler). These pixels can then be assigned to non-moving objects (e.g., objects 130a, 130b, 1 ). The identification at block 330 can be supported if the landscape information is available (from block 325). The landscape information helps to distinguish between the stationary objects 130 in the scene. That is, if the vehicle 120a in 1 is stopped, it may appear as a non-moving object. However, the landscape information (from block 325) may be used to compare the vehicle 120a (which may be non-moving at the time of processing, but is not stationary) with objects 130 (which are both still and stationary at the time of processing). The identification of reflections 225 from static environments at block 330 may first be performed as a type of filtering of the reflections 225 to isolate the reflections 225 associated with moving (or moving) objects (both within and outside the line of sight of the host vehicle 100 of the surveillance system 110). In this filtering of static objects (at block 330), the distinction between line of sight (e.g., 120a) and non-line of sight (e.g., other vehicle 120b, 1 ) that moves objects at block 340. As part of the processing at block 340, pixels associated with motion may first be used to identify objects based on a known clustering algorithm. The remainder of the processing at block 340 then involves categorizing the objects within or outside the line of sight of the platform 10.

Distinguishing line-of-sight moving objects from non-line-of-sight moving objects, at block 340, may be performed according to various embodiments. According to one embodiment, other sensors 105 may be used based on the information obtained at block 325. For example, a camera (105) mounted on the host vehicle 100 may be used and known moving object detection may be performed within the camera's field of view. The azimuth and elevation of pixels associated with any moving objects in the camera's field of view may be translated into an azimuth and elevation associated with the field of view of the detection system 110. If the translated azimuth and elevation values match azimuth and elevation values of moving objects or reflections 225 that are not filtered out as static (at block 330), then those objects or reflections 225 are associated with a line of sight. According to another embodiment, a known statistical modeling approach is used on reflections 225 associated with moving objects. Once the non-line-of-sight moving objects are identified from line-of-sight moving objects (at block 340), locate the non-line-of-sight moving objects (e.g., 120b) at block 350, obtain other information (at block 325) as in 3 The other information may include, for example, a map of the current position of the host vehicle 100. The indication of an approaching non-line-of-sight moving object in block 360 includes determining, based on the location (at block 350), whether the non-line-of-sight moving object of the line-of-sight moving object intersects with the host vehicle 100. For example, in the case of the 1 illustrated scenario, the non-line-of-sight object may intersect with the host vehicle 100 if both vehicles 100 are traveling on the current path. However, in another exemplary scenario, the path on which the other vehicle 120b is shown may go below the path on which the host vehicle 100 is shown. In such a scenario, the other vehicle 120b and the host vehicle 100 would not intersect. As such, localization (at block 350) using a map may prevent the other vehicle 120b from being displayed as an approaching non-line-of-sight object at block 360. The display provided at block 360 may be visible on a map (via display system 280) visible to the driver of the host vehicle 100 as a warning of the approaching object (e.g., another vehicle 120b, FIG. 1). In additional or alternative embodiments, the indication may be provided to a vehicle control system 270 (e.g., collision avoidance or automated steering system) to facilitate decisions regarding control of the host vehicle 100 based on the position and motion of the non-line-of-sight object(s).

Claims (1)

A method for detecting and locating a non-line-of-sight object (120b), the method comprising: receiving reflections at a detection system (110) of a movable platform (10), the reflections including direct and multi-part reflections; obtaining information from sensors (105) other than the detection system (110), wherein a Location of the platform (10) and a map of the location are obtained; identifying the reflections associated with static targets to retain the reflections associated with moving targets; identifying the static targets on the map; distinguishing between line of sight objects (120a) and non-line of sight objects (120b) among the moving targets; locating the non-line of sight objects (120b) relative to the platform (10); and indicating approaching non-line of sight objects (120b) among the non-line of sight objects (120b), the approaching non-line of sight objects (120b) moving toward the platform (10) on a path that intersects the platform (10); characterized in that the non-line of sight objects (120b) are located by determining the location of the non-line of sight objects (120b) on the map; and determining, based on the map, whether the non-line-of-sight objects (120b) are approaching non-line-of-sight objects (120b); wherein obtaining information comprises receiving information about moving objects from a camera, wherein distinguishing between the line-of-sight objects (120a) and the non-line-of-sight objects (120b) comprises translating heading and elevation angles of pixels associated with moving objects in the field of view of the camera into heading and elevation angles associated with the field of view of the detection system (110).

Images