WO2024142893A1 - Electronic device, method for controlling electronic device, and program - Google Patents

Abstract

This electronic device includes a control unit executing first processing of distinguishing an object from a person on the basis of actions of a plurality of portions of the object reflected in a received signal received as a reflection wave corresponding to a transmitted signal transmitted as a transmission wave and reflected from the object.

Description

Electronic device, electronic device control method, and program CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority to patent application No. 2022-208839, filed in Japan on December 26, 2022, the entire disclosure of which is incorporated herein by reference.

This disclosure relates to an electronic device, a control method for an electronic device, and a program.

For example, in fields such as the automobile industry, there is growing importance placed on technology that measures the distance between a vehicle and a specific object. In particular, in recent years, various research has been conducted into RADAR (Radio Detecting and Ranging) technologies, which measure the distance between an object by transmitting radio waves such as millimeter waves and receiving the waves reflected by objects such as obstacles. The importance of such technology for measuring distance is expected to increase in the future with the development of technologies that assist drivers in driving and technologies related to autonomous driving, which automates part or all of driving.

Various proposals have also been made regarding technology that detects the presence of a specific object by receiving the reflected waves of transmitted radio waves that are reflected off the object. For example, Patent Document 1 proposes an estimation device that uses radio signals received by multiple receiving antennas to estimate the direction of a moving object, such as a person.

JP 2019-132850 A

The electronic device according to an embodiment includes:
The device is equipped with a control unit that executes a first process of identifying the object as a person by reflecting the movements of multiple parts of the object in a received signal that is received as a reflected wave from an object, the received signal being transmitted as a transmission wave.

The electronic device according to an embodiment includes:
The device is equipped with a control unit that executes a first process to identify the object as a moving body capable of moving parts of its body by reflecting movements of multiple parts of the object in a received signal that is received as a reflected wave from an object, in which a transmission signal transmitted as a transmission wave is reflected by the object.

A method for controlling an electronic device according to an embodiment includes:
detecting the object based on a transmission signal transmitted as a transmission wave and a reception signal received as a reflected wave of the transmission wave reflected by the object;
performing a first process of identifying the object as a person based on the transmitted signal and the received signal reflecting the motion of a plurality of parts of the object;
including.

A program according to an embodiment includes:
For electronic devices,
detecting the object based on a transmission signal transmitted as a transmission wave and a reception signal received as a reflected wave of the transmission wave reflected by the object;
performing a first process of identifying the object as a person based on the transmitted signal and the received signal reflecting the motion of a plurality of parts of the object;
Execute the command.

FIG. 1 is a diagram illustrating a usage mode of an electronic device according to an embodiment. 1 is a functional block diagram illustrating a schematic configuration of an electronic device according to an embodiment. FIG. 2 is a functional block diagram illustrating a control unit of the electronic device according to an embodiment. FIG. 2 is a diagram illustrating a configuration of a transmission signal according to an embodiment. FIG. 13 is a diagram illustrating an example of a reference probability density distribution of pedestrian speed. FIG. 1 is a diagram illustrating a micro Doppler velocity measured from a pedestrian. 1 is a flowchart illustrating a prerequisite operation for a process of identifying a pedestrian by the electronic device 1 according to an embodiment. 10 is a flowchart illustrating an operation of identifying a pedestrian by the electronic device 1 according to an embodiment. FIG. 13 is a diagram showing an example of Pearson divergence calculated for an object probability density distribution and a reference probability density distribution.

In a technology for detecting a specific object by receiving a reflected wave of a transmitted transmission wave reflected by the object, it is desirable to improve the detection accuracy of the object. An object of the present disclosure is to provide an electronic device, a control method for an electronic device, and a program that can improve the accuracy of detecting a target object. According to one embodiment, it is possible to provide an electronic device, a control method for an electronic device, and a program that can improve the accuracy of detecting a target object. One embodiment will be described in detail below with reference to the drawings.

The electronic device according to one embodiment is mounted on a vehicle (mobile body) such as an automobile and is capable of detecting a specific object present around the mobile body as a target. To this end, the electronic device according to one embodiment can transmit a transmission wave to the surroundings of the mobile body from a transmitting antenna installed on the mobile body. The electronic device according to one embodiment can also receive a reflected wave of the transmitted wave from a receiving antenna installed on the mobile body. At least one of the transmitting antenna and the receiving antenna may be provided on, for example, a radar sensor installed on the mobile body.

Below, as a typical example, a configuration in which the electronic device according to an embodiment is mounted on an automobile such as a passenger car will be described. However, the electronic device according to an embodiment is not limited to being mounted on an automobile. The electronic device according to an embodiment may be mounted on various moving bodies such as self-driving automobiles, buses, trucks, taxis, motorcycles, bicycles, ships, aircraft, helicopters, agricultural equipment such as tractors, snowplows, cleaning vehicles, police cars, ambulances, and drones. Furthermore, the electronic device according to an embodiment is not necessarily limited to being mounted on a moving body that moves by its own power. For example, the moving body on which the electronic device according to an embodiment is mounted may be a trailer part towed by a tractor. The electronic device according to an embodiment can measure the distance between a sensor and an object in a situation in which at least one of the sensor and a predetermined object can move. Furthermore, the electronic device according to an embodiment can measure the distance between a sensor and an object even if both the sensor and the object are stationary. Furthermore, the automobile included in the present disclosure is not limited by its overall length, overall width, overall height, displacement, capacity, or load capacity. For example, automobiles in this disclosure include automobiles with an engine displacement of more than 660 cc, as well as automobiles with an engine displacement of 660 cc or less, so-called light automobiles. Additionally, automobiles in this disclosure also include automobiles that use electricity as part or all of their energy source and that use a motor.

Furthermore, the electronic device according to an embodiment does not necessarily have to be mounted on a moving object. For example, the electronic device according to an embodiment may be installed on a stationary device such as a traffic light or roadside device, and detect pedestrians and/or vehicles around the device.

First, an example of object detection by an electronic device according to an embodiment will be described.

FIG. 1 is a diagram illustrating a usage mode of an electronic device according to one embodiment. FIG. 1 shows an example in which a sensor having a transmitting antenna and a receiving antenna according to one embodiment is installed on a mobile object.

　A sensor 5 having a transmitting antenna and a receiving antenna according to an embodiment is installed on the moving body 100 shown in FIG. 1. The moving body 100 shown in FIG. 1 is equipped with (for example, built-in) an electronic device 1 according to an embodiment. The specific configuration of the electronic device 1 will be described later. The sensor 5 may have at least one of a transmitting antenna and a receiving antenna. The sensor 5 may also include at least one of the other functional units, such as at least a part of the control unit 10 (FIG. 2) included in the electronic device 1, as appropriate. The moving body 100 shown in FIG. 1 may be an automobile vehicle such as a passenger car, but may be any type of moving body. In FIG. 1, the moving body 100 may be moving (driving or slowly moving) in, for example, the positive Y-axis direction (traveling direction) shown in the figure, or may be moving in another direction, or may be stationary without moving.

As shown in FIG. 1, a sensor 5 equipped with a transmitting antenna is installed on the moving body 100. In the example shown in FIG. 1, only one sensor 5 equipped with a transmitting antenna and a receiving antenna is installed in the front of the moving body 100. Here, the position where the sensor 5 is installed on the moving body 100 is not limited to the position shown in FIG. 1, and may be other positions as appropriate. For example, the sensor 5 as shown in FIG. 1 may be installed on the left side, right side, and/or rear of the moving body 100. In addition, the number of such sensors 5 may be any number greater than or equal to one depending on various conditions (or requirements) such as the range and/or accuracy of measurement in the moving body 100. The sensor 5 may be installed inside the moving body 100. The inside of the moving body 100 may be, for example, the space inside the bumper, the space inside the body, the space inside the headlight, or the driving space.

The sensor 5 transmits electromagnetic waves as transmission waves from a transmitting antenna. For example, if a specific object (e.g., object 200 shown in FIG. 1) is present around the moving body 100, at least a portion of the transmission wave transmitted from the sensor 5 is reflected by the object and becomes a reflected wave. Then, by receiving such a reflected wave, for example, by a receiving antenna of the sensor 5, the electronic device 1 mounted on the moving body 100 can detect the object as a target.

The sensor 5 equipped with a transmitting antenna may typically be a RADAR (Radio Detecting and Ranging) sensor that transmits and receives radio waves. However, the sensor 5 is not limited to a radar sensor. The sensor 5 according to one embodiment may be a sensor based on, for example, LIDAR (Light Detection and Ranging, Laser Imaging Detection and Ranging) technology using light waves. Such sensors may be configured to include, for example, a patch antenna. Technologies such as RADAR and LIDAR are already known, so detailed descriptions may be simplified or omitted as appropriate.

The electronic device 1 mounted on the moving body 100 shown in FIG. 1 receives, from a receiving antenna, a reflected wave of a transmission wave transmitted from a transmitting antenna of the sensor 5. In this way, the electronic device 1 can detect a predetermined object 200 that exists within a predetermined distance from the moving body 100 as a target. For example, as shown in FIG. 1, the electronic device 1 can measure a distance A between the moving body 100, which is the host vehicle, and the predetermined object 200. The electronic device 1 can also measure the relative speed between the moving body 100, which is the host vehicle, and the predetermined object 200. Furthermore, the electronic device 1 can also measure the direction (arrival angle θ) in which the reflected wave from the predetermined object 200 arrives at the moving body 100, which is the host vehicle.

Here, the object 200 may be, for example, at least one of an oncoming vehicle traveling in a lane adjacent to the moving body 100, a car traveling parallel to the moving body 100, and a car before or after the moving body 100 traveling in the same lane. The object 200 may also be any object present around the moving body 100, such as a motorcycle, bicycle, baby stroller, human being such as a pedestrian, an animal, an insect or other living thing, a guard rail, a median strip, a road sign, a step on a sidewalk, a wall, a manhole, or an obstacle. Furthermore, the object 200 may be moving or stationary. For example, the object 200 may be a car parked or stopped around the moving body 100. In the present disclosure, the object detected by the sensor 5 includes not only inanimate objects, but also living objects such as a person or an animal. The object detected by the sensor 5 of the present disclosure includes targets including people, objects, and animals detected by radar technology.

In FIG. 1, the ratio between the size of the sensor 5 and the size of the moving body 100 does not necessarily represent the actual ratio. Also, in FIG. 1, the sensor 5 is shown installed outside the moving body 100. However, in one embodiment, the sensor 5 may be installed at various positions on the moving body 100. For example, in one embodiment, the sensor 5 may be installed inside the bumper of the moving body 100 so as not to be visible from the outside of the moving body 100.

Below, as a typical example, the transmitting antenna of the sensor 5 will be described as transmitting radio waves in a frequency band such as millimeter waves (30 GHz or higher) or quasi-millimeter waves (e.g., around 20 GHz to 30 GHz). For example, the transmitting antenna of the sensor 5 may transmit radio waves having a frequency bandwidth of 4 GHz, such as 77 GHz to 81 GHz.

FIG. 2 is a functional block diagram that shows an outline of an example of the configuration of an electronic device 1 according to an embodiment. An example of the configuration of an electronic device 1 according to an embodiment is described below.

When measuring distances and the like using a millimeter wave radar, a frequency modulated continuous wave radar (hereinafter referred to as FMCW radar) is often used. With an FMCW radar, the transmission signal is generated by sweeping the frequency of the radio waves to be transmitted. Therefore, for example, in a millimeter wave FMCW radar using radio waves in the 79 GHz frequency band, the frequency of the radio waves used has a frequency bandwidth of 4 GHz, for example, 77 GHz to 81 GHz. A radar using the 79 GHz frequency band has the characteristic that it has a wider usable frequency bandwidth than other millimeter wave/quasi-millimeter wave radars, such as those using the 24 GHz, 60 GHz, and 76 GHz frequency bands. Below, such an embodiment will be described as an example.

As shown in FIG. 2, the electronic device 1 according to an embodiment may be composed of a sensor 5 and an ECU (Electronic Control Unit) 50. The ECU 50 controls various operations of the mobile object 100. The ECU 50 may be composed of at least one ECU. The electronic device 1 according to an embodiment includes a control unit 10. The electronic device 1 according to an embodiment may also include other functional units, such as a transmission unit 20 and at least one of the reception units 30A to 30D, as appropriate. As shown in FIG. 2, the electronic device 1 may include multiple reception units, such as the reception units 30A to 30D. Hereinafter, when there is no need to distinguish between the reception units 30A, 30B, 30C, and 30D, they will simply be referred to as "reception unit 30".

As shown in FIG. 3, the control unit 10 may include a distance FFT processing unit 11, a speed FFT processing unit 12, a threshold determination unit 13, an arrival angle estimation unit 14, an object detection unit 15, a tracking processing unit 16, a memory unit 17, and an object mark classification unit 18. These functional units included in the control unit 10 will be described further below.

As shown in FIG. 2, the transmitting unit 20 may include a signal generating unit 21, a synthesizer 22, phase control units 23A and 23B, amplifiers 24A and 24B, and transmitting antennas 25A and 25B. Hereinafter, when there is no need to distinguish between phase control unit 23A and phase control unit 23B, they will simply be referred to as "phase control unit 23." Hereinafter, when there is no need to distinguish between amplifier 24A and amplifier 24B, they will simply be referred to as "amplifier 24." Hereinafter, when there is no need to distinguish between transmitting antenna 25A and transmitting antenna 25B, they will simply be referred to as "transmitting antenna 25."

The receiving unit 30 may include corresponding receiving antennas 31A to 31D, as shown in FIG. 2. Hereinafter, when there is no need to distinguish between receiving antennas 31A, 31B, 31C, and 31D, they will simply be referred to as "receiving antennas 31." Also, as shown in FIG. 2, each of the multiple receiving units 30 may include an LNA 32, a mixer 33, an IF unit 34, and an AD conversion unit 35. The receiving units 30A to 30D may each have the same configuration. In FIG. 2, the configuration of only receiving unit 30A is shown generally as a representative example.

The above-mentioned sensor 5 may include, for example, a transmitting antenna 25 and a receiving antenna 31. The sensor 5 may also include at least one of the other functional units, such as a control unit 10, as appropriate.

The control unit 10 of the electronic device 1 according to one embodiment can control the operation of the entire electronic device 1, including the control of each functional unit constituting the electronic device 1. The control unit 10 may include at least one processor, such as a CPU (Central Processing Unit) or a DSP (Digital Signal Processor), to provide control and processing power for executing various functions. The control unit 10 may be realized as a single processor, as well as several processors, or each processor may be realized as an individual processor. The processor may be realized as a single integrated circuit. An integrated circuit is also called an IC (Integrated Circuit). The processor may be realized as multiple integrated circuits and discrete circuits connected to communicate with each other. The processor may be realized based on various other known technologies. In one embodiment, the control unit 10 may be configured as, for example, a CPU (hardware) and a program (software) executed by the CPU. The control unit 10 may include memory necessary for the operation of the control unit 10 as appropriate.

In the electronic device 1 according to one embodiment, the control unit 10 can control at least one of the transmission unit 20 and the reception unit 30. In this case, the control unit 10 may control at least one of the transmission unit 20 and the reception unit 30 based on various information stored in an arbitrary storage unit (memory). Also, in the electronic device 1 according to one embodiment, the control unit 10 may instruct the signal generation unit 21 to generate a signal, or control the signal generation unit 21 to generate a signal.

The signal generating unit 21 generates a signal (transmission signal) to be transmitted as a transmission wave T from the transmitting antenna 25 under the control of the control unit 10. When generating the transmission signal, the signal generating unit 21 may assign a frequency of the transmission signal, for example, based on the control of the control unit 10. Specifically, the signal generating unit 21 may assign a frequency of the transmission signal, for example, according to parameters set by the control unit 10. For example, the signal generating unit 21 receives frequency information from the control unit 10 or an arbitrary storage unit (memory) and generates a signal of a predetermined frequency in a frequency band such as 77 to 81 GHz. The signal generating unit 21 may be configured to include a functional unit such as a voltage controlled oscillator (VCO).

The signal generating unit 21 may be configured as hardware having the relevant function, or may be configured as a microcomputer, for example, or may be configured as a processor such as a CPU and a program executed by the processor. Each of the functional units described below may also be configured as hardware having the relevant function, or may be configured as a microcomputer, for example, if possible, or may be configured as a processor such as a CPU and a program executed by the processor.

In the electronic device 1 according to one embodiment, the signal generating unit 21 may generate a transmission signal (transmission chirp signal) such as a chirp signal. In particular, the signal generating unit 21 may generate a signal (linear chirp signal) whose frequency changes periodically and linearly. For example, the signal generating unit 21 may generate a chirp signal whose frequency increases periodically and linearly from 77 GHz to 81 GHz over time. Also, for example, the signal generating unit 21 may generate a signal whose frequency periodically repeats a linear increase (up chirp) and decrease (down chirp) from 77 GHz to 81 GHz over time. The signal generated by the signal generating unit 21 may be set in advance in, for example, the control unit 10. Also, the signal generated by the signal generating unit 21 may be stored in advance in, for example, an arbitrary storage unit (memory). Since chirp signals used in technical fields such as radar are known, a more detailed description will be simplified or omitted as appropriate. The signal generated by the signal generating unit 21 is supplied to the synthesizer 22.

FIG. 4 is a diagram illustrating an example of a chirp signal generated by the signal generating unit 21.

In FIG. 4, the horizontal axis represents the elapsed time, and the vertical axis represents the frequency. In the example shown in FIG. 4, the signal generating unit 21 generates a linear chirp signal whose frequency changes periodically and linearly. In FIG. 4, each chirp signal is shown as c1, c2, ..., c8. As shown in FIG. 4, in each chirp signal, the frequency increases linearly with the passage of time.

In the example shown in FIG. 4, one subframe includes eight chirp signals such as c1, c2, ..., c8. That is, subframe 1 and subframe 2 shown in FIG. 4 are each composed of eight chirp signals such as c1, c2, ..., c8. In addition, in the example shown in FIG. 4, one frame includes 16 subframes such as subframe 1 to subframe 16. That is, frame 1 and frame 2 shown in FIG. 4 are each composed of 16 subframes. Also, as shown in FIG. 4, a frame interval of a predetermined length may be included between frames. One frame shown in FIG. 4 may be, for example, about 30 to 50 milliseconds long.

In FIG. 4, frames 2 and onward may have the same configuration. Also, in FIG. 4, frames 3 and onward may have the same configuration. In electronic device 1 according to one embodiment, signal generation unit 21 may generate a transmission signal as any number of frames. Also, in FIG. 4, some chirp signals are omitted. In this way, the relationship between time and frequency of the transmission signal generated by signal generation unit 21 may be stored, for example, in any storage unit (memory).

In this way, the electronic device 1 according to one embodiment may transmit a transmission signal consisting of subframes including multiple chirp signals. Also, the electronic device 1 according to one embodiment may transmit a transmission signal consisting of a frame including a predetermined number of subframes.

The electronic device 1 will be described below as transmitting a transmission signal with a frame structure as shown in FIG. 4. However, the frame structure as shown in FIG. 4 is only an example, and the number of chirp signals included in one subframe is not limited to eight. In one embodiment, the signal generating unit 21 may generate a subframe including any number of chirp signals (for example, any multiple). The subframe structure as shown in FIG. 4 is also only an example, and the number of subframes included in one frame is not limited to 16. In one embodiment, the signal generating unit 21 may generate a frame including any number of subframes (for example, any multiple). The signal generating unit 21 may generate signals of different frequencies. The signal generating unit 21 may generate multiple discrete signals with frequencies f each having a different bandwidth.

Returning to FIG. 2, the synthesizer 22 increases the frequency of the signal generated by the signal generating unit 21 to a frequency in a predetermined frequency band. The synthesizer 22 may increase the frequency of the signal generated by the signal generating unit 21 to a frequency selected as the frequency of the transmission wave T to be transmitted from the transmitting antenna 25. The frequency selected as the frequency of the transmission wave T to be transmitted from the transmitting antenna 25 may be set by, for example, the control unit 10. Also, the frequency selected as the frequency of the transmission wave T to be transmitted from the transmitting antenna 25 may be stored in, for example, an arbitrary storage unit (memory). The signal whose frequency has been increased by the synthesizer 22 is supplied to the phase control unit 23 and the mixer 33. When there are multiple phase control units 23, the signal whose frequency has been increased by the synthesizer 22 may be supplied to each of the multiple phase control units 23. Also, when there are multiple receiving units 30, the signal whose frequency has been increased by the synthesizer 22 may be supplied to each of the mixers 33 in the multiple receiving units 30.

The phase control unit 23 controls the phase of the transmission signal supplied from the synthesizer 22. Specifically, the phase control unit 23 may adjust the phase of the transmission signal by appropriately advancing or delaying the phase of the signal supplied from the synthesizer 22 based on, for example, the control by the control unit 10. In this case, the phase control unit 23 may adjust the phase of each transmission signal based on the path difference of each transmission wave T transmitted from the multiple transmission antennas 25. By the phase control unit 23 appropriately adjusting the phase of each transmission signal, the transmission waves T transmitted from the multiple transmission antennas 25 reinforce each other in a predetermined direction to form a beam (beamforming). In this case, the correlation between the direction of beamforming and the phase amount to be controlled of the transmission signal transmitted by each of the multiple transmission antennas 25 may be stored in, for example, an arbitrary storage unit (memory). The transmission signal phase-controlled by the phase control unit 23 is supplied to the amplifier 24.

The amplifier 24 amplifies the power (electricity) of the transmission signal supplied from the phase control unit 23, for example, based on control by the control unit 10. If the sensor 5 has multiple transmission antennas 25, the multiple amplifiers 24 may each amplify the power (electricity) of the transmission signal supplied from a corresponding one of the multiple phase control units 23, for example, based on control by the control unit 10. Since the technology itself for amplifying the power of the transmission signal is already known, a more detailed explanation will be omitted. The amplifier 24 is connected to the transmission antenna 25.

The transmitting antenna 25 outputs (transmits) the transmission signal amplified by the amplifier 24 as a transmission wave T. If the sensor 5 is equipped with multiple transmitting antennas 25, the multiple transmitting antennas 25 may each output (transmit) the transmission signal amplified by a corresponding one of the multiple amplifiers 24 as a transmission wave T. The transmitting antenna 25 may be configured to include an antenna element such as a patch antenna. The transmitting antenna 25 can be configured in the same manner as a transmitting antenna used in known radar technology, and therefore a detailed description will be omitted.

In this way, the electronic device 1 according to one embodiment includes a transmitting antenna 25, and can transmit a transmission signal (e.g., a transmitting chirp signal) as a transmission wave T from the transmitting antenna 25. Here, at least one of the functional parts constituting the electronic device 1 may be housed in a single housing. In this case, the single housing may have a structure that cannot be easily opened. For example, the transmitting antenna 25, the receiving antenna 31, and the amplifier 24 may be housed in a single housing, and the housing may have a structure that cannot be easily opened. Furthermore, here, when the sensor 5 is installed in a moving body 100 such as an automobile, the transmitting antenna 25 may transmit the transmission wave T to the outside of the moving body 100 through a cover member such as a radar cover. In this case, the radar cover may be made of a material that allows electromagnetic waves to pass through, such as synthetic resin or rubber. This radar cover may be, for example, a housing for the sensor 5. By covering the transmitting antenna 25 with a member such as a radar cover, the risk of the transmitting antenna 25 being damaged or malfunctioning due to contact with the outside can be reduced. The radar cover and/or housing may also be called a radome.

The electronic device 1 shown in FIG. 2 shows an example having two transmitting antennas 25. However, in one embodiment, the electronic device 1 may have any number of transmitting antennas 25. On the other hand, in one embodiment, the electronic device 1 may have multiple transmitting antennas 25 when the transmitting wave T transmitted from the transmitting antenna 25 forms a beam in a predetermined direction. In one embodiment, the electronic device 1 may have any number of transmitting antennas 25. In this case, the electronic device 1 may also have multiple phase control units 23 and amplifiers 24 corresponding to the multiple transmitting antennas 25. The multiple phase control units 23 may each control the phase of the multiple transmitting waves supplied from the synthesizer 22 and transmitted from the multiple transmitting antennas 25. The multiple amplifiers 24 may each amplify the power of the multiple transmitting signals transmitted from the multiple transmitting antennas 25. In this case, the sensor 5 may be configured to include multiple transmitting antennas. In this way, when the electronic device 1 shown in FIG. 2 has multiple transmitting antennas 25, it may also be configured to include multiple functional units necessary for transmitting the transmitting wave T from the multiple transmitting antennas 25.

The receiving antenna 31 receives the reflected wave R. The reflected wave R may be the transmitted wave T reflected by a predetermined object 200. The receiving antenna 31 may be configured to include multiple antennas, such as receiving antennas 31A to 31D. The receiving antenna 31 may be configured to include an antenna element, such as a patch antenna. The receiving antenna 31 can be configured in the same manner as a receiving antenna used in known radar technology, so a detailed description will be omitted. The receiving antenna 31 is connected to the LNA 32. A received signal based on the reflected wave R received by the receiving antenna 31 is supplied to the LNA 32.

The electronic device 1 according to one embodiment can receive a reflected wave R that is a result of a transmission wave T transmitted as a transmission signal (transmission chirp signal) such as a chirp signal from a plurality of receiving antennas 31 and reflected by a predetermined object 200. In this way, when a transmission chirp signal is transmitted as a transmission wave T, a reception signal based on the received reflection wave R is referred to as a reception chirp signal. That is, the electronic device 1 receives a reception signal (e.g., a reception chirp signal) as a reflection wave R from the receiving antenna 31. Here, when the sensor 5 is installed in a moving body 100 such as an automobile, the receiving antenna 31 may receive a reflected wave R from the outside of the moving body 100 through a cover member such as a radar cover. In this case, the radar cover may be made of a material that allows electromagnetic waves to pass through, such as synthetic resin or rubber. This radar cover may be, for example, a housing for the sensor 5. By covering the receiving antenna 31 with a member such as a radar cover, the risk of the receiving antenna 31 being damaged or defective due to contact with the outside can be reduced. The above radar cover and/or housing may also be called a radome.

Furthermore, when the receiving antenna 31 is installed near the transmitting antenna 25, these may be configured together as one sensor 5. That is, one sensor 5 may include, for example, at least one transmitting antenna 25 and at least one receiving antenna 31. For example, one sensor 5 may include multiple transmitting antennas 25 and multiple receiving antennas 31. In such a case, one radar sensor may be covered with a cover member such as, for example, a radar cover.

The LNA 32 amplifies, with low noise, the received signal based on the reflected wave R received by the receiving antenna 31. The LNA 32 may be a low noise amplifier, and amplifies, with low noise, the received signal supplied from the receiving antenna 31. The received signal amplified by the LNA 32 is supplied to the mixer 33.

The mixer 33 generates a beat signal by mixing (multiplying) the RF frequency reception signal supplied from the LNA 32 with the transmission signal supplied from the synthesizer 22. The beat signal mixed by the mixer 33 is supplied to the IF unit 34.

The IF unit 34 performs frequency conversion on the beat signal supplied from the mixer 33, thereby lowering the frequency of the beat signal to an intermediate frequency (IF). The beat signal whose frequency has been lowered by the IF unit 34 is supplied to the AD conversion unit 35.

The AD conversion unit 35 digitizes the analog beat signal supplied from the IF unit 34. The AD conversion unit 35 may be configured with any analog-to-digital conversion circuit (Analog to Digital Converter (ADC)). The beat signal digitized by the AD conversion unit 35 is supplied to the distance FFT processing unit 11 of the control unit 10. When there are multiple receiving units 30, each of the beat signals digitized by the multiple AD conversion units 35 may be supplied to the distance FFT processing unit 11 of the control unit 10.

As shown in FIG. 3, in the control unit 10, the distance FFT processing unit 11 estimates the distance between the moving body 100 mounting the electronic device 1 and the object 200 based on the beat signal supplied from the AD conversion unit 35. The distance FFT processing unit 11 may include, for example, a processing unit that performs a fast Fourier transform. In this case, the distance FFT processing unit 11 may be configured with any circuit, chip, and/or software that performs fast Fourier transform (FFT) processing.

The distance FFT processing unit 11 performs FFT processing on the beat signal digitized by the AD conversion unit 35 (hereinafter, appropriately referred to as "distance FFT processing"). For example, the distance FFT processing unit 11 may perform FFT processing on the complex signal supplied from the AD conversion unit 35. The beat signal digitized by the AD conversion unit 35 can be expressed as a time change in signal strength (power). The distance FFT processing unit 11 can express the beat signal as a signal strength (power) corresponding to each frequency by performing FFT processing on such a beat signal. If a peak is equal to or greater than a predetermined threshold in the result obtained by the distance FFT processing, the distance FFT processing unit 11 may determine that a predetermined object 200 is present at a distance corresponding to the peak. For example, a method is known in which, when a peak value equal to or greater than a threshold is detected from the average power or amplitude of a disturbance signal, an object reflecting a transmission wave (a reflecting object) is present, such as a detection process using a constant false alarm rate (CFAR). The determination of whether an object exists or not based on such a threshold value may be performed, for example, by the threshold determination unit 13 described below.

In this way, the electronic device 1 according to one embodiment can detect an object 200 reflecting a transmission wave T as a target based on a transmission signal transmitted as a transmission wave T and a reception signal received as a reflected wave R.

The distance FFT processing unit 11 can estimate the distance to a predetermined object based on one chirp signal (e.g., c1 shown in FIG. 4). That is, the electronic device 1 can measure (estimate) the distance A shown in FIG. 1 by performing distance FFT processing. The technology for measuring (estimating) the distance to a predetermined object by performing FFT processing on a beat signal is well known, so a more detailed explanation will be simplified or omitted as appropriate. The result of the distance FFT processing performed by the distance FFT processing unit 11 (e.g., distance information) may be supplied to the velocity FFT processing unit 12. In addition, the result of the distance FFT processing performed by the distance FFT processing unit 11 may be supplied to other functional units, such as the threshold determination unit 13.

The velocity FFT processing unit 12 estimates the relative velocity between the moving body 100 carrying the electronic device 1 and the object 200 based on the beat signal on which the distance FFT processing has been performed by the distance FFT processing unit 11. The velocity FFT processing unit 12 may include, for example, a processing unit that performs a fast Fourier transform. In this case, the velocity FFT processing unit 12 may be configured with any circuit, chip, and/or software that performs fast Fourier transform (FFT) processing.

The velocity FFT processing unit 12 further performs FFT processing on the beat signal on which the distance FFT processing has been performed by the distance FFT processing unit 11 (hereinafter, appropriately referred to as "velocity FFT processing"). For example, the velocity FFT processing unit 12 may perform FFT processing on the complex signal supplied from the distance FFT processing unit 11. The velocity FFT processing unit 12 can estimate the relative velocity with respect to a predetermined object based on a subframe of the chirp signal (for example, subframe 1 shown in FIG. 4). When the distance FFT processing is performed on the beat signal as described above, multiple vectors can be generated. The relative velocity with respect to a predetermined object can be estimated by determining the phase of the peak in the result of performing velocity FFT processing on these multiple vectors. In other words, the electronic device 1 can measure (estimate) the relative velocity between the moving body 100 and the predetermined object 200 shown in FIG. 1 by performing velocity FFT processing. The technology itself for measuring (estimating) the relative velocity with respect to a predetermined object by performing velocity FFT processing on the result of performing distance FFT processing is well known, so a more detailed description will be simplified or omitted as appropriate. The result of the velocity FFT processing performed by the velocity FFT processing unit 12 (e.g., velocity information) may be supplied to the threshold determination unit 13. In addition, the result of the velocity FFT processing performed by the velocity FFT processing unit 12 may be supplied to another functional unit, such as the arrival angle estimation unit 14.

The threshold determination unit 13 performs a determination process for the distance and/or the relative velocity based on the result of the distance FFT processing performed by the distance FFT processing unit 11 and/or the result of the velocity FFT processing performed by the velocity FFT processing unit 12. Here, the threshold determination unit 13 may perform a determination based on a predetermined threshold. For example, the threshold determination unit 13 may determine whether or not the result of the distance FFT processing performed by the distance FFT processing unit 11 and/or the result of the velocity FFT processing performed by the velocity FFT processing unit 12 exceeds a predetermined threshold. The threshold determination unit 13 may determine that an object has been detected at a distance and/or relative velocity that exceeds the predetermined threshold.

The threshold determination unit 13 may output only the results of the distance FFT processing performed by the distance FFT processing unit 11 and/or the results of the speed FFT processing performed by the speed FFT processing unit 12 that exceed a predetermined threshold. The operation performed by the threshold determination unit 13 may be similar to a detection process based on a constant false alarm rate (CFAR). In one embodiment, the operation performed by the threshold determination unit 13 may be a process based on order statistics CFAR (OS-CFAR). OS-CFAR is a method of setting a threshold based on ordered statistics and determining that a target exists when the threshold is exceeded. The result of the threshold determination process performed by the threshold determination unit 13 may be supplied to the angle of arrival estimation unit 14. The result of the process performed by the threshold determination unit 13 may also be supplied to other functional units, such as the object detection unit 15.

The arrival angle estimation unit 14 estimates the direction in which the reflected wave R arrives from the specified object 200 based on the result of the velocity FFT processing performed by the velocity FFT processing unit 12 and/or the output from the threshold judgment unit 13. The arrival angle estimation unit 14 may estimate the direction in which the reflected wave R arrives from the specified object 200 based on the result output from the threshold judgment unit 13 among the results of the velocity FFT processing performed by the velocity FFT processing unit 12. The electronic device 1 can estimate the direction in which the reflected wave R arrives by receiving the reflected wave R from the multiple receiving antennas 31. For example, the multiple receiving antennas 31 are arranged at a predetermined interval. In this case, the transmission wave T transmitted from the transmitting antenna 25 is reflected by the specified object 200 and becomes the reflected wave R, and the multiple receiving antennas 31 arranged at a predetermined interval each receive the reflected wave R. The arrival angle estimation unit 14 can estimate the direction in which the reflected wave R arrives at the receiving antenna 31 based on the phase of the reflected wave R received by each of the multiple receiving antennas 31 and the path difference of each reflected wave R. In other words, the electronic device 1 can measure (estimate) the arrival angle θ shown in FIG. 1 based on the result of the speed FFT processing.

In the electronic device 1 according to one embodiment, the arrival angle estimation unit 14 may estimate the arrival direction of the reflected wave based on the complex signals received by the multiple receiving antennas 31 at the speed at which it is determined that an object is present. In this way, the electronic device 1 according to one embodiment can estimate the angle of the direction in which the object is present.

In the electronic device 1 according to one embodiment, the arrival angle estimation unit 14 may perform arrival direction estimation when the signal strength (quality) of a detection point determined by the threshold determination unit 13 to have exceeded the threshold is equal to or less than the threshold.

Various techniques have been proposed for estimating the direction of arrival of the reflected wave R based on the results of velocity FFT processing. For example, known algorithms for estimating the direction of arrival include MUSIC (MUltiple SIgnal Classification) and ESPRIT (Estimation of Signal Parameters via Rotational Invariance Technique). Therefore, detailed descriptions of known techniques will be simplified or omitted as appropriate. Information on the arrival angle θ (angle information) estimated by the arrival angle estimation unit 14 may be supplied to the object detection unit 15.

In the electronic device 1 according to one embodiment, the object detection unit 15 determines whether or not an object has been detected as a target (e.g., clustered) based on information on the direction of arrival (angle) of the reflected wave, information on the relative speed with respect to the target, and/or information on the distance to the target. Here, the information on the direction of arrival (angle) of the reflected wave may be acquired from the angle of arrival estimation unit 14. Information on the relative speed and distance with respect to the target may be acquired from the threshold determination unit 13. Information on the relative speed with respect to the target may be acquired from the speed FFT processing unit 12. Information on the distance to the target may be acquired from the distance FFT processing unit 11. The object detection unit 15 may calculate the average power of the points constituting the object detected as a target. Information on the direction of arrival (angle) of the reflected wave, information on the relative speed with respect to the target, and/or information on the distance to the target may be stored in an arbitrary storage unit (memory) for each frame, for example.

The object detection unit 15 detects an object present within the range where the transmission wave T is transmitted, based on information supplied from at least one of the distance FFT processing unit 11, the speed FFT processing unit 12, the threshold determination unit 13, and the arrival angle estimation unit 14. The object detection unit 15 may perform object detection by, for example, performing clustering processing based on the supplied distance information, speed information, and angle information. As an algorithm used for clustering data, for example, DBSCAN (Density-based spatial clustering of applications with noise) is known. In the clustering processing, for example, the average power of the points constituting the detected object may be calculated. The distance information, speed information, angle information, and power information of the object detected by the object detection unit 15 may be supplied to the tracking processing unit 16. In addition, the output from the object detection unit 15 may be supplied to other functional units, such as the ECU 50. In this case, when the moving body 100 is an automobile, communication may be performed using a communication interface such as a CAN (Controller Area Network).

The tracking processing unit 16 performs processing to predict the target position in the next frame of the object that has been clustered by the object detection unit 15. The tracking processing unit 16 may predict the position in the next frame of the object that has been clustered by using, for example, a Kalman filter. The tracking processing unit 16 may store the predicted position of the object in the next frame in, for example, the memory unit 17.

The storage unit 17 may be configured, for example, from a semiconductor memory or a magnetic disk, but is not limited to these and may be any storage device. For example, the storage unit 17 may be a storage medium such as a memory card inserted into the electronic device 1 according to this embodiment. The storage unit 17 may be an internal memory of the CPU used as the control unit 10, as described above. The storage unit 17 may store programs executed in the control unit 10 and results of processing executed in the control unit 10. The storage unit 17 may function as a work memory for the control unit 10. In one embodiment, the storage unit 17 may be any storage unit (memory) described above.

In one embodiment, the storage unit 17 may store various parameters for setting the range in which an object is detected using the transmission wave T transmitted from the transmission antenna 25 and the reflected wave R received from the reception antenna 31.

In one embodiment, the tracking processing unit 16 may perform data association between the predicted position in the previous frame and the clustering position detected in the current frame. Here, the tracking processing unit 16 may impose restrictions on, for example, the minimum Euclidean distance or the minimum Mahalanobis distance, or on speed. In this way, the tracking processing unit 16 may perform time frame association.

The tracking processing unit 16 may also perform interframe association based on the correlation between object information (distance, angle, and/or speed) predicted in the previous frame (stored in any storage unit (memory)) and object information observed in the current frame. In this case, the tracking processing unit 16 may store the associated object information in any storage unit (memory) such as the storage unit 17. Here, the associated object information may be, for example, information calculated by a Kalman filter, and may be information on the forward/backward (vertical) speed and/or left/right (horizontal) speed of the detected object. The forward/backward (vertical) direction of the detected object may be, for example, a direction including a component parallel to the propagation direction of the transmitted wave and/or the reflected wave. On the other hand, the left/right (horizontal) direction of the detected object may be, for example, a direction including a component perpendicular to the propagation direction of the transmitted wave and/or the reflected wave.

When performing the inter-frame association as described above, the tracking processing unit 16 may accumulate the cumulative probability density function (PDF) of the point cloud velocity in an arbitrary storage unit (memory) such as the storage unit 17 during different time frames. In addition, in one embodiment, the PDF of the reference velocity for each class measured in advance by a millimeter wave radar may be accumulated in an arbitrary storage unit (memory) such as the storage unit 17. The tracking processing unit 16 may calculate the Pearson divergence based on the PDF of the reference velocity for each class accumulated in the memory and the PDF of the point cloud velocity of the detected object. The Pearson divergence calculated in this manner is also referred to as the Pearson value or PE value. The PDF of the point cloud velocity of the detected object may also be stored in an arbitrary storage unit (memory) such as the storage unit 17. The tracking processing unit 16 may predict the next frame using a Kalman filter from the information of the object observed in the current frame associated as described above. The tracking processing unit 16 may then store the object information thus obtained in any storage unit (memory), such as the storage unit 17.

Furthermore, in one embodiment, the tracking processing unit 16 may accumulate velocity information of point clouds indicating the same target in, for example, the memory unit 17. Then, when velocity information of point clouds indicating the same target has been accumulated in memory for the required number of point clouds and the required number of frames, the tracking processing unit 16 may calculate a probability density distribution of the speed. Furthermore, the memory unit 17 may store in advance a reference probability density distribution (reference probability density distribution) for each target. Hereinafter, the reference probability density distribution of a target stored in a memory such as the memory unit 17 is also referred to as a "reference probability density distribution" as appropriate.

The object identification unit 18 identifies whether the object is a predetermined target based on the information of the detected object. For example, the object identification unit 18 may identify whether the object is a pedestrian based on the information of the detected object. For this purpose, the object identification unit 18 may calculate a probability density distribution based on a speed that satisfies the required number of point clouds and the required number of frames in a frame. Furthermore, the object identification unit 18 may identify the target based on a reference probability density distribution stored in a memory such as the memory unit 17 and a probability density distribution calculated from a speed that satisfies the required number of point clouds and the required number of frames in the frame. Here, the object identification unit 18 may calculate a Pearson divergence from the reference probability density distribution stored in the memory and the calculated probability density distribution. The object identification unit 18 may identify a target whose calculated Pearson divergence is equal to or less than a predetermined threshold as a predetermined target (e.g., a pedestrian). The object identification unit 18 may output the result of identifying the object in this manner (detection result) to, for example, the ECU 50.

The object identification unit 18 may perform sequential updates and calculate the probability density distribution in frames after the required number of point clouds and the required number of frames are met. Also, in a memory such as the storage unit 17, information on old point clouds may be deleted.

The ECU 50 provided in the electronic device 1 according to one embodiment can control the operation of the entire mobile body 100, including the control of each functional unit constituting the mobile body 100. The ECU 50 may include at least one processor, such as a CPU (Central Processing Unit) or a DSP (Digital Signal Processor), to provide control and processing power for executing various functions. The ECU 50 may be realized as a single processor, or may be realized as several processors, or each may be realized as an individual processor. The processor may be realized as a single integrated circuit. The integrated circuit is also called an IC (Integrated Circuit). The processor may be realized as multiple integrated circuits and discrete circuits connected to each other in a communicative manner. The processor may be realized based on various other known technologies. In one embodiment, the ECU 50 may be configured as, for example, a CPU and a program executed by the CPU. The ECU 50 may include memory necessary for the operation of the ECU 50 as appropriate. Additionally, at least some of the functions of the control unit 10 may be functions of the ECU 50, and at least some of the functions of the ECU 50 may be functions of the control unit 10.

The electronic device 1 shown in FIG. 2 has two transmitting antennas 25 and four receiving antennas 31. However, the electronic device 1 according to one embodiment may have any number of transmitting antennas 25 and any number of receiving antennas 31. For example, by having two transmitting antennas 25 and four receiving antennas 31, the electronic device 1 can be considered to have a virtual antenna array consisting of eight virtual antennas. In this way, the electronic device 1 may receive the reflected waves R of the 16 subframes shown in FIG. 4 by using, for example, eight virtual antennas.

Next, we will explain the object label-specific processing performed by the electronic device 1 according to one embodiment.

As described above, the electronic device 1 according to one embodiment transmits a transmission wave from a transmitting antenna, and receives a reflected wave, which is the transmission wave reflected by an object, from a receiving antenna. Then, the electronic device 1 according to one embodiment can detect an object reflecting the transmission wave based on the transmission signal and/or the reception signal. The electronic device 1 according to one embodiment identifies whether or not the object detected in this manner is a predetermined target. In particular, the electronic device 1 according to one embodiment determines whether or not the object detected in the manner described above is a pedestrian. The algorithm for such processing will be further explained below.

The electronic device 1 according to one embodiment performs target identification processing based on the Doppler velocity observed by the radar. Here, the Doppler velocity observed by the radar of the electronic device 1 may be, for example, the relative velocity of an object detected by the electronic device 1 with respect to the electronic device 1. According to the electronic device 1, one Doppler velocity is observed for each object in one frame, such as frame 1 shown in FIG. 4. Thus, the electronic device 1 may continuously acquire and analyze such Doppler velocities for a predetermined period of time or more. The electronic device 1 performs target identification processing based on such analysis.

　Target objects identified in the electronic device 1 according to one embodiment may be classified into classes and subclasses. Here, the target class may indicate the type of target object. For example, the target class may indicate the type of target object, such as whether the target object is a car, a motorcycle, a bicycle, a pedestrian, or the like. That is, the electronic device 1 according to one embodiment may determine whether the target object class is a pedestrian class. Furthermore, the target subclass may be a subclassification of the target object in each target class and/or a motion state of the target object. For example, the target subclass may be a subclassification of pedestrians, such as adult pedestrians, child pedestrians, and elderly pedestrians. Furthermore, the target subclass may be a subclassification of pedestrians, such as a slow-walking pedestrian and a brisk-walking pedestrian, as a motion state of the pedestrian.

In addition, the electronic device 1 according to an embodiment may calculate a probability density function (PDF) from the Doppler velocity continuously acquired as described above. For example, the electronic device 1 may organize the list V _r of velocity distributions acquired for the l (lowercase letter L)-th target into a frequency distribution table, normalize the total frequency to 1, and set this as the probability density distribution p _t ^l . Hereinafter, the probability density distribution thus obtained is also referred to as "observation-based probability density distribution" as appropriate. In addition, the electronic device 1 according to an embodiment may calculate a reference probability density distribution p _ri ^,k in advance and store it in an arbitrary memory such as the storage unit 17. The reference probability density distribution p _ri ^,k may be stored in an arbitrary location such as the electronic device 1 or an external processor or storage.

FIG. 5 is a diagram showing an example of a reference probability density distribution of a pedestrian. The horizontal axis of FIG. 5 indicates the speed, and the vertical axis indicates the probability. FIG. 5 may show an example of a reference probability density distribution (PDF) of the micro-Doppler velocities of people walking in various ways. Pedestrian 1 to pedestrian 4 shown in FIG. 5 may each indicate a different pedestrian, or may indicate the same person walking in different ways. The electronic device 1 according to an embodiment generates and stores a reference probability density distribution (PDF) such as that shown in FIG. 5. Although the reference probability density distributions (PDF) of four pedestrians are shown in FIG. 5, the electronic device 1 according to an embodiment may store (accumulate) reference probability density distributions of more pedestrians.

Here, the micro-Doppler velocity measured from a pedestrian will be described. FIG. 6 is a diagram for describing the micro-Doppler velocity measured from a pedestrian. As shown in FIG. 6, a situation is assumed in which a pedestrian Ped is present within the range of the millimeter waves from the electronic device 1. As shown in FIG. 6, a situation is assumed in which the pedestrian Ped is present in front of the electronic device 1. In FIG. 6, the electronic device 1 transmits a transmission wave T including a component in a direction parallel to the Y axis. At least a part of this transmission wave T is reflected by the pedestrian Ped and becomes a reflected wave R. The electronic device 1 receives the reflected wave R including a component in a direction parallel to the Y axis. In FIG. 6, the direction parallel to the Y axis may be referred to as the front-back (vertical) direction. Also, in FIG. 6, the direction parallel to the X axis may be referred to as the left-right (horizontal) direction.

As shown in FIG. 6, the walker Ped not only has a moving speed of the whole body, but also has a speed of, for example, swinging both arms and both legs. These speeds reflect different movements of each arm of the walker Ped (typically a person) and/or movements of each leg in different directions. The reflected wave R received by the electronic device 1 reflects multiple movements of the walker Ped, such as (both) arms and (both) legs. That is, the received signal received by the electronic device 1 as the reflected wave R reflects the movements of multiple parts of the object that reflects the transmitted wave. Therefore, the electronic device 1 detects not only the micro-Doppler speed of the movement of the walker Ped's whole body, but also the micro-Doppler speed of, for example, swinging both arms and both legs. Therefore, the probability density distribution as shown in FIG. 5 is calculated based on the micro-Doppler speed (first speed) detected from the whole body including each part of each walker's body. Here, the first speed may be the speed of multiple parts of an object that reflects a transmitted wave relative to the electronic device 1.

Next, the operation of the electronic device 1 according to one embodiment will be described. First, the operations that are the prerequisite for the process of identifying pedestrians performed by the electronic device 1 according to one embodiment will be described.

FIG. 7 is a flowchart illustrating the operations that are the basis for the process of identifying a pedestrian by the electronic device 1 according to one embodiment. FIG. 7 may show the process at time t that is executed by the electronic device 1 according to one embodiment.

When the process shown in FIG. 7 starts, the control unit 10 of the electronic device 1 transmits a transmission wave T from the transmission antenna 25 (step S11) and receives a reflected wave R, which is the transmission wave T reflected by an object, from the receiving antenna 31 (step S12).

Then, the control unit 10 detects an object reflecting the transmission wave T based on the transmission signal transmitted as the transmission wave T and the reception signal received as the reflected wave R (step S13). In step S13, the distance FFT processing unit 11 and the velocity FFT processing unit 12 may each perform FFT processing on the beat signal. Furthermore, the threshold determination unit 13 may perform processing based on, for example, a CFAR threshold. Furthermore, the arrival angle estimation unit 14 may estimate the direction from which the reflected wave R arrives. Furthermore, the object detection unit 15 may detect an object reflecting the transmission wave T based on information about the detected object (distance, angle, and velocity), for example through a predetermined clustering process.

Next, the control unit 10 acquires the observation values of the cluster in order to track the target (step S14). In step S14, the control unit 10 may acquire the position of the target and the speed (relative speed) of the target calculated from the position.

The control unit 10 then performs data association between the posterior estimates and observed values at the previous time (time t-1) (step S15). The data association in step S15 may involve comparing the observed values of the object's position and velocity calculated from the position acquired at time t with the posterior estimates calculated by a Bayesian inference processing method such as a Kalman filter at the previous time t-1.

Then, the control unit 10 performs a tracking process using, for example, a Kalman filter (step S16). In step S16, the control unit 10 may execute a process of calculating a predicted value (pre-estimated value) and a process of calculating a posterior estimated value. In the process of calculating a predicted value (pre-estimated value), the control unit 10 may derive a predicted value (pre-estimated value) for the current time t from a posterior estimated value for the previous time t-1. In addition, in the process of calculating a posterior estimated value, the control unit 10 may calculate a posterior estimated value by Bayesian estimation that combines the predicted value (pre-estimated value) for the current time t with the observed value acquired at the current time t that is data-associated with the data.

The control unit 10 may then store the results of the tracking process in step S16 in any storage unit (memory) such as the storage unit 17 (step S17). The operation shown in FIG. 7 may be repeatedly executed, for example, at predetermined time intervals. For example, the electronic device 1 according to one embodiment may repeatedly execute the operation shown in FIG. 7 for each frame of the transmission wave T, for each subframe of the transmission wave T, or for each chirp signal of the transmission wave T.

Next, the operation of identifying pedestrians using the electronic device 1 according to one embodiment will be described.

FIG. 8 is a flowchart illustrating the operation of identifying a pedestrian by the electronic device 1 according to one embodiment. The operation shown in FIG. 8 may be executed in parallel with the operation shown in FIG. 7, or may be executed subsequent to the operation shown in FIG. 7.

When the operation shown in FIG. 8 starts, the control unit 10 determines the direction of movement of the detected object (step S21). Here, in step S21, the control unit 10 determines whether, for example, the pedestrian Ped shown in FIG. 6 is moving mainly in the front-back (vertical) direction (i.e., the Y-axis direction) relative to the electronic device 1, or mainly in the left-right (horizontal) direction (i.e., the X-axis direction) relative to the electronic device 1.

In step S21, the control unit 10 may compare the forward/backward (vertical) velocity component (i.e., Y-axis direction) of the object calculated by, for example, a Kalman filter with the left/right (horizontal) velocity component (i.e., X-axis direction). Here, the forward/backward (vertical) velocity component calculated as a result of processing by the Kalman filter is denoted as Vy, and the left/right (horizontal) velocity component calculated as a result of processing by the Kalman filter is denoted as Vx. In this case, in step S21, the control unit 10 may determine, for example, whether the absolute value of Vy multiplied by a constant C is greater than the absolute value of Vx (i.e., |Vy|*C>|Vx| holds). In this way, in step S21, the control unit 10 may determine whether the object is moving forward/backward with respect to the propagation direction of the transmission wave based on information predicted as the temporal displacement of the object. That is, in step S21, the control unit 10 may determine whether the object is moving forward or backward relative to the propagation direction of the transmission wave, based on first position information of the object detected at a first time and second position information that is predicted as the position of the object at a second time that is later than the first time.

In step S21, if the constant C times the absolute value of Vy is greater than the absolute value of Vx (|Vy|*C>|Vx| holds), the control unit 10 may determine that the detected object is moving mainly in the forward/backward (vertical) direction (Yes in step S22). On the other hand, if the constant C times the absolute value of Vy is not greater than the absolute value of Vx (|Vy|*C>|Vx| does not hold), the control unit 10 may determine that the detected object is moving mainly in the left/right (horizontal) direction (No in step S22).

If the object is moving mainly in the forward/backward (vertical) direction (Yes in step S22), the control unit 10 calculates the probability density function (PDF) of the micro-Doppler velocity of the points that make up the cluster (step S23).

Next, the control unit 10 calculates the Pearson divergence based on the PDF calculated in step S23 and the reference probability density distribution (PDF) as illustrated in FIG. 5 (step S24). In step S24, the control unit 10 may calculate the Pearson divergence according to the calculation formulas shown in the following formulas (1) and (2). However, the class classification conditions shall satisfy the following formulas (3) and (4). In one embodiment, several reference probability density distributions (PDFs) of the target class may be prepared according to the conditions. In this way, the reference probability density distributions (PDFs) of the target class prepared according to the conditions may be subclasses. Here, the subclass may indicate, for example, the operating state of the moving target, such as the direction/speed, the size of the target, and/or the type of the target, or other subclassifications. Examples of subclasses include, for example, in the case of pedestrians, the walking speed of the pedestrian (slow walking, brisk walking, normal walking speed, etc.), whether the pedestrian is an adult or a child, and the walking direction of the pedestrian (parallel or perpendicular to the radar normal of the electronic device 1). In one embodiment, a reference probability density distribution (PDF) and a relative Pearson value of the pedestrian's speed may be calculated for each target object, and whether the target object is a pedestrian or not may be identified. Below, identification for the kth (k=1, ..., K) object class is shown. However, an example is shown in which the classes are limited to two classes, "pedestrian" and "other".

A relative Pearson value may be calculated for I references of the kth class shown in the above formula (1). Then, a search may be performed to see if there is a reference probability density distribution (PDF) that is equal to or smaller than the threshold shown in the above formula (3), and the PDF may be added to the object class (k). From the object classes shown in the above formula (4), the one with the smallest relative Pearson value for p _t ^j indicating the observed PDF may be selected. In each of the above formulas, i indicates a subclass (operation state, small classification) of the kth class. j indicates the index of the observed target. Γ indicates a set of k, i that satisfies the class classification conditions. m indicates the speed index. α indicates the forgetting factor.

FIG. 9 is a diagram showing an example of the Pearson divergence calculated in step S24. FIG. 9 may show the Pearson divergence calculated based on the PDF of the micro-Doppler velocity of the detected object and a reference probability density distribution (PDF) such as that shown in FIG. 5. In the graph shown in FIG. 9, the horizontal axis represents the frame time, and the vertical axis represents the value of the Pearson divergence, i.e., the Pearson value (or PE value).

In FIG. 9, the Pearson values for pedestrians 1 to 4 are shown as an example. The lower the Pearson value in FIG. 9, the higher the correlation between the PDF of the micro-Doppler velocity of the detected object and the reference probability density distribution (PDF) as shown in FIG. 5. In addition, in the example shown in FIG. 9, the Pearson values have not converged because there is insufficient data up to the seventh frame. As shown in FIG. 9, the Pearson values for pedestrians 1 to 4 converge as the number of frames increases. Therefore, data that has not converged, for example, up to the seventh frame, does not need to be taken into account (can be ignored) in subsequent processing.

In the Pearson values for pedestrians 1 to 4 shown in Figure 9, the Pearson value for pedestrian 4 is the lowest among the converged values. Therefore, Figure 9 shows that the reference probability density distribution (PDF) for pedestrian 4 has the highest correlation with the PDF of the micro-Doppler velocity of the detected object.

After the Pearson value is calculated in step S24, the control unit 10 determines whether the calculated Pearson value is equal to or less than the pedestrian threshold (the threshold corresponding to a pedestrian) (step S25). If the calculated Pearson value is equal to or less than the threshold corresponding to a pedestrian, the control unit 10 may determine that the detected object belongs to the pedestrian class, i.e., that the detected object is a pedestrian (step S26).

For example, as shown in FIG. 9, the Pearson value of pedestrian 4 converges to the lowest value, but the minimum value of the Pearson value of this pedestrian 4 is equal to or less than the threshold th shown in FIG. 9. In this case, the control unit 10 may determine in step S26 that the detected object belongs to the pedestrian class, i.e., that the detected object is a pedestrian. In FIG. 9, the pedestrian threshold th of the Pearson value (threshold corresponding to a pedestrian) is set to a value of approximately 0.3. However, the pedestrian threshold th of the Pearson value (threshold corresponding to a pedestrian) may be set to an appropriate value, for example, through experiments, etc.

On the other hand, if the calculated Pearson value exceeds the threshold value corresponding to a pedestrian in step S25, the control unit 10 may determine that the detected object belongs to a class other than a pedestrian, i.e., the detected object is not a pedestrian (step S27).

Also, if the answer is No in step S22, that is, if it is determined that the detected object is moving mainly in the left-right (horizontal) direction, the control unit 10 calculates the average speed from the left-right (horizontal) speed (step S28). That is, in step S28, the control unit 10 may calculate the average speed from the above-mentioned Vx and Vy. Here, Vx and Vy are the forward-backward (vertical) speed component and the left-right (horizontal) speed component calculated as the result of processing by the Kalman filter. Hereinafter, the average speed calculated in step S28 is also referred to as the "second speed" of the object. Here, the second speed of the object may be calculated based on the first position information of the object detected at the first time and the second position information predicted as the position of the object at a second time later than the first time.

Next, the control unit 10 determines whether the average speed (second speed) calculated from the above Vx and Vy is within the speed range of the pedestrian (step S29).

If the average speed calculated from Vx and Vy in step S29 is within the speed range of a pedestrian, the control unit 10 may determine that the detected object belongs to a pedestrian class, i.e., that the detected object is a pedestrian (step S26). Also, if the average speed calculated from Vx and Vy in step S29 (second speed) is not within the speed range of a pedestrian, the control unit 10 may determine that the detected object belongs to a class other than a pedestrian, i.e., that the detected object is not a pedestrian (step S27).

The threshold value for determining whether the average speed calculated from Vx and Vy in step S29 is within the pedestrian's speed range may be set to an appropriate value, for example, through experiments. The operation shown in FIG. 8 may be repeatedly executed, for example, at predetermined time intervals. For example, the electronic device 1 according to one embodiment may repeatedly execute the operation shown in FIG. 8 for each frame of the transmission wave T, etc.

As described above, the electronic device 1 according to one embodiment may be realized as a device such as a millimeter wave radar that transmits and receives radio waves using multiple antennas. The electronic device 1 according to one embodiment may perform density-based clustering on a point cloud obtained by estimating the distance, relative speed, and angle of arrival of a detected object.

The control unit 10 of the electronic device 1 according to an embodiment may execute a first process for identifying an object as a person by reflecting the motions of multiple parts of the object in a received signal received as a reflected wave of a transmission signal transmitted as a transmission wave reflected by the object. Here, the control unit 10 may execute the above-mentioned first process by reflecting a first speed, which is the speed of multiple parts of the object relative to the electronic device 1, in a received signal received as a reflected wave. The control unit 10 may execute the above-mentioned first process when the object is moving forward or backward with respect to the propagation direction of the transmission wave. The control unit 10 may execute the above-mentioned first process based on the correlation between the probability density distribution calculated from the first speed and the reference probability density distribution. The control unit 10 may execute the above-mentioned first process when the value of the Pearson divergence, which indicates the correlation and is calculated from the probability density distribution calculated from the first speed and the reference probability density distribution, is equal to or less than a threshold value.

The control unit 10 may also execute a second process to identify an object as a person when the object is moving left or right with respect to the propagation direction of the transmission wave. In this case, the control unit 10 may execute the second process described above based on first position information of the object detected at a first time and a second speed of the object calculated based on second position information that predicts the position of the object at a second time that is later than the first time.

The electronic device 1 according to one embodiment may use a Kalman filter to predict a point cloud by clustering processing for the next frame, and associate the predicted point cloud with the point cloud that constitutes the cluster of the detected object. The electronic device 1 according to one embodiment may determine whether the absolute value of the forward/backward moving speed (Vy) is greater than the absolute value of the lateral moving speed (Vx) by a threshold percentage or more, based on the average moving speed of the object predicted using the Kalman filter as described above. The electronic device 1 according to one embodiment may store the point cloud of the detected points associated using the Kalman filter as information indicating the same object, and accumulate the cumulative probability density distribution (PDF) of the speed of the point cloud in a memory such as the storage unit 17.

The electronic device 1 according to an embodiment may calculate the Pearson divergence of the PDF of the micro-Doppler velocity for each class stored in advance in a memory such as the storage unit 17 and the reference probability density distribution (PDF). When the value of the Pearson divergence (Pearson value) calculated as described above is equal to or less than a predetermined threshold, the electronic device 1 according to an embodiment may determine that the object is of a class such as a pedestrian. When the absolute value of the forward/backward moving speed (Vy) is smaller than a threshold compared to the absolute value of the lateral moving speed (Vx), the electronic device 1 according to an embodiment may determine that the object is moving in the lateral direction, and identify the class of the object from the moving average speed obtained by Kalman filtering.

The electronic device 1 according to one embodiment determines whether a detected object is moving laterally, and if it is moving forward or backward, it can perform high-speed identification by identifying it based on the micro-Doppler velocity. Furthermore, according to the electronic device 1 according to one embodiment, for lateral velocity, which is generally difficult to detect using radar technology, it is possible to identify an object based on the velocity from a positioning position in which data is associated between frames using a Kalman filter. Generally, in devices such as millimeter wave radar, identifying an object using multiple feature amounts tends to be difficult to implement due to the large processing load. However, the electronic device 1 according to one embodiment can provide an electronic device, a control method for an electronic device, and a program that can improve the accuracy of target detection.

Although the present disclosure has been described based on the drawings and examples, it should be noted that a person skilled in the art can easily make various modifications or corrections based on the present disclosure. Therefore, it should be noted that these modifications or corrections are included in the scope of the present disclosure. For example, the functions included in each functional unit can be rearranged so as not to cause logical inconsistencies. Multiple functional units, etc. may be combined into one or divided. Each embodiment of the present disclosure described above is not limited to being implemented faithfully to each of the embodiments described, and may be implemented by combining each feature as appropriate or omitting some of them. In other words, the contents of the present disclosure can be modified and corrected in various ways by a person skilled in the art based on the present disclosure. Therefore, these modifications and corrections are included in the scope of the present disclosure. For example, in each embodiment, each functional unit, each means, each step, etc. can be added to other embodiments so as not to cause logical inconsistencies, or replaced with each functional unit, each means, each step, etc. of other embodiments. In addition, in each embodiment, multiple functional units, each means, each step, etc. can be combined into one or divided. Furthermore, each of the above-described embodiments of the present disclosure is not limited to being implemented faithfully according to each of the described embodiments, but may be implemented by combining each feature or omitting some features as appropriate.

The above-described embodiment is not limited to implementation as the electronic device 1. For example, the above-described embodiment may be implemented as a control method for a device such as the electronic device 1. Furthermore, for example, the above-described embodiment may be implemented as a program executed by a device such as the electronic device 1 or a computer, or as a storage medium or recording medium on which a program is recorded.

The electronic device 1 according to one embodiment may have, as a minimum configuration, at least a part of only one of the sensor 5 or the control unit 10. On the other hand, the electronic device 1 according to one embodiment may be configured to include at least one of the signal generating unit 21, the synthesizer 22, the phase control unit 23, the amplifier 24, and the transmitting antenna 25 as shown in FIG. 2 in addition to the control unit 10. The electronic device 1 according to one embodiment may also be configured to include at least one of the receiving antenna 31, the LNA 32, the mixer 33, the IF unit 34, and the AD conversion unit 35 in place of or in addition to the above-mentioned functional units. Furthermore, the electronic device 1 according to one embodiment may be configured to include an arbitrary storage unit (memory). In this way, the electronic device 1 according to one embodiment can have various configurations. When the electronic device 1 according to one embodiment is mounted on the moving body 100, for example, at least one of the above-mentioned functional units may be installed in an appropriate location such as inside the moving body 100. On the other hand, in one embodiment, for example, at least one of the transmitting antenna 25 and the receiving antenna 31 may be installed outside the mobile body 100.

In the above-described embodiment, the electronic device 1 identifies whether or not the detected object is a pedestrian using the value of the Pearson divergence, which indicates the correlation between the probability density function of the detected object and the reference probability density distribution. However, in one embodiment, the control unit 10 may identify whether or not the detected object is a pedestrian based on the speed that is the most frequent value of the probability density function of the detected object and the trend of the micro-Doppler speed centered on that speed. Here, the "speed that is the most frequent value of the probability density function of the detected object" may be, for example, the speed with the highest probability in each reference density distribution in FIG. 5. In addition, the "trend of the micro-Doppler speed centered on that speed" may be, for example, a trend showing how the value indicating the probability corresponding to each speed changes centered on the speed with the highest probability in the reference density distribution in FIG. 5. For example, if the magnitude of the speed that is the most frequent value is greater than the general speed of a pedestrian, the electronic device 1 may identify that the detected object is not a pedestrian but another object such as a car or a motorcycle.

In the above-described embodiment, the reference probability density distribution used by the electronic device 1 to identify whether a detected object is a pedestrian or not may be based on pedestrians in various situations. For example, the reference probability density distribution used by the electronic device 1 according to one embodiment may be a reference probability density distribution corresponding to a child walking, or a reference probability density distribution corresponding to an elderly person walking with a cane.

In the above-described embodiment, the electronic device 1 identifies whether or not the detected object is a pedestrian (human). However, the object to be identified is not limited to a pedestrian such as a human, and may be a moving object such as an animal that can perform different movements with its front and back legs or movements in different directions. In this case, the electronic device 1 may store (accumulate) multiple reference probability distributions according to the type of animal. The electronic device 1 may use this reference probability distribution to determine whether or not the object is an animal to be identified.

REFERENCE SIGNS LIST 1 Electronic device 5 Sensor 10 Control unit 11 Distance FFT processing unit 12 Speed FFT processing unit 13 Threshold determination unit 14 Arrival angle estimation unit 15 Object detection unit 16 Tracking processing unit 17 Memory unit 18 Object identification unit 20 Transmitting unit 21 Signal generating unit 22 Synthesizer 23 Phase control unit 24 Amplifier 25 Transmitting antenna 30 Receiving unit 31 Receiving antenna 32 LNA
33 Mixer 34 IF section 35 AD conversion section 50 ECU
100 Moving object 200 Object

Claims (14)

An electronic device comprising a control unit that executes a first process for identifying an object as a person by reflecting the movements of multiple parts of the object in a received signal that is received as a reflected wave from an object, the received signal being transmitted as a transmission wave. The electronic device according to claim 1, wherein the control unit identifies the object as a person by reflecting in the received signal a first speed that is the speed of multiple parts of the object relative to the electronic device. The electronic device according to claim 1 or 2, wherein the received signal reflects the movements of a person's arms and legs. The electronic device according to claim 1 or 2, wherein the received signals reflect movements of both arms of a person in different directions. The electronic device according to claim 1 or 2, wherein the received signals reflect movements of each of the person's legs in different directions. The electronic device according to claim 1, wherein the control unit executes the first process when the object is moving forward or backward relative to the propagation direction of the transmission wave. The electronic device according to claim 6, wherein the control unit determines whether the object is moving forward or backward with respect to the propagation direction of the transmission wave based on first position information of the object detected at a first time and second position information predicted as the position of the object at a second time after the first time. The electronic device according to claim 1, wherein the control unit executes a second process for identifying the object as a person based on first position information of the object detected at a first time and a second velocity of the object calculated based on second position information that predicts the position of the object at a second time that is later than the first time, when the object is moving in a left-right direction relative to the propagation direction of the transmission wave. The electronic device according to claim 2, wherein the control unit executes the first process based on a correlation between a probability density distribution calculated from the first speed and a reference probability density distribution. The electronic device of claim 9, wherein the control unit identifies the object as a person when a value of the Pearson divergence indicating the correlation calculated from the probability density distribution calculated from the first speed and the reference probability density distribution is equal to or less than a threshold value. The electronic device of claim 2, wherein the control unit identifies the object as a person based on a speed that is a most frequent value in a probability density distribution calculated from the first speed and a tendency of the first speed centered on that speed. An electronic device having a control unit that executes a first process to identify an object as a moving body capable of moving parts of its body by reflecting the movements of multiple parts of the object in a received signal that is received as a reflected wave from an object, which is a transmission signal transmitted as a transmission wave. detecting the object based on a transmission signal transmitted as a transmission wave and a reception signal received as a reflected wave of the transmission wave reflected by the object;
performing a first process of identifying the object as a person based on the transmitted signal and the received signal reflecting the motion of a plurality of parts of the object;
A method for controlling an electronic device, comprising: For electronic devices,
detecting the object based on a transmission signal transmitted as a transmission wave and a reception signal received as a reflected wave of the transmission wave reflected by the object;
performing a first process of identifying the object as a person based on the transmitted signal and the received signal reflecting the motion of a plurality of parts of the object;
A program to execute.

Images