JP2002117494A - Peripheral information display device - Google Patents

Abstract

(57) [Summary] [Problem] A peripheral information display device that identifies an object that is expected to affect the traveling of a vehicle and transmits the information to a driver by highlighting the image on an image obtained from an imaging unit. I will provide a. The other vehicle transmits information such as a traveling position and a traveling speed of the vehicle acquired from an own vehicle position estimating device (3) to a periphery from a transmitting antenna (10) via an oscillator (5) to a transmitter (9). The own vehicle receives the information transmitted by the other vehicles in the vicinity by the receiver 12 and combines the information with the information such as the running position and the steering wheel angle of the own vehicle obtained from the own vehicle position evaluation device 3 and the own vehicle mounted sensor 4. Then, a target vehicle that is expected to affect the traveling of the own vehicle is specified. Next, the size, shape, position, and range of the target vehicle on the image are specified, and the actual video image read by the on-vehicle camera 13 is scanned.
The target vehicle is displayed to the driver by a display method corresponding to the presence state in the image.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a peripheral information display device for optimally transmitting to a driver information obtained by communication between a vehicle and an object, such as between vehicles or between vehicles. Based on the information given by, the target that is expected to affect the driving around the own vehicle is specified, and the target is transmitted to the driver by highlighting the target on the image obtained from the imaging unit. And a peripheral information display device.

[0002]

2. Description of the Related Art Conventionally, information provision and warning to a driver about traffic information around the own vehicle obtained by communication between vehicles or between people and vehicles are performed by simple notifying means such as display lamps and buzzers. I do. Such a conventional technique is disclosed in JP-A-7-251691 or JP-A-7-260930. According to the technology described in Japanese Patent Application Laid-Open No. Hei 7-251691, an occupant (driving) when a moving object is detected is detected by a moving object detecting device that receives a signal transmitted from another moving object present in the surroundings and detects the moving object. In-vehicle indicator lamps that display or warn
And display means such as a buzzer, and external display means for performing visual display to the outside when a moving object is detected, and when a signal transmitted by another moving object is received at a predetermined reception intensity or more, It is determined that the moving object is in the vicinity of the own vehicle, and this is transmitted to the occupant or the outside by the prepared in-vehicle display lamp, buzzer, and out-of-vehicle lamp. Also, in the technology described in JP-A-7-260930,
Further, a request signal transmitting means for requesting transmission of a signal to a moving object existing in the surroundings is provided, and only the moving object approaching the own vehicle is requested to transmit the above signal, and other unnecessary signals are transmitted. It eliminates the waste of transmitting signals to mobiles.

[0003]

However, in the above-described apparatus, the display information when a moving object is detected in the vicinity of the own vehicle is only a display such as a simple lamp. It is difficult for the driver to judge which object is an object that affects the driving of the vehicle based on only the information that the vehicle has been detected. was there.

SUMMARY OF THE INVENTION The present invention has been made in view of the above-described problems, and specifies an object which is expected to affect the traveling of the vehicle and drives the object by highlighting the object on an image obtained from an imaging means. It is an object of the present invention to provide a peripheral information display device for transmitting information to a person. More specifically, the movement information for each object given by communication between the own vehicle and the target object such as between vehicles or between people and vehicles, by using an object that is expected to affect the traveling of the own vehicle, Regardless of the driver's judgment,
Identified as an object, by an appropriate display form according to the presence state of the object on the image obtained from the imaging means,
An object of the present invention is to provide a peripheral information display device that transmits peripheral information to a driver and facilitates the driver's recognition of an object.

[0005]

In order to solve the above-mentioned problems, the invention according to claim 1 provides the driver with information on the surroundings of the own vehicle obtained by communication between the own vehicle and the object. A peripheral information display device for displaying a target on an image obtained from an imaging unit based on information obtained through communication (for example, step S1 in the embodiment);
1 to step S14), matching means for determining the presence state of the specified object in the image (for example, steps S21 to S27 in the embodiment), and the presence state of the object in the image. A display means (for example, steps S3 to S5 in the embodiment) for selecting whether the display reference is the image priority display or the prediction target priority display and displaying information around the own vehicle is provided. It is characterized by having. With the above configuration, by specifying an object such as another vehicle or a pedestrian that is expected to affect the traveling of the own vehicle without depending on the driver's will,
It is possible to eliminate the uncertainty of the driver's judgment. In addition, by highlighting the specified object on the real image, it is possible to transmit the information as accurate and easily recognizable information to the driver. Furthermore, by changing the display method in accordance with the specified state of the target object on the image, information on the target object located on the other side of the shield that cannot be seen by the driver can be transmitted. Here, the “prediction target” refers to a target located on the other side of a shield that cannot be visually recognized by a driver, which is specified by the control device 1 of the peripheral information display device according to the embodiment. .

[0006]

Embodiments of the present invention will be described below with reference to the drawings. In the present embodiment, a description will be given of a case where the object is a vehicle and the peripheral information display device is mounted on the vehicle to simplify the description. FIG. 1 is a block diagram illustrating a configuration of the peripheral information display device according to the present embodiment.
In FIG. 1, reference numeral 1 denotes a control device using a microcomputer for controlling the operation of the peripheral information display device according to the present embodiment. Symbol 2 is GPS (Global Positioning S)
ystem) shows the antenna to receive the signal. Symbol 3 is
1 shows an own-vehicle position estimating apparatus that calculates a running position and a running speed of a vehicle based on a GPS signal received by an antenna 2. Reference numeral 4 denotes the acceleration / deceleration of the vehicle, the steering wheel angle, the brake /
1 shows sensors mounted on a vehicle mounted on various parts of a vehicle to detect the opening / closing degree of a throttle and the like. Reference numeral 5 denotes an oscillator that oscillates a signal specified by the control device 1. Code 6
Denotes a frequency mixer for converting the output of the oscillator 5 to a desired transmission frequency. Reference numeral 7 denotes a PLL synthesizer that generates a frequency signal to be supplied to the frequency mixer 6. Reference numeral 8 denotes an amplifier that amplifies the output of the frequency mixer 6 to the power required for transmission. Reference numeral 9 denotes a transmitter that supplies the transmission signal amplified by the amplifier 8 to the antenna. Code 10
Denotes a transmitting antenna for inter-vehicle communication for transmitting information of the own vehicle to another vehicle. Reference numeral 11 denotes a receiving antenna for inter-vehicle communication for receiving information from another vehicle to the own vehicle. Reference numeral 12 denotes a receiver for inputting, from the other vehicle to the own vehicle, information from the other vehicle to the vehicle received by the antenna 11 to the control device 1. Reference numeral 13 denotes a vehicle-mounted camera (imaging means) for acquiring a vehicle peripheral image. Reference numeral 14 denotes an information providing device that displays information that the control device 1 wants to notify the driver.

In the above configuration, in the present embodiment,
The control device 1 of the peripheral information display device mounted on each vehicle,
The information such as the traveling position, traveling speed, azimuth, and course of the vehicle acquired from the vehicle position evaluation device 3 is transmitted from the oscillator 5 to the transmitter 9.
The transmission antenna 10 transmits the signal to another vehicle other than the own vehicle via the transmission system wireless communication device up to this point. Under such circumstances, the peripheral information display device of any vehicle transmits information such as the traveling position, traveling speed, direction, course, and the like of the other vehicle transmitted by the surrounding vehicle to the antenna 11 and the receiver 12. In addition, information such as the traveling position, traveling speed, azimuth, and course of the own vehicle is received from the own vehicle position estimating device 3, and the acceleration / deceleration and steering of the own vehicle are received from the own vehicle mounted sensor 4. Information such as the angle and the degree of opening / closing of the brake / throttle is acquired, whereby the control device 1 extracts another vehicle that is expected to affect the traveling of the vehicle, and specifies this as a target vehicle. After specifying the target vehicle, the control device 1 of the peripheral information display device of the vehicle is specified.
Reads the actual video image from the on-board camera 13, further acquires the position and type of the target vehicle received by the receiving wireless communication device, and the position and traveling direction of the own vehicle obtained from the own vehicle position evaluation device 3, In order to perform the scan, the size and shape of the target vehicle existing on the video and the position and range of the target vehicle existing on the video are specified. Next, the onboard camera 1
3 scans the captured real video image of the target vehicle, and if the shape of the target vehicle can be extracted from the image, displays on the information providing device 14 a highlighted display in which the edge of the target vehicle in the real video is marked. In addition, the image after the edge processing of the actual image of the target vehicle is scanned, and if the shape of the target vehicle cannot be extracted from the image, the outline of the scan shape of the target vehicle is displayed in the actual image around the scan point. It is displayed on the information providing device 14 together with a message notifying the status.

Next, a control procedure of the peripheral information display device of the present embodiment will be described with reference to the drawings. First, the overall control procedure of the peripheral information display device of the present embodiment will be described with reference to FIG. FIG. 2 is a flowchart illustrating a control procedure of the peripheral information display device according to the present embodiment. In FIG. 2, first, a target vehicle extraction process for extracting another vehicle that is expected to affect the traveling of the own vehicle is performed (step S).
1). The details of the target vehicle extraction processing will be described later. Next, when a target vehicle to be noted in the target vehicle extraction process can be extracted, the target vehicle is scanned from the real video image read from the on-vehicle camera 13 to determine whether the target vehicle exists on the real video image. Is performed (step S2). The details of the collation determination processing will be described later. Then, as a result of the collation determination processing, it is determined whether or not the shape of the target vehicle can be extracted from the image (step S3).
In step S3, when it is determined that the shape of the target vehicle can be extracted from the image (YES in step S3), highlighting is performed by marking the edge portion of the target vehicle in the actual video (step S4), and the processed image is displayed as information. The information is displayed on the providing device 14 (step S5). If it is determined in step S3 that the shape of the target vehicle cannot be extracted from the image (NO in step S3), the outline of the scan shape of the target vehicle is displayed on the actual video centering on the scan points in the collation determination process. (Step S6), the processed image is displayed on the information providing device 14 together with the message notifying the traffic condition (Step S5).

Next, referring to the drawings, step S1 in FIG.
The control procedure of the target vehicle extraction process of the peripheral information display device in the above will be described. FIG. 3 is a diagram showing an example of a real scene of a driver's field of view for explaining an operation example of the peripheral information display device of the present embodiment. FIG. 4 is a flowchart illustrating a detailed control procedure of target vehicle extraction processing of the peripheral information display device according to the present embodiment. For example, it is assumed that the vehicle is running in a situation as shown in FIG. 3 and the two-wheeled vehicle is running toward the front from here. In such a situation, the operation of the target vehicle extraction processing when the steering wheel is directed to a course where the host vehicle is going to cross the front of the two-wheeled vehicle will be described. In the control procedure of FIG. 4, first, various information necessary for the target vehicle extraction processing is collected (step S11). Here, the collection of various types of information means information collection (step S11a) for judging a position situation such as a vehicle speed, a position, an azimuth, and a route obtained from the own vehicle position evaluation device 3 (step S11a) and obtained from the own vehicle mounted sensor 4. In addition to the opening / closing degree of the brake / throttle, information collection for judging the driving condition of the vehicle such as the own vehicle speed (Step S11)
b) Information collection (step S11c) for judging the surrounding conditions such as the vehicle speed, position, azimuth, and course of another vehicle obtained from the receiving wireless communication device including the receiver 12. Next, the direction of the own vehicle calculated from the own vehicle position locating device 3
Judgment of the position of the vehicle and the position of the intersection to be approached (step S11a) and the sensor 4 mounted on the vehicle
Of vehicle operation such as brake, throttle opening, own vehicle speed, etc. (step S11b), and surrounding conditions such as the position, traveling direction, approaching intersection, etc. of another vehicle obtained by the receiving wireless communication device (step S11c). ), It is determined whether or not there is another vehicle whose traveling vector intersects with the traveling direction of the own vehicle (step S12). In the example shown in FIG. 3, the own vehicle tries to cross the front of the two-wheeled vehicle, and thus the two-wheeled vehicle is recognized as another vehicle whose traveling vector intersects the traveling direction of the own vehicle.
In step S12, when it is determined that there is another vehicle whose traveling vector intersects with the traveling direction of the own vehicle (YES in step S12), there is another vehicle whose estimated arrival time difference at the intersection is less than a certain time. It is determined whether or not (step S13). Step S13
In the case where it is determined that there is another vehicle in which the estimated arrival time difference at the intersection is less than a certain time (YES in step S13), the vehicle is extracted as a target vehicle,
The target vehicle is specified by the vehicle ID included in the received signal (Step S14). Here, it is assumed that the two-wheeled vehicle is specified as the target vehicle in the example shown in FIG.

On the other hand, in step S12, when it is determined that there is no other vehicle whose traveling vector intersects the traveling direction of the own vehicle (NO in step S12), or in step S13, there is an estimated arrival time difference at the intersection. When it is determined that there is no other vehicle that is shorter than the predetermined time (NO in step S13), the process returns to step S11 and the acquisition of various information is continued.

In the above-described extraction of the target vehicle, the intersection of the traveling vectors of the vehicles and the estimated arrival time difference at the intersection are obtained as follows. That is, the vector of the target vehicle and the destination is obtained from the information on the position of the destination such as an intersection that is about to approach the position of the target vehicle received from the target vehicle.
The intersection of the vector is obtained by superimposing the position of the own vehicle calculated from the vector and the vector obtained from the destination such as the intersection to be approached, the position of the target vehicle, and the distance from the position of the own vehicle to the vector intersection And calculate
Further, using the information on the vehicle speed / acceleration etc. of the target vehicle received from the target vehicle and the information on the vehicle speed / acceleration etc. obtained from the own-vehicle mounted sensor 4, calculating the mutual arrival time up to the intersection of the vectors. As a result, an estimated arrival time difference can be obtained.

Next, referring to the drawing, step S2 in FIG.
The control procedure of the collation determination process of the peripheral information display device in the above will be described. FIG. 5 is a flowchart illustrating a detailed control procedure of the collation determination processing of the peripheral information display device according to the present embodiment. In FIG. 5, first, an actual video image is read from the on-vehicle camera 13 (step S21). Next, edge processing for enhancing and extracting boundaries between images on the video is performed (step S2). 6A and 6B are diagrams illustrating an example of an image handled in the collation determination processing of the peripheral information display device according to the present embodiment. FIG. 6A illustrates an actual image of the vehicle-mounted camera 13 and FIG. Indicates images after edge processing of the actual video. After acquiring the image after the edge processing, various information necessary for the collation determination processing is collected next (step S2).
3). Here, the collection of various types of information refers to the collection of information such as the vehicle speed, position, azimuth, course, and vehicle type of the target vehicle acquired from the receiving system wireless communication device including the receiver 12 (step S23).
a), information collection of the vehicle speed, position, azimuth, course, etc. acquired from the own vehicle position evaluation device 3 (step S23b). After collecting the various information, the relative distance is calculated from the position of the target vehicle and the position of the own vehicle, and the size to be displayed on the video is specified. A specific method for obtaining the size will be described with reference to FIG. FIG. 7 is a diagram illustrating a relationship between the position of the own vehicle and the traveling method of the peripheral information display device according to the present embodiment, and the position of the target vehicle. Here, the object width W of the target vehicle
Is from the vehicle type information obtained in the information collection in step S23a, and the position (Xv, Yv) is
It is determined from the position of the target vehicle obtained in the information collection in 23a. Therefore, the object width W ′ on the image is (Xo, Yo) the position of the own vehicle obtained in the information collection in step S23b, and L is the focal length of the on-vehicle camera 13.
Then, as shown in FIG. 7, W ′ = W × L / | (Xv−Xo) | can be expressed by the ratio calculation.

Next, the scan shape is specified based on the determination of the positional relationship between the vehicle type information obtained in the information collection in step S23a and the position of the target vehicle and the position of the own vehicle obtained in the information collection in step S23b ( Step S25). FIG. 8 is a diagram illustrating an example of a database that records the scan shape of the target vehicle specified in the matching determination process of the peripheral information display device according to the present embodiment. According to FIG. 8, the database records the corresponding scan shape using a vehicle type information and a matrix based on the positional relationship between the host vehicle and the target vehicle. For example, as a vehicle type, a truck, a bus, a passenger car, a two-wheeled vehicle, and the like are recorded, and in the positional relationship, a scan shape is distinguished between an oncoming vehicle and a preceding vehicle.
In step S25, a corresponding scan shape is extracted from the database shown in FIG. After the scan shape of the target vehicle is determined, the scan position of the target vehicle is specified (step S26). Similarly, the scan position of the target vehicle can be specified by calculating the relative position from the position of the target vehicle obtained in the information collection in step S23a and the position and orientation of the own vehicle obtained in the information collection in step S23b. One point on the image is determined in consideration of the characteristics when the vehicle is displayed on the image.

If a point on the image when the target vehicle is displayed on the image can be determined, then a scan range is determined from the progress of the own vehicle and the target vehicle (step S).
27). In the example shown in FIG. 3, since the own vehicle is going to cross the front of the two-wheeled vehicle, the relative relationship with the target vehicle is as shown in FIG. 9 from the change in the traveling direction of the own vehicle. FIG. 9 is a diagram showing the relationship between the traveling direction of the vehicle and the image rotation axis at the time of the collation determination processing of the peripheral information display device of the present embodiment. In FIG. 9, reference numeral 20 denotes the own vehicle. Reference numeral 21 denotes the two-wheeled vehicle in the example of FIG. Reference numeral 22 indicates an intersection where both the own vehicle and the two-wheeled vehicle are about to approach. Here, the object on the image located at the focal length L from the position of the own vehicle is 2
In the state where both wheels are traveling straight, the vehicle is located on the right front of the vehicle as shown in FIG. 9 (a), but in the state where the vehicle is going to cross the front of the motorcycle, FIG. 9 (b) It appears to be located to the left front of the vehicle as shown in FIG. Therefore, FIG.
As can be seen from (b), as the two-wheeled vehicle advances, the object on the image located at the focal length L from the position of the vehicle is:
It is expected to move from left to right and from the back of the image to the front. Therefore, the scan position and scan range of the target vehicle are determined as shown in FIG. FIG. 10 is a diagram showing a scan position specified by the collation determination processing of the peripheral information display device of the present embodiment and a scan range obtained from characteristics on an image. For the scan position obtained as shown in FIG.
The scan range indicated by 0 (b) is obtained. As mentioned above,
When the scan conditions for the target vehicle are determined, the actual video image of the target vehicle captured by the on-vehicle camera 13 is scanned (step S28), and the collation determination process ends.

Next, a display example of the peripheral information in the peripheral information display device of the present embodiment will be specifically described with reference to the drawings. FIG. 11 shows the display of the information providing device in the peripheral information display device of the present embodiment. It is a figure showing an example of a screen. FIG.
FIG. 3 is a diagram showing an example of the arrangement of an information providing device in a vehicle in the peripheral information display device of the embodiment. In FIG. 12, reference numeral 50 denotes a NAVI Display arranged near the driver's seat that does not obstruct the driver, and displays information of a navigation system that transmits the position of the vehicle to the driver and guides the driver to the destination. Reference numeral 51 denotes a HUD (Head Up
Display) to display information on the windshield in front of the driver. Reference numeral 52 denotes a meter-integrated Displa
y is displayed, and information including contents necessary for traveling such as speed is displayed.

For example, in the traffic situation shown in FIGS. 3 and 9, when the traveling vehicle 20 tries to cross the front of the opposing two-wheeled vehicle 21 as described above, the surrounding information display device performs the target vehicle extraction processing. Accordingly, it is assumed that the motorcycle 21 is specified as a target vehicle that is expected to affect the traveling of the own vehicle. Next, the target vehicle is scanned from the real video image read from the on-vehicle camera 13 and a collation determination process is performed to check whether the target vehicle exists on the real video image. As a result, the shape of the target vehicle is changed. When it is determined that the image can be extracted from the image, as shown in FIG. 11A, highlighting is performed by marking the edge portion of the target vehicle of the actual video, and the processed image is displayed on the information providing device 14. Also, for example, FIG.
As shown in (b), when there is a preceding vehicle ahead of the own vehicle, the two-wheeled vehicle of the target vehicle is located on the other side of the shield, and the shape of the target vehicle is displayed in the collation determination processing as an image. Is determined to be impossible to extract. As described above, for a target vehicle that cannot be seen by the driver, FIG.
As shown in (b), the outline of the scan shape of the target vehicle is displayed on the actual video centering on the scan point in the collation determination process, and the processed image is displayed on the information providing device 14 together with a message notifying the traffic situation.

In the above-described embodiment, the vehicle has been described as an object that is expected to affect the traveling of the own vehicle. If it is possible to transmit and receive the same information as the information to determine and obtain the position status such as the vehicle speed, position, direction, course, the position of the intersection to be approached, etc. It may be an object other than a vehicle such as a pedestrian carried. When an object other than a vehicle is handled, the scan shape of an object other than the vehicle (for example, a pedestrian) may be further recorded in the contents of the database shown in FIG. Further, if the pedestrian or the like carries the peripheral information display device with the pedestrian or the like, the above-mentioned on-vehicle sensor 4 is not required for the peripheral information display device carried by the pedestrian or the like.

[0018]

As described above, according to the first aspect of the present invention, the object of the driver such as another vehicle or a pedestrian which is expected to affect the traveling of the own vehicle is determined by the driver. By doing so, it is possible to eliminate the uncertainty of the driver's judgment. In addition, by highlighting the specified object on the real image, it is possible to transmit the information to the driver as accurate and easily recognizable information. By changing the display method according to the state, it is also possible to transmit information on an object located on the other side of the shield that cannot be seen by the driver.

Therefore, the display information when a moving object is detected in the vicinity of the own vehicle is not only notified by a simple display lamp that the moving object is detected in the vicinity of the own vehicle,
The effect of being able to notify the driver of the surrounding information of the own vehicle by accurate information of the target object shown on the image that facilitates the recognition of the target object is obtained. In particular, even with respect to an object located behind the obstruction that cannot be seen by the driver, the position and shape of the object can be accurately notified to the driver, thereby facilitating the recognition by the driver. Can be

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration of a peripheral information display device according to an embodiment of the present invention.

FIG. 2 is a flowchart showing a control procedure of the peripheral information display device of the embodiment.

FIG. 3 is a diagram showing an example of a driver's visual field real scene for explaining an operation example of the peripheral information display device of the embodiment.

FIG. 4 is a flowchart showing a detailed control procedure of a target vehicle extraction process of the peripheral information display device of the embodiment.

FIG. 5 is a flowchart showing a detailed control procedure of a collation determination process of the peripheral information display device of the embodiment.

FIG. 6 is a diagram showing an example of a video handled in a collation determination process of the peripheral information display device of the embodiment.

FIG. 7 is a diagram showing the relationship between the position of the own vehicle and the traveling method of the peripheral information display device of the embodiment, and the position of the target object.

FIG. 8 is a diagram illustrating an example of a database that records a scan shape of an object specified in the collation determination process of the peripheral information display device according to the embodiment.

FIG. 9 is a diagram illustrating a relationship between a traveling direction of the own vehicle and an image rotation axis during a collation determination process of the peripheral information display device of the embodiment.

FIG. 10 is a diagram illustrating a scan position specified by a collation determination process of the peripheral information display device according to the embodiment and a scan range obtained from characteristics on an image.

FIG. 11 is a diagram showing an example of an object highlight display screen in the peripheral information display device of the embodiment.

FIG. 12 is a diagram showing an example of an in-vehicle arrangement of an information providing device in the peripheral information display device of the embodiment.

[Explanation of symbols]

 S3 to S5 display means S11 to S14 identification means S21 to S27 collation means

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat ゛ (Reference) G08G 1/04 G08G 1/04 C 1/09 1/09 H G09G 5/00 510 G09G 5/00 510Z 550 550C F-term (reference) 5C082 AA27 BA12 BA27 BA41 CA54 CA82 MM10 5H180 AA01 BB04 BB15 CC04 FF05 FF13 FF32 FF35 LL01 LL04 LL08

Claims (1)

[Claims] 1. A peripheral information display device for displaying information about the surroundings of a vehicle obtained by communication between the vehicle and an object to a driver, based on the information obtained by the communication. Specifying means for specifying an object on the image obtained from the imaging means; matching means for determining the presence state of the specified object in the image; and Display means for selecting whether the display standard for the user is image priority display or prediction target priority display, and displaying information about the surroundings of the own vehicle; and apparatus.