JP2016119558A - Video processing device and on-vehicle video processing system - Google Patents

Abstract

Provided is a video processing apparatus capable of changing the viewpoint position of a video when displaying an image of a rear side of a vehicle, and appropriately displaying an obstacle when an obstacle exists behind the vehicle. . When there is an obstacle behind the vehicle (S40: YES), the video processing apparatus displays a first output video on the monitor (S60), and when there is no obstacle (S40). : NO), the second output video is displayed on the monitor (S60). When generating at least one of the first output video and the second output video, video conversion processing is performed to change at least one of the viewpoint position, the line-of-sight direction, and the visual field range (S50). . The video conversion process includes a video conversion process in which an obstacle is displayed larger in the first output video than the second output video, or an obstacle that is not displayed in the second output video. Part or all of the video conversion processing to be displayed on the first output video is executed. [Selection] Figure 2

Description

  The present invention relates to a video processing device and an in-vehicle video processing system.

  2. Description of the Related Art A vehicle periphery monitoring device that captures a rear side of a vehicle with a camera and displays the image on a display unit inside the vehicle is known (for example, see Patent Document 1).

Japanese Patent No. 3877761

  By the way, when the vehicle moves backward, the viewpoint position, the line-of-sight direction, the visual field range, etc. of the video displayed on the display unit of the apparatus as described above (hereinafter collectively referred to as the viewpoint position, etc.) There is a case where it is desired to change to a viewpoint position or the like different from that when the vehicle moves forward.

  For example, if you want to put the vehicle in the parking frame while moving the vehicle backward, or if you want to stop the vehicle while moving the vehicle backward, close the parking frame or the road shoulder to the vicinity of the rear wheel of the vehicle. You may want to display a video at a viewpoint position that makes it easy to check As another example, for example, when a vehicle is moving forward, a display that can confirm a wider range is displayed, but when the vehicle is moved backward, a narrower range behind the vehicle is enlarged and displayed. Sometimes you want to display a video. Such a change in the viewpoint position or the like can be realized by appropriately setting the cutout range of the camera video and performing appropriate distortion correction or the like if necessary.

  However, if there is an obstacle behind the vehicle and the viewpoint position or the like is changed as described above, the location where the obstacle exists may be excluded from the display target. In addition, if correction is performed to remove distortion by focusing on a different location from the obstacle, the distortion at the location where the obstacle is located may increase. In this case, the distorted obstacle is displayed. However, there is a possibility that an obstacle cannot be properly visually recognized.

  Due to the above circumstances, when displaying the image of the rear side of the vehicle, the viewpoint position of the image can be changed, and if there is an obstacle behind the vehicle, It is desirable to provide a displayable video processing device and an in-vehicle video processing system.

  The video processing apparatus described below includes an obstacle determination unit, a video input unit, a video processing unit, and a video output unit. The obstacle determination unit obtains information on the obstacle from the obstacle detection unit capable of detecting the obstacle existing behind the vehicle, and determines the presence or absence of the obstacle. The video input unit inputs a camera video captured by the camera from a camera configured to capture the rear side of the vehicle.

  The video processing unit is configured to generate a first output video and a second output video based on the camera video input by the video input unit. When the obstacle determination unit determines that there is an obstacle, the video processing unit generates a first output video. When the obstacle determination unit determines that there is no obstacle, the video processing unit generates a second output video.

  When generating at least one of the first output video and the second output video, the video processing unit uses at least one of the viewpoint position, the line-of-sight direction, and the field-of-view range as a change target. A video conversion process for changing the change target is performed on the video. As a result, the first output video and the second output video that are different from each other (that is, at least one of the viewpoint position, the line-of-sight direction, and the visual field range) are generated.

  In such a video processing unit, as the video conversion process, a video conversion process in which an obstacle is displayed larger in the first output video than the second output video, or the second output video A video conversion process is executed in which part or all of the obstacles that are not displayed are displayed on the first output video.

The video output unit outputs the first output video or the second output video generated by the video processing unit to a display device disposed in the vehicle interior.
According to the video processing apparatus configured as described above, when there is an obstacle behind the vehicle, the first output video can be displayed on the screen of the display device. In the first output video, an obstacle is displayed larger than the second output video, or a part or all of the obstacle not displayed in the second output video is displayed. Therefore, the driver of the vehicle can easily recognize an obstacle behind the vehicle, and can perform an appropriate driving operation in consideration of the obstacle.

  Further, when there is no obstacle behind the vehicle, the second output video can be displayed on the screen of the display device. The second output video has a wider range than the first output video (that is, if there is an obstacle, the obstacle will be displayed smaller than the first output video). Or a range different from the first output video (that is, a range in which part or all of the obstacles to be displayed are not displayed in the first output video). ) Is displayed. Therefore, if there are no obstacles behind the vehicle, the periphery of the vehicle is displayed over a wider range, or a direction in which a part of the rear of the vehicle is not visible (for example, a direction in which the vicinity of the rear wheel of the vehicle is watched). It is possible to display. Therefore, it is possible to display an image that makes it easy to recognize an object other than the rear of the vehicle.

  The video conversion process as described above may be executed when generating at least one of the first output video and the second output video. That is, the video conversion process may be executed when generating one of the first output video and the second output video, or the video conversion process may be executed when generating each of both. Also good.

  When the video conversion process is executed when generating one and the video conversion process is not executed when generating the other, one of the change objects is an image that does not match the camera image, and the other is the change object is a camera. The video matches the video. Therefore, in this case, the first output video and the second output video, which are different from each other, are generated.

  In addition, when the video conversion process is executed when generating each of the both, the change target is a video that does not match the camera video. However, also in this case, the first output video and the second output video with different change targets are generated by making the change target different from each other.

  The vehicle-mounted video processing system described below includes an obstacle detection unit, a camera, a video processing device, and a display device. The video processing apparatus includes the configuration as described above (that is, the above-described obstacle determination unit, video input unit, video processing unit, and video output unit). Therefore, the operation and effect as described for the above-described video processing apparatus are achieved.

FIG. 1 is a block diagram showing a configuration of an in-vehicle video processing system. FIG. 2 is a flowchart showing video processing in the first embodiment. FIG. 3 is an explanatory diagram showing the positional relationship of the shooting range by the vehicle, the obstacle, and the camera exemplified in the first embodiment. FIG. 4 is an explanatory diagram illustrating the camera video and the first output video in the first embodiment. FIG. 5 is an explanatory diagram illustrating a second output video in the first embodiment. FIG. 6 is a flowchart showing video processing in the second embodiment. FIG. 7 is an explanatory diagram illustrating the first output video in the second embodiment. FIG. 8 is a flowchart showing video processing in the third embodiment. FIG. 9 is an explanatory diagram illustrating the first output video in the third embodiment. FIG. 10 is an explanatory view illustrating a mirror image reflected on the door mirror. FIG. 11 is an explanatory diagram illustrating the first output video in the fourth embodiment. FIG. 12 is a flowchart showing video processing in the fifth embodiment. FIG. 13 is an explanatory diagram illustrating the first output video in the fifth embodiment.

Next, the above-described video processing device and in-vehicle video processing system will be described with reference to exemplary embodiments.
(1) First embodiment [Configuration of video processing apparatus]
As shown in FIG. 1, an in-vehicle video processing system described below is mainly configured by an electronic control unit 1 (Electronic Control Unit; hereinafter abbreviated as ECU 1) that functions as the above-described video processing device. In addition to the ECU 1, a left camera 3, a right camera 4, a left monitor 5, a right monitor 6, an obstacle detection unit 7, a shift position detection unit 8, and the like are provided.

  The left camera 3 is attached to a position corresponding to the attachment position of the door mirror on the left side of the vehicle, and is configured to be able to photograph the rear side of the left side of the vehicle. The right camera 4 is attached to a position corresponding to the attachment position of the door mirror on the right side of the vehicle, and is configured to be able to photograph the rear side of the right side of the vehicle. Both the left camera 3 and the right camera 4 are configured to be able to photograph a range wider than a range that can be seen through a general door mirror.

  The left monitor 5 and the right monitor 6 are constituted by, for example, a liquid crystal display device or the like, and each is disposed in the vehicle interior. In the case of the present embodiment, the left monitor 5 is installed at a position that is diagonally forward left when viewed from the driver's seat, and images that are mainly captured by the left camera 3 are displayed on the left monitor 5. The right monitor 6 is installed at a position that is diagonally forward to the right when viewed from the driver's seat, and an image captured mainly by the right camera 4 is displayed on the right monitor 6.

  However, the disposition positions of the left monitor 5 and the right monitor 6 are not limited to the disposition positions described above, and the left monitor 5 and the right monitor 6 are configured by the display devices corresponding to each as described above. However, it may be composed of a single display device. For example, a single center display is disposed in the vicinity of the center of the dashboard (for example, a location facing the position between the driver seat and the passenger seat, such as a center console), and the left monitor 5 and the right monitor 6. May be provided on the screen of the center display.

The obstacle detection unit 7 is configured to be able to detect the presence of an obstacle behind the vehicle and to measure the distance to the detected obstacle. In the case of this embodiment, the obstacle detection unit 7 is configured by an ultrasonic distance measuring sensor (hereinafter also referred to as an ultrasonic sonar), and a plurality of ultrasonic sonars are disposed at the rear of the vehicle. By transmitting ultrasonic waves from these multiple ultrasonic sonars to the rear of the vehicle and receiving reflected waves reflected by the obstacles, the presence of the obstacles is detected, and the distance to the detected obstacles Can be measured. Obstacle information obtained by the obstacle detection unit 7 is transmitted to the ECU 1 described above. The obstacle detection unit 7 may be configured by a distance measuring sensor other than the ultrasonic method (for example, an infrared distance measuring sensor).

  The shift position detector 8 is configured by a sensor that detects a shift position of a transmission (not shown). Shift position information obtained by the shift position detector 8 is transmitted to the ECU 1 described above.

  The ECU 1 includes a video input unit 11, a video processing unit 12, a video output unit 13, a storage unit 14, an obstacle determination unit 15, a shift position determination unit 16, and the like. The video input unit 11 inputs camera video shot by each of the left camera 3 and the right camera 4. The video processing unit 12 performs video processing on each of the camera video input by the video input unit 11 (camera video shot by the left camera 3 and camera video shot by the right camera 4) for output. Generate video. Details of the video processing executed in the video processing unit 12 will be described later.

  The video output unit 13 outputs, to the left monitor 5, the output video generated based on the camera video captured by the left camera 3 among the output videos generated by the video processing unit 12, and the right camera 4. The video for output generated based on the camera video shot in step 1 is output to the right monitor 6. As a result, the driver can grasp the situation on the rear side of the vehicle by looking at the images displayed on the left monitor 5 and the right monitor 6 when the vehicle moves backward or when changing lanes. .

  The storage unit 14 stores various types of data required when the video processing unit 12 executes video processing. The obstacle determination unit 15 determines the presence or absence of an obstacle behind the vehicle based on the obstacle information obtained from the obstacle detection unit 7. The shift position determination unit 16 determines the shift position of the transmission based on the shift position information obtained from the shift position detection unit 8.

  In the case of the present embodiment, the door mirror itself (that is, a configuration in which the rear side of the vehicle can be confirmed by a mirror image reflected in the mirror) is not attached to the vehicle. Instead, the left camera 3 and the right camera 4 described above are attached at a position corresponding to the attachment position of the door mirror. The driver can check the situation on the rear side of the vehicle by looking at the left monitor 5 and the right monitor 6 instead of looking at the door mirror.

  However, the in-vehicle image processing system described in the present embodiment may be adopted as a configuration used in combination with a door mirror in a vehicle to which a door mirror is attached. For example, door mirrors may be attached to the left and right sides of the vehicle, and the left camera 3 and the right camera 4 may be attached to the door mirrors. In this case, the left monitor 5 and the right monitor 6 can be used in combination with a door mirror to check the rear side of the vehicle.

[Details of processing executed in ECU]
Next, processing executed in the ECU 1 will be described based on the flowchart shown in FIG. The process described below is executed as the vehicle starts to operate, and thereafter is repeatedly executed until the vehicle stops operating.

  When this process is started, the ECU 1 acquires camera images captured by the left camera 3 and the right camera 4 (S10). In the case of the present embodiment, as the left camera 3 and the right camera 4, cameras that output a video image as a mirror image by inverting the left and right of the subject to be photographed are employed. Therefore, in S10, it is possible to acquire a mirrored camera image without performing any additional image processing. However, as the left camera 3 and the right camera 4, a camera that outputs a video as a normal image without reversing the left and right of the subject to be photographed can be adopted. In this case, the ECU 1 may perform video processing that inverts the left and right of the video acquired as a normal image and converts it into a mirror image in S10 or a subsequent processing step.

  The camera image obtained in this way is, for example, an image taken by the right camera 4 and is an image as illustrated in FIG. 4 under the situation illustrated in FIG. In FIG. 3, the vehicle 20 is stopped inside the parking frame 22 (white line drawn on the pavement surface) in a state of being brought to the white line on the right side of the vehicle 20. Further, a pylon (road cone) 24 is placed near the vehicle 20 behind the vehicle 20, and another vehicle 26 is stopped farther than the pylon 24.

  For example, the left camera 3 and the right camera 4 can photograph the photographing ranges A1 and A2 as illustrated by the two-dot chain line in FIG. 3 with respect to the photographing direction at an angle looking down near the center of the pylon 24. Yes. However, the body of the vehicle 20 is reflected in the shooting ranges B1 and B2. Therefore, for example, a part of the pylon 24 in the photographing range B2 is hidden behind the vehicle 20 reflected in the camera image as shown in FIG. Similarly, another vehicle 26 is also hidden in a position behind the vehicle 20 that is partially reflected in the camera image. Although the left camera 3 is different from the right camera 4 in that the object to be photographed is the rear side of the left side of the vehicle body, it is not an important matter for understanding the processing shown in FIG. Illustration of the video to be performed is omitted.

  Now, if the above camera images are acquired in S10, ECU1 will acquire shift position information from the shift position detection part 8, and will determine whether a shift position is reverse (reverse | reverse) (S20). In S20, when the shift position is reverse (S20: YES), the process proceeds to S30. In S20, when the shift position is not reverse (S20: YES), the process proceeds to S60 without executing S30 to S50.

  When it progresses to S30, ECU1 acquires obstruction information from the obstruction detection part 7 (S30). Then, the ECU 1 determines whether there is an obstacle behind the vehicle (S40). In S40, when there is no obstacle behind the vehicle (S40: NO), the process proceeds to S50. In S40, when there is an obstacle behind the vehicle (S40: NO), the process proceeds to S60 without executing S50.

  When the process proceeds to S50, the ECU 1 performs viewpoint conversion for the backward movement of the vehicle on the camera images taken by the left camera 3 and the right camera 4 (S50). In the case of the present embodiment, in S50, by performing viewpoint conversion on the camera image having the viewpoint position as illustrated in FIG. 4 and the like, an output image as illustrated in FIG. Corresponding to the second output video). However, although the pylon 24 and another vehicle 26 are reflected in the camera image illustrated in FIG. 4, since S50 is executed when there is no obstacle behind the vehicle, for example, the pylon 24 from FIG. Or an image (not shown) such that another vehicle 26 is lost is the image before the viewpoint conversion in S50.

The output video (see FIG. 5) generated in S50 has a viewpoint position and the like so that the positional relationship between the parking frame 22 and the rear wheel 20A of the vehicle 20 can be confirmed more easily than the camera video illustrated in FIG. It is the converted video. When such viewpoint conversion is performed, in the output video illustrated in FIG. 5, a location where the pylon 24 or another vehicle 26 exists is a location that does not fit in the video. However, since S50 is executed when there is no obstacle behind the vehicle as described above, even if the location where the pylon 24 or another vehicle 26 exists is not within the image, there is an obstacle at such a location. There should be nothing, and there is no problem with that.

  When S50 as described above is completed, the process proceeds to S60. Further, as described above, the process may proceed from S20 or S40 to S60. When the process proceeds from S20 or S40 to S60, S50 is not executed. In this case, the camera video itself illustrated in FIG. 4 corresponds to the output video (this corresponds to the first output video in this specification). ). Then, in S60, the ECU 1 displays one of the output videos having different viewpoint positions and the like as exemplified in FIGS. 4 and 5 on the left monitor 5 and the right monitor 6 as a monitor video. Display (S60).

  When S60 is executed, for example, the right monitor 6 displays an output image as illustrated in FIG. 4 if there is any obstacle behind the vehicle, and as illustrated in FIG. 5 if there is no obstacle behind the vehicle. The output video is displayed. A similar output video is also displayed on the left monitor 5 (not shown). The above processing is repeatedly executed at a processing speed of, for example, about 30 to 120 times per second. As a result, the left monitor 5 and the right monitor 6 are animated on the rear side of the vehicle with a frame rate of about 30 to 120 fps. An image is displayed.

[effect]
As described above, according to the in-vehicle video processing system, when the vehicle is moving forward or stopped, a negative determination is made in S20, and the right monitor 6 is as illustrated in FIG. An image is displayed at the viewpoint position or the like. Therefore, for example, while the vehicle is traveling, it is possible to visually recognize the adjacent lane and the vehicle traveling in the adjacent lane over a wide range.

  Further, when the vehicle moves backward, if there is no obstacle behind the vehicle, an affirmative determination is made in S20 and a negative determination is made in S40, and an image is displayed on the right monitor 6 at a viewpoint position as illustrated in FIG. The Therefore, on the right monitor 6, the image near the rear wheel 20A of the vehicle 20 is displayed larger than when the output image illustrated in FIG. 4 is displayed, and the parking frame 22 and the vehicle The positional relationship of the 20 rear wheels 20A can be easily confirmed. Therefore, for example, even when it is desired to stop the vehicle by closely approaching the white line of the parking frame 22, it is possible to try fine adjustment of the stop position more strictly.

  On the other hand, when the vehicle moves backward, if there is an obstacle behind the vehicle, an affirmative determination is made in S20 and an affirmative determination is made in S40, and an image is displayed on the right monitor 6 at the viewpoint position shown in FIG. The Therefore, in this case, although the image near the rear wheel 20A of the vehicle 20 is displayed smaller than the case where the output image as illustrated in FIG. 5 is displayed, the pylon 24 or another vehicle 26 is displayed. Becomes the output video. Therefore, the presence of obstacles such as the pylon 24 and another vehicle 26 can be appropriately visually recognized on the right monitor 6, and the vehicle can be prevented from moving backward without noticing those obstacles.

(2) Second Embodiment Next, a second embodiment will be described. Since the second and subsequent embodiments have many common parts with the first embodiment, the differences from the first embodiment will be mainly described. About the common part with 1st embodiment, the same code | symbol as 1st embodiment is attached | subjected and the detailed description is abbreviate | omitted.

In each embodiment after the second embodiment, the configuration of hardware or the like is the same as that of the first embodiment, and a part of the processing executed in the ECU 1 is different from that of the first embodiment. Hereinafter, the process performed in ECU1 is demonstrated based on the flowchart shown in FIG. In addition, each process demonstrated in each embodiment after 2nd embodiment is also performed with the start of operation | movement of a vehicle, and the point which is a process repeatedly performed until operation | movement stop of a vehicle after that is 1st embodiment. It is the same.

  The process of the second embodiment differs from the first embodiment in that S52 is executed when there is an obstacle behind the vehicle in S40 (S40: NO). That is, when the process proceeds to S52, the ECU 1 performs viewpoint conversion for the presence of an obstacle when the vehicle moves backward on the camera images taken by the left camera 3 and the right camera 4 (S52). If there is no obstacle behind the vehicle (S40: YES), the ECU 1 performs viewpoint conversion for no obstacle when the vehicle moves backward on the camera images taken by the left camera 3 and the right camera 4 ( S50), which is the same processing step as S50 described in the first embodiment.

  In the case of the present embodiment, in S52, an output video (referred to in this specification) as illustrated in FIG. 7 is obtained by performing viewpoint conversion on the camera video having the viewpoint position as illustrated in FIG. Corresponding to the first output video). The video for output (see FIG. 7) generated in S52 is such that the position of the viewpoint is displayed so that the location where the pylon 24 or another vehicle 26 is present is displayed larger than the camera video illustrated in FIG. Is converted video. Thereby, the obstacle (pylon 24 or another vehicle 26) in the rear of the vehicle can be easily recognized more than the camera image illustrated in FIG.

  Incidentally, the output video illustrated in FIG. 7 has a narrower angle of view than that of the camera video illustrated in FIG. In addition, the visual field range of the output video illustrated in FIG. 7 is set to the same level as a general door mirror. Therefore, for a driver who is accustomed to the backward confirmation using a general door mirror, the image is less uncomfortable than when the camera image illustrated in FIG. 4 is displayed as it is.

  Even in the in-vehicle video processing system configured as described above, when the vehicle is moving forward, when the vehicle is stopped, or when there is no obstacle when the vehicle moves backward, the same output as in the first embodiment The video for use is displayed on the left monitor 5 and the right monitor 6. Therefore, in these cases, the same operation and effect as the first embodiment are achieved.

  Further, when there is an obstacle behind the vehicle when the vehicle moves backward, an affirmative determination is made in S20 and an affirmative determination is made in S40, and the right monitor 6 displays an image at the viewpoint position as illustrated in FIG. Is done. Therefore, in this case, the pylon 24 or another vehicle 26 is displayed in a larger image as compared with the apparatus of the first embodiment in which the camera image illustrated in FIG. 4 is displayed as it is. Therefore, even if the obstacle is reflected in the video for output, even if it is a smaller obstacle, its presence can be easily recognized, and the vehicle is moved backward without being aware of such an obstacle. Can be suppressed.

(3) Third Embodiment Next, a third embodiment will be described. In the case of the third embodiment, the processing executed in the ECU 1 is different from the second embodiment in that S56 is executed after S52 is executed. When the process proceeds to S56, the ECU 1 synthesizes the vehicle body video with the output video generated in S52 (S56). Except for S56, the processing steps described with the same reference numerals in the first and second embodiments are exactly the same.

The host vehicle body image synthesized in S56 is an image for masking a part of the vehicle body reflected in the output image generated in S52. For example, in S52, when a video as illustrated in FIG. 7 is generated, when S56 is further executed, as shown in FIG. 9, the vehicle body video 28 is compared with the original video (see FIG. 7). Will be synthesized.

  As the own vehicle body image 28, an image of an actual vehicle shot at an angle that does not cause the driver to feel uncomfortable is stored in the storage unit 14 in advance, and the image is read from the storage unit 14 and used. That's fine. Alternatively, data (for example, polygon data) necessary for generating an image equivalent to such an image is stored in the storage unit 14 in advance, and an image corresponding to an actual vehicle is based on such data. May be generated and used.

  If a video as shown in FIG. 9 is generated, a video with less sense of incongruity than the second embodiment can be displayed on the right monitor 6 for a driver who is accustomed to the backward confirmation using a general door mirror. More specifically, under the situation illustrated in FIG. 3, when a backward confirmation is performed using a general door mirror, for example, a scene as shown in FIG. 10 is reflected on the right door mirror. When the camera image shown in FIG. 7 is compared with the mirror image of the door mirror shown in FIG. 10, it is reflected in each due to the difference between the mounting position of the right camera 4 and the position of the eyes of the driver viewing the door mirror. It can be seen that there is a difference in the area of the vehicle 20.

  For the driver who is accustomed to the rear view confirmation using the door mirror, the vehicle 20 and the parking frame 22 can be seen in the positional relationship as shown in FIG. You may know that. In such a case, when an image as shown in FIG. 7 is displayed on the right monitor 6, it seems that there is still a distance between the vehicle 20 and the white line on the right side thereof, which contributes to a sense of incongruity. Can be.

  On the other hand, if an image obtained by synthesizing the vehicle body image 28 as shown in FIG. 9 is displayed on the right monitor 6, the image of the vehicle 20 that causes a sense of discomfort is masked by the vehicle body image 28. Is done. As a result, an image having a composition similar to the mirror image of the door mirror shown in FIG. 10 can be displayed on the right monitor 6, so that a driver who is accustomed to confirming the rear by the door mirror can display an image with a little uncomfortable feeling. it can.

  Incidentally, as is apparent from the mirror image of the door mirror shown in FIG. 10, when the rear view is confirmed with the door mirror, the pylon 24 is in a position that is hardly visible with the door mirror. For a driver who is used to confirming rearward with a general door mirror, such a view is more appropriate and less uncomfortable. Therefore, when emphasis is placed on reducing the sense of incongruity, it is not a problem that the pylon 24 is invisible by the vehicle body image 28 itself. If you still want to see the pylon 24, there is a solution such as using a back monitor (not shown).

(4) Fourth Embodiment Next, a fourth embodiment will be described. In the case of the fourth embodiment, the processing executed in the ECU 1 is configured by processing steps equivalent to those of the third embodiment. However, in S56, the vehicle body video combined with the output video generated in S52 is configured differently from the third embodiment.

  That is, as described above, in the case of the third embodiment, a configuration in which the pylon 24 is not visible by the own vehicle body image 28 so that a feeling of discomfort is reduced for a driver accustomed to the backward confirmation using a general door mirror. It was as. However, it is one idea to make the configuration so that the pylon 24 can be seen.

Therefore, in the fourth embodiment, in order to achieve both the mask based on the vehicle body image 28 and the display of the pylon 24, as shown in FIG. A part of the transmissive part 28B.

  The opaque part 28A is an area where the original video is not transparently displayed. The transmissive portion 28B is an area where the original video is transmissively displayed. In the case of the present embodiment, a range including a portion where an image of an actual vehicle body is reflected is masked by the non-transparent portion 28A, and a portion where an image of the actual vehicle body is not reflected is transparently displayed on the transmission portion 28B.

  In the video, the range in which the actual image of the vehicle body is reflected depends on the viewpoint position of the video shot by the left camera 3 and the right camera 4 and the specific shape of the vehicle 20. It can change depending on etc. However, it is possible to determine the mask range by adding a certain amount of margin to the specified range if necessary, specifying the range in which the image of the vehicle body is captured in advance by shooting with an actual vehicle. The range of the opaque portion 28A and the transparent portion 28B may be set by the method.

  By synthesizing the vehicle body video 28 having such a non-transmissive portion 28A and the transmissive portion 28B, the overall output video has a composition similar to the mirror image of the door mirror shown in FIG. Therefore, similarly to the third embodiment, it is possible to show an image with a little uncomfortable feeling to a driver who is used to confirming rearward with a door mirror. Moreover, in the transmission part 28B, it is possible to see an obstacle (for example, the pylon 24 shown in FIG. 11) at the place where the vehicle body image 28 is synthesized. Therefore, unlike the third embodiment, by synthesizing the vehicle body image 28, it is possible to suppress the loss of useful information regarding the obstacle. Further, since the image of the actual vehicle body is not transmissively displayed in the impermeable portion 28A, for example, it is possible to suppress misidentifying a part of the image of the actual vehicle body as an obstacle.

  When the vehicle body video 28 having the non-transparent part 28A and the transmissive part 28B as described above is synthesized, the range of the transmissive part 28B in the output video (hereinafter also referred to as the first range 29A) and the output. Regarding the range in which the vehicle body video 28 is not synthesized in the video for use (hereinafter also referred to as the second range 29B), the shape and size of each range, the position of the boundary between both ranges, and the like can be determined in advance. . For example, as the vehicle body image 28, an image corresponding to an actual vehicle generated from the image or data of the actual vehicle as described above may be used. Since the synthesis range does not change, it is possible to specify the synthesis range.

  Therefore, by utilizing the fact that the first range 29A and the second range 29B can be specified in advance, by changing the presence or absence or degree of viewpoint conversion between the first range 29A and the second range 29B, in the second range 29B May be such that image distortion is smaller than in the first range 29A.

  For example, the viewpoint conversion that causes distortion in the video may be performed in the first range 29A, while the viewpoint conversion corresponding to the first range 29A may not be performed in the second range 29B. Or, for example, although the viewpoint conversion is performed in both the first range 29A and the second range 29B, the first range 29A performs the viewpoint conversion that causes distortion in the image more than the second range 29B, while the second range 29B May perform viewpoint conversion that does not cause distortion in the image as compared with the first range 29A.

  If comprised in this way, in the 2nd range 29B which is not hidden in the own vehicle body image | video 28, an image | video with smaller distortion can be displayed. Further, in the first range 29A in which the distortion is larger than that in the second range 29B, the transmissive display in the transmissive portion 28B in the vehicle body image 28 is made inconspicuous that the image has a larger distortion. Can be.

In addition, since the boundaries of the images with different degrees of distortion are synthesized so as to coincide with the contour of the vehicle body image 28, even if there is a slight shift in the images on both sides of the boundary, such a shift is caused. Can be made inconspicuous by superimposing on the contour of the vehicle body image 28.

(5) Fifth Embodiment Next, a fifth embodiment will be described. In the case of the fifth embodiment, the process executed in the ECU 1 is different from the third embodiment and the fourth embodiment in that S54 is executed after S52 is executed. When the process proceeds to S54, the ECU 1 sets a transmission range and a mask range according to the distance to the obstacle (S54). Except for S54, the processing steps described in the first embodiment, the second embodiment, the third embodiment, and the fourth embodiment are the same as those described with the same reference numerals.

  In the fifth embodiment, as shown in FIG. 13, the vehicle body image 28 is composed of a first opaque portion 28A, a transparent portion 28B, and a second opaque portion 28C. The first non-transparent portion 28A and the second non-transparent portion 28C are regions where the original video is not transmissively displayed, and the transmissive portion 28B is a region where the original video is transmissively displayed.

  The first opaque portion 28A is set to a range that coincides with the opaque portion 28A in the fourth embodiment, and the range is the viewpoint position of the video imaged by the left camera 3 and the right camera 4, and the vehicle 20 It is set statically according to the specific shape and the like. On the other hand, the transmission part 28B and the second non-transmission part 28C are set to a range in which both of them match the non-transmission part 28A in the fourth embodiment. Moreover, the range of the second impermeable portion 28C is dynamically increased or decreased according to the distance to the obstacle.

  For example, the output video illustrated in FIG. 13 is a video assuming that another vehicle 26 exists behind the vehicle but the pylon 24 illustrated in FIG. 11 does not exist. In this case, as the distance to the obstacle, the distance to the other vehicle 26 is detected, and the range from the host vehicle to the other vehicle 26 is masked by the second impermeable portion 28C. A range farther from the other vehicle 26 when viewed from the host vehicle side is a transmission portion 28B. Although illustration is omitted, when the pylon 24 exists, the distance to the pylon 24 is detected as the distance to the obstacle, and the range from the own vehicle to the pylon 24 in the transmission part 28B is detected. The second opaque portion 28C is masked. A range farther from the pylon 24 when viewed from the host vehicle side is a transmission portion 28B.

  As the distance to the obstacle, a distance actually measured by the obstacle detection unit 7 may be adopted. If necessary, the transmission part 28B may be increased by a predetermined margin. In this case, the distance corresponding to the predetermined margin may be subtracted from the distance actually measured by the obstacle detection unit 7. Good.

  If the configuration as described above is adopted, it can be determined that there is no obstacle in the range masked by the second impermeable portion 28C. Further, for example, a portion where there is an obstacle that is not an obstacle such as the parking frame 22 (that does not have a height protruding from the road surface) is masked by the second non-permeable portion 28C, and thus the transmissive portion 28B. Unlike the case where the parking frame 22 can be seen through, it is possible to suppress misidentifying the parking frame 22 as an obstacle.

In the fifth embodiment, the range of the transmission part 28B in the output video (first range 29A) and the range in which the vehicle body video 28 in the output video is not combined (second range 29B) Similarly to the fourth embodiment, the presence or absence or degree of viewpoint conversion may be changed. As a result, in the second range 29B, the image distortion is smaller than that in the first range 29A, so that an image with less distortion is displayed in the second range 29B that is not hidden by the vehicle body image 28. can do. Further, in the first range 29A in which the distortion is larger than that in the second range 29B, the transmissive display in the transmissive portion 28B in the vehicle body image 28 is made inconspicuous that the image has a larger distortion. Can be. Further, even when a slight shift occurs in the video at the boundary of the video with different degrees of distortion, the shift can be made inconspicuous by superimposing such a shift on the contour of the vehicle body video 28.

(6) Other Embodiments While the video processing device and the vehicle-mounted video processing system have been described with reference to exemplary embodiments, the above-described embodiments are merely examples of the present invention. . That is, the present invention is not limited to the exemplary embodiments described above, and can be implemented in various forms without departing from the technical idea of the present invention.

  For example, in the above embodiment, for convenience of illustration, the storage unit 14 is illustrated as a configuration provided in the ECU 1 for convenience. However, the storage unit 14 is located outside the ECU 1 as long as the memory is accessible by the ECU 1. Also good.

  In the above embodiment, a predetermined function realized by one component may be configured so that a plurality of components cooperate to realize it. Or in the said embodiment, it is comprised so that one component may implement | achieve the several function which each of several component each had, and the predetermined function which the plurality of component realized in cooperation It may be. Further, at least a part of the configuration of the above embodiment may be replaced with a known configuration having the same function. Moreover, you may abbreviate | omit a part of structure of the said embodiment. In addition, at least a part of the configuration of the above embodiment may be added to or replaced with the configuration of the other embodiment. In addition, all the aspects included in the technical idea specified only by the wording described in the claim are embodiment of this invention.

  In addition to the above-described video processing device and in-vehicle video processing system, the present invention can also be realized in various forms such as a program for causing a computer to function as the video processing device and a medium storing the program.

  In addition, in order to make it easy to understand the correspondence between the embodiments and the claims, the reference numerals shown in the embodiments are described in parentheses in the claims. This is not a description meaning that the technical scope of the present invention is limited to the above-described embodiment.

  DESCRIPTION OF SYMBOLS 1 ... Electronic control unit (ECU), 3 ... Left camera, 4 ... Right camera, 5 ... Left monitor, 6 ... Right monitor, 7 ... Obstacle detection part, 8 ... Shift position detection part, 11 ... Video input part, 12 ... Video processing unit, 13 ... Video output unit, 14 ... Storage unit, 15 ... Obstacle determination unit, 16 ... Shift position determination unit, 20 ... Vehicle, 20A ... Rear wheel, 22 ... Parking frame, 24 ... Pylon, 26 ... Another vehicle, 28 ... own body image, 28A ... opaque portion or first opaque portion, 28B ... transparent portion, 28C ... second opaque portion, 29A ... first range, 29B ... second range .

Claims (7)

An obstacle determination unit (15, S30) for acquiring information on the obstacle from an obstacle detection unit (7) capable of detecting an obstacle existing behind the vehicle and determining the presence or absence of the obstacle;
A video input unit (11, S10) capable of inputting a camera video captured by the camera from a camera (3,4) configured to be able to capture the rear side of the vehicle;
When the first output video and the second output video can be generated based on the camera video input by the video input unit, and the obstacle determination unit determines that there is the obstacle, The first output video is generated, and when the obstacle determination unit determines that there is no obstacle, the video processing unit (12, S50, S52) generates the second output video. )When,
A video output unit (13, S60) that outputs the first output video or the second output video generated by the video processing unit to a display device (5, 6) disposed in a vehicle interior. )When,
With
The video processing unit changes at least one of a viewpoint position, a line-of-sight direction, and a visual field range when generating at least one of the first output video and the second output video. As described above, it is possible to generate the first output video and the second output video that are different from each other by performing video conversion processing for changing the change target on the camera video (S50, S52), as the video conversion process, the video conversion process in which the obstacle is displayed larger than the second output video in the first output video, or the second output video. A video processing device (1) configured to be capable of executing a video conversion process in which a part or all of the obstacle that is not displayed is displayed in the first output video. The video processing apparatus according to claim 1,
The said obstacle determination part is comprised so that the presence or absence of the said obstacle may be determined when the said vehicle reverses (S20: YES). In the video processing device according to claim 1 or 2,
When the video processing unit generates at least one of the first output video and the second output video, the vehicle body video (28) corresponding to a part of the vehicle body is stored in the storage unit. (14) is generated based on the data read from or stored in the storage unit, and the camera image or the image of the actual vehicle body reflected in the image generated based on the camera image is covered. A video processing apparatus configured to execute the process (S56) of drawing the vehicle body video. The video processing apparatus according to claim 3.
The video processing unit (S56) is generated based on the camera video or the camera video and the non-transparent unit (28A, 28C) where the video generated based on the camera video or the camera video is not transmissively displayed. The vehicle body image having a transmissive portion (28B) through which the image is transmissively displayed can be generated, and a range including a portion where the actual vehicle image is reflected is masked by the non-transmissive portion, The video processing apparatus is configured such that a portion where the video of the vehicle body is not reflected is displayed in a transmissive manner in the transmissive portion. The video processing apparatus according to claim 4,
The video processing unit (S54) measures or estimates the distance from the vehicle to the obstacle based on the information acquired from the obstacle detection unit, and the range in a position closer to the obstacle is the A video processing apparatus configured to be masked by a transmissive portion (28C) and to be transparently displayed in the transmissive portion at a location beyond the position where the obstacle exists. In the video processing device according to claim 4 or 5,
The video processing unit (S54, S56) determines whether or not the video conversion process is performed in a first range (29A) that is the transmission unit and a second range (29B) in which the vehicle body video is not combined. The video processing apparatus configured to execute the video conversion process in which the video distortion is smaller in the second range than in the first range by changing. An obstacle detection unit (7) capable of detecting obstacles existing behind the vehicle;
A camera (3, 4) configured to photograph the rear side of the vehicle;
A video processing device (1) configured to perform predetermined video conversion processing on video captured by the camera, and a video that is disposed in a vehicle interior and that has been subjected to the video processing by the video processing device. A display device (5, 6) capable of displaying;
The video processing device includes:
An obstacle determination unit (15, S30) that acquires information on the obstacle from the obstacle detection unit (7) and determines the presence or absence of the obstacle;
A video input unit (11, S10) capable of inputting a camera video captured by the camera from the camera (3, 4);
When the first output video and the second output video can be generated based on the camera video input by the video input unit, and the obstacle determination unit determines that there is the obstacle, The first output video is generated, and when the obstacle determination unit determines that there is no obstacle, the video processing unit (12, S50, S52) generates the second output video. )When,
A video output unit (13, S60) that outputs the first output video or the second output video generated by the video processing unit to a display device (5, 6) disposed in a vehicle interior. )When,
With
The video processing unit changes at least one of a viewpoint position, a line-of-sight direction, and a visual field range when generating at least one of the first output video and the second output video. As described above, it is possible to generate the first output video and the second output video that are different from each other by performing video conversion processing for changing the change target on the camera video (S50, S52), as the video conversion process, the video conversion process in which the obstacle is displayed larger than the second output video in the first output video, or the second output video. A vehicle-mounted video processing system configured to be able to execute a video conversion process in which a part or all of the obstacle that is not displayed is displayed in the first output video.