JP2020162033A - Image processing device and image processing method - Google Patents

Abstract

An object of the present invention is to provide an image processing apparatus and an image processing method for generating a composite image that allows passengers of a vehicle to easily understand the inclination angle of the vehicle without reducing the visibility of an image of the exterior of the vehicle. . A camera image generated by a camera 11 and an inclination calculator 22 for calculating a roll angle 22A and a pitch angle 22B of the vehicle 1 are acquired from the output of a sensor 12 that detects posture information of the vehicle 1. a first image generating unit 23 for generating a first image 100A having a first predetermined shape having a display mode according to the roll angle 22A; and a second predetermined shape having a display mode according to the pitch angle. The second image generation unit 24 that generates the second image 200A of the second image 100A is arranged at the upper end or the lower end of the camera image acquired by the image acquisition unit 21, and the first image 100A is arranged at the left end or right end of the camera image. and an image composition unit 25 that arranges the second image 200A and generates a composite image 25A. [Selection drawing] Fig. 3

Description

The present invention relates to an image processing apparatus and an image processing method.

There is known an image processing device that simultaneously displays an image of the outside of the vehicle and the situation of the vehicle on a display mounted on the vehicle. Patent Document 1 discloses an image processing device that displays an image of the front of the vehicle and an icon indicating an inclination angle in the left-right direction of the vehicle on a display mounted on the vehicle.

When displaying the tilt angle of the vehicle using an icon or the like, it is considered that the passenger of the vehicle can easily understand the tilt angle of the vehicle as the size of the icon or the like is large. However, if the size of the icon or the like indicating the inclination angle of the vehicle is increased, the display space of the image of the front of the vehicle or the like becomes smaller, so that the visibility of the image of the front of the vehicle or the like is lowered. On the other hand, if the size of the icon or the like indicating the tilt angle of the vehicle is reduced, it is considered that it becomes difficult for the passengers of the vehicle to understand the tilt angle of the vehicle.

Therefore, it is preferable to allow the passengers of the vehicle to easily understand the inclination angle of the vehicle without deteriorating the visibility of the image of the outside of the vehicle.

Japanese Unexamined Patent Publication No. 2016-49868

The present invention provides an image processing device and an image processing method for generating a composite image that allows a occupant of a vehicle to easily understand the tilt angle of the vehicle without deteriorating the visibility of an image taken outside the vehicle. The purpose is to do.

(First invention)
The image processing device according to the first invention uses an inclination calculation unit that calculates the roll angle of the vehicle and the pitch angle of the vehicle from the output of a sensor that detects the attitude information of the vehicle, and a camera image generated by the camera. An image acquisition unit to be acquired, a first image generation unit that generates a first image of a first predetermined shape having a display mode corresponding to the roll angle, and a second predetermined shape having a display mode corresponding to the pitch angle. The first image is arranged at the upper end or the lower end of the camera image acquired by the image acquisition unit and the second image generation unit that generates the second image, and the first image is placed at the left end or the right end of the camera image. It includes an image compositing unit for arranging a second image and generating a compositing image.

In the first invention, the image processing device synthesizes the camera image generated by the camera with the first image and the second image displaying the inclination of the vehicle to generate a composite image. Since the first image has the first predetermined shape and the second image has the second predetermined shape, a wide display area of the camera image can be secured in the composite image. Therefore, the image processing device according to the first invention can easily make the occupants of the vehicle understand the inclination angle of the vehicle without deteriorating the visibility of the camera image taken from the outside of the vehicle.

(Second invention)
In the second invention, the first image has a first frame area corresponding to the first predetermined shape, a roll angle display area arranged inside the first frame area and showing the roll angle, and the first image. The first image generation unit includes a first background area which is arranged inside one frame area and is a region different from the roll angle display area, and the first image generation unit is a roll angle display area based on the roll angle. The position is determined, the pixel value in the first background region is determined to be a pixel value different from the pixel value in the roll angle display region, and the second image is the second frame region corresponding to the second predetermined shape. A pitch angle display area arranged inside the second frame area and showing the pitch angle, and a second background area arranged inside the second frame area and different from the pitch angle display area. The second image generation unit determines the position of the pitch angle display area based on the pitch angle, and the pixel value in the second background area is different from the pixel value in the pitch angle display area. To decide.

In the second invention, the position of the roll angle display area is changed according to the roll angle, and the position of the pitch angle display area is changed according to the pitch angle. The pixel value of the roll angle display area is different from the pixel value of the first background area, and the pixel value of the pitch angle display area is different from the pixel value of the second background area. Therefore, the passenger of the vehicle can easily recognize the roll angle and the pitch angle.

(Third invention)
In the third invention, the first image generation unit determines the color of the roll angle display area based on the roll angle, and the second image generation unit determines the pitch angle display area based on the pitch angle. Determine the color of.

In the third invention, the color of the roll angle display area is changed according to the roll angle, and the color of the pitch angle display area is changed according to the pitch angle. Therefore, the passenger of the vehicle can easily understand the inclination angle of the vehicle by the color of the roll angle display area and the color of the pitch angle display area.

(Fourth invention)
In the fourth invention, the first image is generated so as to extend in the lateral direction, and the first image generation unit arranges the roll angle display area to the right of a reference line indicating a roll angle of 0 degrees. Whether or not it is determined based on the roll angle, the second image is formed so as to extend in the vertical direction, and the second image generation unit displays the pitch angle display area from a reference line indicating a pitch angle of 0 degrees. Whether or not to place the image on the top is determined based on the pitch angle.

In the fourth invention, it is determined whether or not to arrange the roll angle display area to the right of the reference line based on the roll angle of the vehicle, and the pitch angle display area is arranged on the reference line based on the pitch angle of the vehicle. Decide whether to do it or not. Therefore, the passenger of the vehicle can easily understand which direction is tilted in the left-right direction and which direction is tilted in the front-rear direction.

(Fifth invention)
In a fifth invention, the first image is generated so as to extend in the lateral direction, and the first background area includes a left background area arranged to the left of the roll angle display area and the roll angle display area. The first image generation unit gradually increases the pixel values of the left background area and the right background area as the distance from the roll angle display area increases, including a right background area arranged to the right of the above. The second image is changed and generated so as to extend in the vertical direction, and the second background area is arranged above the pitch angle display area and below the pitch angle display area. The second image generation unit gradually changes the pixel values of the upper background region and the right background region as the distance from the pitch angle display region increases.

In the fifth invention, the first image generation unit gradually changes the pixel values of the left background area and the right background area as the distance from the roll angle display area increases. In addition, the second image generation unit gradually changes the pixel values of the upper background region and the right background region as the distance from the pitch angle display region increases. Therefore, the image processing device according to the fifth invention can easily make the passenger of the vehicle understand the inclination angle of the vehicle.

(Sixth Invention)
In the sixth invention, the first image is generated so as to extend in the horizontal direction, and the first image generation unit determines the length of the roll angle display area in the horizontal direction based on the roll angle. It is determined based on the roll angle whether the roll angle display area is arranged so as to extend to the right or to the left from the reference line indicating the roll angle of 0 degrees, and the second image is vertical. The second image generation unit is formed so as to extend in the direction, determines the length of the pitch angle display area in the vertical direction based on the pitch angle, and determines the pitch angle display area as a reference indicating that the pitch angle is 0 degrees. Whether to arrange so as to extend upward or downward from the line is determined based on the pitch angle.

In the sixth invention, the roll angle display region extends in a direction based on the roll angle and has a length based on the roll angle. Further, the pitch angle display area extends in a direction based on the pitch angle and has a length based on the pitch angle. Therefore, the passenger of the vehicle can easily understand the inclination angle of the vehicle depending on the direction and length in which the roll angle display area and the pitch angle display area extend.

(7th invention)
In the seventh invention, the roll angle display area has a constant length and is arranged at a position based on the roll angle, and the pitch angle display area has a constant length and the pitch angle. It is placed in a position based on.

In the seventh invention, the position of the roll angle display area changes based on the roll angle, and the position of the pitch angle display area changes based on the pitch angle. The passenger of the vehicle can easily understand the inclination angle of the vehicle based on the position of the pitch angle display area and the position of the roll angle display area.

(8th invention)
In the eighth invention, the first image is generated so as to extend in the lateral direction, and the first image generation unit rotates the first image extending in the lateral direction at an angle corresponding to the roll angle.

In the eighth invention, the inclination of the first image extending in the lateral direction changes according to the roll angle. Therefore, the passenger of the vehicle can understand the tilt angle of the vehicle 1 in the left-right direction from the tilt of the first image.

(Ninth invention)
In the ninth invention, the first image generation unit arranges characters representing the roll angle in the vicinity of the first frame region, and the second image generation unit places characters representing the pitch angle in the second. Place it near the frame area.

In the image processing apparatus according to the ninth invention, the characters representing the roll angle of the vehicle are arranged in the vicinity of the first frame area, and the characters representing the pitch angle of the vehicle are arranged in the vicinity of the second frame area. The occupants of the vehicle can more easily understand the tilt angle of the vehicle.

(10th invention)
The image processing device according to the tenth invention includes a tilt calculation unit that calculates the pitch angle of the vehicle from the output of a sensor that detects the attitude information of the vehicle, and an image acquisition unit that acquires a camera image generated by the camera. , The relative position of the front end surface image corresponding to the front end surface of the vehicle and the rear end surface image corresponding to the rear end surface of the vehicle is determined based on the pitch angle, and the front end surface arranged at the determined relative position. A composite image is generated by combining a pitch angle image generation unit that generates a pitch angle display image including an image and a rear end surface image, and the camera image and the pitch angle display image acquired by the image acquisition unit. It includes an image compositing unit.

In the tenth invention, the relative position between the front end surface image showing the front end surface of the vehicle and the rear end surface of the vehicle changes based on the pitch angle. Therefore, the passenger of the vehicle can easily understand the inclination angle in the front-rear direction of the vehicle.

(11th invention)
The image processing method according to the eleventh invention includes a tilt calculation step of calculating the roll angle of the vehicle and the pitch angle of the vehicle from the output of a sensor that detects the attitude information of the vehicle, and a camera image generated by the camera. An image acquisition step to be acquired, a first image generation step of generating a first image of a first predetermined shape having a display mode corresponding to the roll angle, and a second predetermined shape having a display mode corresponding to the pitch angle. The first image is arranged at the upper end or the lower end of the camera image acquired by the second image generation step of generating the second image and the image acquisition step, and the first image is placed at the left end or the right end of the camera image. It includes an image compositing step of arranging a second image and generating a compositing image.

In the eleventh invention, the image processing method combines the camera image generated by the camera with the first image and the second image displaying the inclination of the vehicle to generate a composite image. Since the first image has the first predetermined shape and the second image has the second predetermined shape, a wide display area of the camera image can be secured in the composite image. Therefore, the image processing method according to the eleventh invention can easily make the occupants of the vehicle understand the inclination angle of the vehicle without deteriorating the visibility of the camera image taken from the outside of the vehicle.

The present invention provides an image processing device and an image processing method for generating a composite image that allows a occupant of a vehicle to easily understand the tilt angle of the vehicle without deteriorating the visibility of an image taken outside the vehicle. can do.

An example of a schematic view of a vehicle equipped with the image recording device according to the first embodiment of the present invention is shown. It is a figure explaining the direction in the vehicle shown in FIG. It is a functional block diagram of the image processing apparatus shown in FIG. It is a figure explaining the composite image synthesized by the image processing apparatus shown in FIG. It is a figure explaining the operation of the image processing apparatus shown in FIG. It is a figure explaining the 1st image which concerns on Embodiment 2 of this invention. It is a figure explaining the 2nd image which concerns on Embodiment 2 of this invention. It is a figure explaining the 1st image and 2nd image which concerns on Embodiment 3 of this invention. It is a figure explaining the 1st image and 2nd image which concerns on Embodiment 4 of this invention. It is a figure explaining the 1st image and 2nd image which concerns on Embodiment 5 of this invention. It is a functional block diagram of the image processing apparatus which concerns on Embodiment 6 of this invention. It is a figure explaining the composite image synthesized by the image processing apparatus shown in FIG. It is the first example of the figure explaining the 1st image and the 2nd image which concerns on the modification of this invention. It is the 2nd example of the figure explaining the 1st image and the 2nd image which concerns on the modification of this invention. It is the 3rd example of the figure explaining the 1st image and the 2nd image which concerns on the modification of this invention. It is the 4th example of the figure explaining the 1st image and the 2nd image which concerns on the modification of this invention. It is a bus block diagram which connects the CPU and the like which concerns on the modification of this invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. The present invention is not limited to the embodiments shown below. In addition, these examples of implementation are merely examples, and the present invention can be implemented in a form in which various modifications and improvements have been made based on the knowledge of those skilled in the art. In addition, the components having the same reference numerals in the present specification and the drawings shall indicate the same or equivalent components.

(Embodiment 1)
FIG. 1 shows a schematic view of a vehicle 1 equipped with an image processing device according to a first embodiment of the present invention. As shown in FIG. 1, the camera 11, the acceleration sensor 12, the display 13, and the image processing device 20 are mounted on the vehicle 1.

The camera 11 is a front camera mounted on the front portion of the vehicle 1. The camera 11 photographs the front of the vehicle 1 and generates a moving image including a plurality of frames. The camera 11 supplies the generated moving image to the image processing device 20 in frame units. In the following description, the frame of the moving image generated by the camera 11 will be referred to as “camera image 11A”.

The acceleration sensor 12 is attached to the center of the vehicle 1 or near the center of the vehicle 1 and detects acceleration in the three axes of the X-axis, the Y-axis, and the Z-axis of the vehicle 1 as attitude information of the vehicle 1. The X-axis, Y-axis and Z-axis of the vehicle 1 will be described later. The acceleration sensor 12 supplies the detected acceleration in the three axial directions to the image processing device 20 as acceleration information 12A which is posture information of the vehicle 1.

The image processing device 20 generates a first image showing the inclination of the vehicle 1 in the left-right direction and a second image showing the inclination of the vehicle 1 in the front-rear direction based on the acceleration information 12A. The image processing device 20 combines the first image and the second image with the camera image 11A to generate a composite image 25A. The image processing device 20 supplies the generated composite image 25A to the display 13.

The display 13 receives the composite image 25A from the image processing device 20 and displays the received composite image 25A.

In FIG. 1, the camera 11, the acceleration sensor 12, the display 13, and the image processing device 20 are shown by solid lines for easy viewing, but are actually arranged inside the vehicle 1.

FIG. 2 shows a three-dimensional Cartesian coordinate system defined by vehicle 1. The X-axis, Y-axis, and Z-axis are coordinate axes of a three-dimensional Cartesian coordinate system with the center of the vehicle 1 as the origin. In FIG. 2, the X-axis, Y-axis, and Z-axis conform to, for example, ISO (International Organization for Standardization) 8855. The X-axis is an axis extending in the front-rear direction of the vehicle 1. The Y-axis is an axis extending in the left-right direction of the vehicle 1. The Z axis is an axis extending in the vertical direction of the vehicle 1.

The positive direction of the X-axis is the direction from the driver's seat to the steering wheel. The positive direction of the Y-axis is the left direction with respect to the front of the vehicle 1. The positive direction of the Z axis is above the vehicle 1 when viewed from the driver's seat.

<Configuration of image processing device 20>
FIG. 3 shows a functional block diagram showing the configuration of the image processing device 20. As shown in FIG. 3, the image processing device 20 includes an image acquisition unit 21, a tilt calculation unit 22, an image correction unit 28, a tilt image generation unit 26A, and an image composition unit 25.

The image acquisition unit 21 acquires the camera image 11A from the camera 11. The image acquisition unit 21 supplies the acquired camera image 11A to the image correction unit 28.

The inclination calculation unit 22 receives the acceleration information 12A from the acceleration sensor 12, and calculates the roll angle 22A of the vehicle 1 and the pitch angle 22B of the vehicle 1 from the received acceleration information 12A. The roll angle 22A is a rotation angle of the vehicle 1 with the X axis as the rotation axis. That is, the roll angle 22A indicates the inclination of the vehicle 1 in the left-right direction. The positive direction of the roll angle is the clockwise direction when the X-axis is viewed in the positive direction from the origin. The pitch angle 22B is the rotation angle of the vehicle 1 with the Y axis as the rotation axis. That is, the pitch angle 22B indicates the inclination of the vehicle 1 in the front-rear direction. The positive direction of the pitch angle is the clockwise direction when the Y-axis is viewed in the positive direction from the origin. The tilt calculation unit 22 supplies the calculated roll angle 22A and pitch angle 22B to the tilt image generation unit 26A. Further, the inclination calculation unit 22 supplies the calculated roll angle 22A to the image correction unit 28.

The tilt image generation unit 26A receives a roll angle 22A and a pitch angle 22B from the tilt calculation unit 22. The tilt image generation unit 26A generates the first image 100A using the received roll angle 22A, and generates the second image 200A using the received pitch angle 22B. The tilt image generation unit 26A includes a first image generation unit 23 and a second image generation unit 24.

The first image generation unit 23 receives a roll angle 22A from the inclination calculation unit 22. The first image generation unit 23 generates a first image 100A having a first predetermined shape having a display mode corresponding to the received roll angle 22A. That is, the first image 100A is an image showing the inclination of the vehicle 1 in the left-right direction. Details of the first image 100A will be described later. The first image generation unit 23 supplies the generated first image 100A to the image composition unit 25.

The second image generation unit 24 receives the pitch angle 22B from the inclination calculation unit 22. The second image generation unit 24 generates a second image 200A having a second predetermined shape having a display mode according to the received pitch angle 22B. That is, the second image 200A is an image showing the inclination of the vehicle 1 in the front-rear direction. The details of the second image 200A will be described later. The second image generation unit 24 supplies the generated second image 200A to the image composition unit 25.

The image correction unit 28 receives the camera image 11A from the image acquisition unit 21 and the roll angle 22A from the tilt calculation unit 22. The image correction unit 28 corrects the inclination of the received camera image 11A in the left-right direction by using the received roll angle 22A. The camera image 11A corrected for tilt in the left-right direction corresponds to an image generated by the camera 11 when the roll angle is 0 degrees. The image correction unit 28 supplies the camera image 11A whose inclination in the left-right direction has been corrected to the image composition unit 25 as the camera image 11B.

The image composition unit 25 receives the camera image 11B from the image correction unit 28, the first image 100A from the first image generation unit 23, and the second image 200A from the second image generation unit 24. The image synthesizing unit 25 arranges the first image 100A at the upper end of the camera image 11B and arranges the second image 200A at the right end of the corrected camera image 11B to generate the composite image 25A. The details of the composite image 25A will be described later. The image synthesizing unit 25 supplies the generated composite image 25A to the display 13.

<Generation of composite image 25A>
FIG. 4 shows an example of the composite image 25A generated by the image compositing unit 25. The composite image 25A has a camera image 11B, a first image 100A, and a second image 200A. In FIG. 4, the display of the subject included in the camera image 11B is omitted.

In the description of FIG. 4, the roll angle 22A will be described as −10 degrees, and the pitch angle 22B will be described as 10 degrees.

The first image 100A is a bar-shaped image extending in the lateral direction of the camera image 11B, and is arranged at the upper end of the camera image 11B. Specifically, the first image 100A is arranged along the upper side of the camera image 11B. The upper side of the camera image 11B is the side of the upper end of the camera image 11B. The second image 200A is a bar-shaped image extending in the vertical direction of the camera image 11B, and is arranged at the right end of the camera image 11B. Specifically, the second image 200A is arranged along the right side of the camera image 11B. The right side of the camera image 11B is the right side of the camera image 11B. The first images 100A and 200A are arranged on the camera image 11B.

The first image 100A includes a first frame area 101A, a roll angle display area 102A, a first background area 103A, and a first reference line 104A.

The first frame region 101A indicates the maximum region of the first image 100A. The first frame area 101A extends in the lateral direction of the camera image 11B. That is, the horizontal size of the first frame area 101A is longer than the vertical size of the first frame area 101A. Inside the first frame area 101A, a roll angle display area 102A, a first background area 103A, and a first reference line 104A are arranged.

The first reference line 104A is a line arranged at the center of the first frame area 101A in the lateral direction. The position of the first reference line 104A corresponds to the case where the roll angle 22A is 0 degrees.

The first image generation unit 23 determines the position and color of the roll angle display area 102A based on the roll angle 22A.

First, the position of the roll angle display area 102A will be described. When the roll angle 22A is positive, the first image generation unit 23 arranges the roll angle display area 102A to the right of the first reference line 104A. When the roll angle 22A is negative, the first image generation unit 23 arranges the roll angle display area 102A to the left of the first reference line 104A. When the roll angle is 0 degrees, the first image generation unit 23 does not arrange the roll angle display area 102A inside the first frame area 101A.

Next, the color of the roll angle display area 102A will be described. The first image generation unit 23 determines the color of the roll angle display area 102A based on the absolute value of the roll angle 22A. For example, when the absolute value of the roll angle 22A is greater than 0 degrees and less than 10 degrees, the first image generation unit 23 fills the roll angle display area 102A with yellow. When the absolute value of the roll angle 22A is 10 degrees or more and less than 20 degrees, the first image generation unit 23 fills the roll angle display area 102A in orange. When the absolute value of the roll angle 22A is 20 degrees or more, the first image generation unit 23 fills the roll angle display area 102A with red.

In the example shown in FIG. 4, since the roll angle 22A is −10 degrees, the roll angle display area 102A is arranged to the left of the first reference line 104A. Further, the roll angle display area 102A is filled with orange.

The first background area 103A is an area inside the first frame area 101A that is different from the roll angle display area 102A. Specifically, the first background area 103A is arranged on the opposite side of the roll angle display area 102A with respect to the first reference line 104A. The pixel value of the first background region 103A shows a color different from the color indicating the angle of the roll angle 22A. For example, the color of the first background region 103A is fixed and white.

The second image 200A includes a second frame area 201A, a pitch angle display area 202A, a second background area 203A, and a second reference line 204A.

The second frame area 201A indicates the maximum area of the second image 200A. The second frame area 201A extends in the vertical direction of the camera image 11B. That is, the vertical size of the second frame area 201A is longer than the horizontal size of the second frame area 201A. Inside the second frame area 201A, a pitch angle display area 202A, a second background area 203A, and a second reference line 204A are arranged.

The second reference line 204A is a line arranged at the center of the second frame area 201A in the vertical direction. The position of the second reference line 204A corresponds to the case where the pitch angle 22B is 0 degrees.

The second image generation unit 24 determines the position and color of the pitch angle display area 202A based on the pitch angle 22B.

First, the position of the pitch angle display area 202A will be described. When the pitch angle 22B is positive, the second image generation unit 24 arranges the pitch angle display area 202A above the second reference line 204A. When the pitch angle 22B is negative, the second image generation unit 24 arranges the pitch angle display area 202A below the second reference line 204A. When the pitch angle 22B is 0 degrees, the second image generation unit 24 does not arrange the pitch angle display area 202A inside the second frame area 202A.

Next, the color of the pitch angle display area 202A will be described. The second image generation unit 24 determines the color of the pitch angle display area 202A based on the pitch angle 22B. For example, when the absolute value of the pitch angle 22B exceeds 0 degrees and is less than 10 degrees, the second image generation unit 24 fills the pitch angle display area 202A with yellow. When the absolute value of the pitch angle 22B is 10 degrees or more and less than 20 degrees, the second image generation unit 24 fills the pitch angle display area 202A in orange. When the absolute value of the pitch angle 22B is 20 degrees or more, the second image generation unit 24 fills the pitch angle display area 202A with red.

In the example shown in FIG. 4, since the pitch angle 22B is 10 degrees, the pitch angle display area 202A is arranged above the second reference line 204A. The pitch angle display area 202A is filled with orange.

The second background area 203A is a region inside the second frame area 201A that is different from the pitch angle display area 202A. The second background area 203A is arranged on the side opposite to the pitch angle display area 202A with respect to the second reference line 204A. The pixel value of the second background region 203A shows a color different from the color indicating the angle of the pitch angle 22B. For example, the color of the second background region 203A is fixed and white.

<Operation of image processing device 20>
FIG. 5 shows a flowchart showing the operation of the image processing device 20. The image processing method according to the present embodiment includes each step shown in this flowchart. The image processing device 20 executes this flowchart when the ignition signal is turned on. Further, the image processing device 20 periodically executes this flowchart until the ignition signal is turned off. For example, when the frame rate of the camera 11 is 30 fps (frames per second), the image processing device 20 executes this flowchart every 10 frames.

The inclination calculation unit 22 receives acceleration information 12A from the acceleration sensor 12. The inclination calculation unit 22 calculates the roll angle 22A and the pitch angle 22B based on the received acceleration information 12A (step S101). Step S101 corresponds to the inclination calculation step in the image processing method according to the present embodiment.

The image acquisition unit 21 acquires the camera image 11A generated by the camera 11 (step S102). Step S102 corresponds to the image acquisition step in the image processing method according to the present embodiment. The image acquisition unit 21 supplies the acquired camera image 11A to the image correction unit 28.

The image correction unit 28 corrects the camera image 11A acquired in step S102 based on the roll angle 22A calculated in step S101 (step S103). The image correction unit 28 supplies the camera image 11B generated in step S103 to the image composition unit 25.

The first image generation unit 23 generates the first image 100A based on the roll angle 22A calculated in step S101 (step S104). Step S104 corresponds to the first image generation step in the image processing method according to the present embodiment. The first image generation unit 23 supplies the generated first image 100A to the image synthesis unit 25.

The second image generation unit 24 generates the second image 200A based on the pitch angle 22B calculated in step S101 (step S105). Step S105 corresponds to the second image generation step in the image processing method according to the present embodiment. The second image generation unit 24 supplies the generated second image 200A to the image synthesis unit 25.

The image synthesizing unit 25 generates a composite image 25A by using the camera image 11B generated in step S103, the first image 100A generated in step S104, and the second image 200A generated in step S105. (Step S106). Step S106 corresponds to the image composition step in the image processing method according to the present embodiment.

As described above, the image processing device 20 combines the camera image 11B with the bar-shaped first image 100A and the bar-shaped second image 200A for displaying the inclination of the vehicle to generate the composite image 25A. .. Therefore, the display area of the camera image 11B in the composite image 25A can be widened. The image processing device 20 can easily make the passengers of the vehicle 1 understand the inclination angle of the vehicle 1 without deteriorating the visibility of the camera image taken from the outside of the vehicle 1.

Further, the image processing device 20 changes the position of the roll angle display area 102A according to the roll angle 22A, and changes the position of the pitch angle display area 202A according to the pitch angle 22B. Therefore, the colors of the roll angle display area 102A and the pitch angle display area 202A can make the passenger of the vehicle 1 easily understand the inclination angle of the vehicle 1.

(Embodiment 2)
FIG. 6 shows the first image 100B generated by the image processing apparatus according to the second embodiment of the present invention, and FIG. 7 shows the second image 200B generated by the image processing apparatus according to the second embodiment of the present invention. The configuration of the image processing device according to the present embodiment is the same as the configuration of the image processing device 20 according to the first embodiment. Therefore, the image processing device according to the present embodiment is described as "image processing device 20", and detailed description of its configuration will be omitted.

The composite image generated by the image processing apparatus 20 according to the present embodiment is the same as the composite image 25A according to the first embodiment except for the first image 100B and the second image 200B. Therefore, in the description of the present embodiment, the description of the parts other than the first image 100B and the second image 200B will be omitted.

The pixel value of the pixel in this embodiment includes not only the color of the pixel such as RGB but also the transmittance. In the present embodiment, the first image generation unit 23 separates the transmittance of the pixels of the background area arranged to the left and right of the roll angle display area 102B from the roll angle display area 102B in the first image 100B. Gradually change according to. Further, the second image generation unit 24 gradually changes the transmittance of the pixels in the background region arranged above and below the pitch angle display area 202B in the second image 200B as the distance from the pitch angle display area 202B increases. Let me. The details will be described below.

In the description of FIGS. 6 and 7, the roll angle 22A will be described as −10 degrees, and the pitch angle 22B will be described as 10 degrees.

First, the first image 100B according to the present embodiment will be described with reference to FIG. The first image generation unit 23 generates the first image 100B according to the present embodiment. The first image 100B has a first frame area 101B, a roll angle display area 102B, background areas 103LB and 103RB, and a first reference line 104B.

The position of the roll angle display area 102B will be described. The roll angle display area 102B is arranged inside the first frame area 101B. The roll angle display area 102B has a constant length in the horizontal method. The lateral length of the roll angle display area 102B is the length from the first reference line 104B to the left end 105B of the first image 100B and from the first reference line 104B to the right end 106B of the first image 100B. Shorter than the length.

The roll angle display area 102B indicates the roll angle 22A of the vehicle 1 depending on the position where it is arranged. Specifically, the roll angle display area 102B indicates the direction and angle of inclination of the vehicle 1 in the left-right direction depending on the arranged position.

When the roll angle 22A is negative, the first image generation unit 23 arranges the roll angle display area 102B to the left of the first reference line 104B. Further, the first image generation unit 23 increases the distance between the roll angle display area 102B and the reference line 104B as the absolute value of the roll angle 22A increases. Further, when the roll angle 22A is positive, the first image generation unit 23 arranges the roll angle display area 102B on the right side of the first reference line 104B. Further, the first image generation unit 23 increases the distance between the roll angle display area 102B and the reference line 104B as the absolute value of the roll angle 22A increases. When the roll angle 22A is 0 degrees, the first image generation unit 23 arranges the roll angle display area 102B so as to overlap the first reference line 104B.

In the description of FIG. 6, the roll angle 22A of the vehicle 1 is −10 degrees. Therefore, the roll angle display area 102B is to the left of the first reference line 104B and is arranged at a position corresponding to 10 degrees.

The color of the roll angle display area 102B will be described. The color of the roll angle display area 102B is, for example, blue. The color of the roll angle display area 102B does not change depending on the roll angle 22A.

The background area 103LB is a left background area arranged to the left of the roll angle display area 102B. The color of the background area 103LB is the same as the color of the roll angle display area 102B. The first image generation unit 23 gradually changes the pixel values of the pixels included in the background area 103LB as the distance from the roll angle display area 102B increases. For example, the first image generation unit 23 sets the transmittance of the pixels included in the background area 103LB to 0% at the portion in contact with the roll angle display area 102B, and 62% at the left end 105B of the first image 100B. Set.

The background area 103RB is a right background area arranged to the right of the roll angle display area 102B. The color of the background area 103RB is the same as the color of the roll angle display area 102B. The first image generation unit 23 gradually changes the pixel values of the pixels included in the background area 103RB as the distance from the roll angle display area 102B increases. For example, the first image generation unit 23 sets the transmittance of the pixels of the background area 103RB to 0% at the portion in contact with the roll angle display area 102B, and sets it to 100% at the right end 106B of the first image 100B. ..

The second image 200B according to the present embodiment will be described with reference to FIG. 7. The second image generation unit 24 generates the second image 200B according to the present embodiment. The second image 200B has a second frame area 201B, a pitch angle display area 202B, background areas 203TB and 203BB, and a second reference line 204B.

The pitch angle display area 202B is arranged inside the second frame area 201B. The pitch angle display area 202B has a constant length in the vertical direction. The vertical length of the pitch angle display area 202B is based on the length from the second reference line 204B to the upper end 205B of the second image 200B and the length from the second reference line 204B to the lower end 206B of the second image 200B. short.

The pitch angle display area 202B indicates the pitch angle 22B of the vehicle 1 depending on the position where it is arranged. Specifically, the pitch angle display area 202B indicates the direction and angle of the vertical inclination of the vehicle 1 depending on the position where the vehicle 1 is arranged.

When the pitch angle 22B is negative, the second image generation unit 24 arranges the pitch angle display area 202B below the second reference line 204B. Further, the second image generation unit 24 increases the distance between the pitch angle display area 202B and the second reference line 204B as the absolute value of the pitch angle 22B increases. When the pitch angle 22B is positive, the second image generation unit 24 arranges the pitch angle display area 202B above the second reference line 204B. Further, the second image generation unit 24 increases the distance between the pitch angle display area 202B and the second reference line 204B as the absolute value of the pitch angle 22B increases. When the pitch angle 22B is 0 degrees, the second image generation unit 24 arranges the pitch angle display area 202B so as to overlap the second reference line 204B.

In the description of FIG. 7, the pitch angle 22B of the vehicle 1 is 10 degrees. Therefore, the pitch angle display area 202B is above the second reference line 204B and is arranged at a position corresponding to 10 degrees.

The color of the pitch angle display area 202B will be described. The color of the pitch angle display area 202B is, for example, blue. The color of the pitch angle display area 202B does not change depending on the pitch angle 22B.

The background area 203TB is an upper background area arranged above the pitch angle display area 202B. The second image generation unit 24 gradually changes the pixel values of the pixels of the background region 203TB as the distance from the pitch angle display region 202B increases. For example, the second image generation unit 24 sets the transmittance of the pixels in the background region 203TB to 0% at the portion in contact with the pitch angle display region 202B and 62% at the upper end 205B of the second image 200B.

The background area 23BB is a lower background area arranged below the pitch angle display area 202. The second image generation unit 24 gradually changes the pixel values of the pixels of the background region 203BB as the distance from the pitch angle display region 202B increases. For example, the second image generation unit 24 sets the transmittance of the pixels in the background region 203BB to 0% at the portion in contact with the pitch angle display region 202B, and sets it to 100% at the lower end 206B of the second image 200B.

In the present embodiment, the first image generation unit 23 gradually changes the transmittance of the background regions 103LB and 103RB as the distance from the roll angle display region 102B increases. Further, the second image generation unit 24 gradually changes the transmittance of the background regions 203TB and 203BB as the distance from the pitch angle display region 202B increases. Therefore, the roll angle display area 102B showing the roll angle 22A is more conspicuous than the background area 103LB and the background area 103RB, and the pitch angle display area 202B showing the pitch angle 22B is more conspicuous than the background area 203TB and the background area 23BB. As a result, the image processing device 20 according to the present embodiment can easily make the passenger of the vehicle 1 understand the inclination angle of the vehicle 1.

In the present embodiment, the first image generation unit 23 keeps the length of the roll angle display area 102B constant, and arranges the roll angle display area 102B at a position based on the roll angle 22A. Further, the second image generation unit 24 keeps the length of the pitch angle display area 202B constant, and arranges the pitch angle display area 202B at a position based on the pitch angle 22B. Therefore, the roll angle display area 102B functions as a cursor indicating the roll angle 22A, and the pitch angle display area 202B functions as a cursor indicating the pitch angle 22B. Therefore, the image processing device 20 according to the present embodiment can easily make the passenger of the vehicle 1 understand the inclination angle of the vehicle 1.

(Embodiment 3)
FIG. 8 shows a composite image 25C generated by the image processing apparatus according to the third embodiment of the present invention. The configuration of the image processing device according to the present embodiment is the same as the configuration of the image processing device 20 according to the first embodiment. Therefore, the image processing device according to the present embodiment is described as "image processing device 20", and detailed description of its configuration will be omitted.

The composite image 25C generated by the image processing apparatus 20 according to the present embodiment is the same as the composite image 25A according to the first embodiment except for the first image 100C and the second image 200C. Therefore, in the description of the present embodiment, the description of the parts other than the first image 100C and the second image 200C will be omitted.

The first image 100C has a first frame area 101C, a roll angle display area 102C, background areas 103LC and 103RC, and a first reference line 104C.

The position and length of the roll angle display area 102C will be described. In the present embodiment, the first image generation unit 23 changes the lateral length of the roll angle display area 102C based on the absolute value of the roll angle 22. The details will be described below.

The first image generation unit 23 determines whether to extend the roll angle display area 102C from the first reference line 104C to the right or to the left based on the positive or negative of the roll angle 22A. When the roll angle 22A is positive, the first image generation unit 23 aligns the left side of the roll angle display area 102C with the first reference line 104C, and sets the roll angle display area 102C to the right of the first reference line 104C. Decide to extend. When the roll angle 22A is negative, the first image generation unit 23 aligns the right side of the roll angle display area 102C with the first reference line 104C, and sets the roll angle display area 102C to the left of the first reference line 104C. Decide to extend. When the roll angle 22A is 0 degrees, the first image generation unit 23 determines that the roll angle display area 102C is not arranged inside the first frame area 101C.

The first image generation unit 23 determines that the larger the absolute value of the roll angle 22A, the larger the size of the roll angle display area 102C in the lateral direction. In FIG. 8, the length D11 corresponds to the lateral size when the absolute value of the roll angle 22A is 20 degrees. The length D12 corresponds to the lateral size when the absolute value of the roll angle 22A is 15 degrees. The length D13 corresponds to the lateral size when the absolute value of the roll angle 22A is 10 degrees. The length D11 is longer than the length D12. The length D12 is longer than the length D13.

The color of the roll angle display area 102C will be described. The color of the roll angle display area 102C is, for example, blue, and does not change depending on the roll angle 22A.

The background area 103LC is a left background area arranged to the left of the roll angle display area 102C. Further, the background area 103RC is a right background area arranged to the right of the roll angle display area 102C. The colors of the background areas 103LC and 103RC are different from the colors of the roll angle display area 102C. The colors of the background regions 103LC and 103RC are constant regardless of the roll angle 22A, and are, for example, white.

The position and length of the pitch angle display area 202C will be described. In the second image 200C, the second image generation unit 24 sets the vertical length of the pitch angle display area 202C with respect to the second reference line 204C to a length based on the absolute value of the pitch angle 22B. The second image generation unit 24 determines whether to extend the pitch angle display area 202C upward or downward from the second reference line 204C based on the positive or negative of the pitch angle 22B. When the pitch angle 22B is positive, the second image generation unit 24 makes the lower side of the pitch angle display area 202C coincide with the second reference line 204C, and extends the pitch angle display area 202C above the second reference line 204C. To decide. When the pitch angle 22B is negative, the second image generation unit 24 makes the upper side of the pitch angle display area 202C coincide with the second reference line 204C, and extends the pitch angle display area 202C below the second reference line 204C. To decide. When the pitch angle 22B is 0 degrees, the second image generation unit 24 determines that the pitch angle display area 202C is not arranged inside the second frame area 201C.

The second image generation unit 24 determines that the larger the absolute value of the pitch angle 22B, the larger the size of the pitch angle display area 202C in the vertical direction. In FIG. 8, the length D21 corresponds to the size in the vertical direction when the absolute value of the pitch angle 23A is 20 degrees. The length D22 corresponds to the size in the vertical direction when the absolute value of the pitch angle 23A is 15 degrees. The length D23 corresponds to the size in the vertical direction when the absolute value of the pitch angle 22B is 10 degrees. The length D21 is longer than the length D22. The length D22 is longer than the length D23.

The background area 203TC is an upper background area arranged above the pitch angle display area 202C. Further, the background area 203BC is a lower background area arranged below the pitch angle display area 202C. The colors of the background areas 203TC and 203BC are different from the colors of the pitch angle display area 202C. The colors of the background regions 203TC and 203BC are constant regardless of the pitch angle 22B, and are, for example, white.

The background area 103LC is a left background area arranged to the left of the roll angle display area 102C. Further, the background area 103RC is a right background area arranged to the right of the roll angle display area 102C. The colors of the background areas 103LC and 103RC are different from the colors of the roll angle display area 102C. The colors of the background regions 103LC and 103RC are constant regardless of the roll angle, for example, white.

As described above, in the present embodiment, the first image generation unit 23 extends the roll angle display area 102C from the first reference line 104C in the direction based on the roll angle 22A, and lengthens the roll angle display area 102C. Is set to a length based on the absolute value of the roll angle 22A. Further, the second image generation unit 24 extends the pitch angle display area 202C from the second reference line 204C in a direction based on the pitch angle 22B, and sets the length of the pitch angle display area 202C based on the absolute value of the pitch angle 22B. Set the length. Therefore, the passenger of the vehicle 1 can easily understand the roll angle 22A and the pitch angle 22B from the extending direction and length of the roll angle display area 102C and the pitch angle display area 202C. Therefore, the image processing device 20 according to the present embodiment can easily make the passenger of the vehicle 1 understand the inclination angle of the vehicle 1.

(Embodiment 4)
FIG. 9 shows a composite image 25D generated by the image processing apparatus according to the fourth embodiment of the present invention. The configuration of the image processing device according to the present embodiment is the same as the configuration of the image processing device 20 according to the first embodiment. Therefore, the image processing device according to the present embodiment is described as "image processing device 20", and detailed description of its configuration will be omitted.

The composite image 25D has a camera image 11B, a first image 100D, and a second image 200D.

The first image 100D has a first frame area 101D, a roll angle display area 102D, background areas 103LD and 103RD, a first reference line 104D, and a first character area 107D. The first frame area 101D, the roll angle display area 102D, and the first reference line 104D are the same as the first frame area 101B, the roll angle display area 102B, and the first reference line 104B shown in FIG. Therefore, the description of the first image 100D excluding the background areas 103LD and 103RD and the first character area 107D will be omitted.

The second image 200D has a second frame area 201D, a pitch angle display area 202D, background areas 203TD and 203BD, a second reference line 204D, and a second character area 207D. The second frame area 201D, the pitch angle display area 202D, and the second reference line 204D are the same as the second frame area 201B, the pitch angle display area 202B, and the second reference line 204B shown in FIG. 7. Therefore, the description of the second image 200D excluding the background areas 203TD and 203BD and the second character area 207D will be omitted.

In the description of FIG. 9, the roll angle 22A will be described as −10 degrees, and the pitch angle 22B will be described as 10 degrees.

The first image generation unit 23 arranges the first character region 107D having characters representing the received roll angle 22A on the outside of the first frame region 101D and near the first frame region 101D. The shape of the first character region 107D is arbitrary. Since the roll angle 22A in the description of FIG. 9 is −10 degrees, the first image generation unit 23 arranges the numerical value “-10” in the first character area 107D.

The transmittance and color of the pixels of the background regions 103LD and 103RD are constant regardless of both the roll angle and the distance from the roll angle display region 102D.

The second image generation unit 24 arranges the second character region 207D having the characters representing the received pitch angle 22B on the outside of the second frame region 201D and near the second frame region 201D. The shape of the second character region 207D is arbitrary. The pitch angle 22B in the description of FIG. 9 is 10 degrees. Therefore, the second image generation unit 24 arranges the numerical value "10" in the second character area 207.

The colors of the background areas 103LD and 103RD are constant regardless of both the roll angle and the distance from the roll angle display area 102D.

In the present embodiment, the first image generation unit 23 arranges the first character area 107D indicating the value of the roll angle 22A in the vicinity of the roll angle display area 102D. Further, the second image generation unit 24 arranges the second character area 207 showing the value of the pitch angle 22B in the vicinity of the pitch angle display area 202D. Therefore, the passenger of the vehicle 1 can grasp the values of the roll angle 22A and the pitch angle 22B in characters. As a result, the image processing device 20 according to the present embodiment allows the passenger of the vehicle 1 to easily understand the inclination angle of the vehicle 1.

Further, the image processing apparatus 20 according to the present embodiment simultaneously uses the roll angle display area 102D and the first character area 107D to represent the roll angle 22A, and simultaneously uses the pitch angle display area 202D and the second character area 207 to pitch. Represents the angle 22B. Therefore, the image processing device 20 according to the present embodiment makes the passenger of the vehicle 1 more easily understand the inclination angle of the vehicle 1 as compared with the case where the roll angle display area 102D and the pitch angle display area 202D are used. be able to.

(Embodiment 5)
FIG. 10 shows a composite image 25E generated by the image processing apparatus according to the fifth embodiment of the present invention. The configuration of the image processing device according to the present embodiment is the same as the configuration of the image processing device 20 according to the first embodiment. Therefore, the image processing device according to the present embodiment is described as "image processing device 20", and detailed description of its configuration will be omitted.

The composite image 25E has a first image 100E and a second image 200A. The composite image 25E includes the first image 100E in place of the first image 100A included in the composite image 25A. Therefore, in the present embodiment, the description other than the first image 100E will be omitted.

The first image generation unit 23 rotates the first image 100E having the first predetermined shape at an angle corresponding to the roll angle 22A. Further, a part of the first image 100E is not included in the composite image 25E. The details will be described below.

The first image generation unit 23 first generates a bar-shaped first image 100E extending in the lateral direction of the camera image 11B. The initial position of the first image is a position where the upper side of the first image 100 is in contact with the upper side of the camera image 100B. The first image generation unit 23 arranges the generated first image 100E at a position where the first image 100A shown in FIG. 4 is arranged. The first image 100E has a first frame area 101E, a roll angle display area 102E, and a first reference line 104E. The range of the roll angle display area 102E coincides with the range of the first frame area 101E. At the time of generating the first image 100E, the position of the first reference line 104E is the same as the position of the first reference line 101A shown in FIG. The color of the roll angle display area 102E is constant regardless of the roll angle 22A. The color of the roll angle display area 102E is, for example, yellow.

The first image generation unit 23 rotates the generated first image 100E with the point 108E as the center of rotation according to the roll angle 22A. Point 108E is any point above the first reference line 104E. In the example shown in FIG. 10, the point 108E is the upper end of the first reference line 104E.

The first image 100E is divided into a display area 100E1 and a non-display area 100E2 by being rotated. The display area 100E1 is an area of the first image 100E that is arranged on the camera image 11B. The non-display area 100E2 is an area of the first image 100E that is arranged above the camera image 11B. In the first image 100E, the ratio indicated by each of the display area 100E1 and the non-display area 100E2 is determined by the rotation angle of the first image 100E. As shown in FIG. 10, the composite image 25E includes the display area 100E1, but does not include the non-display area 100E2.

In the present embodiment, the inclination of the first image 100E changes according to the roll angle 22E. Therefore, the passenger of the vehicle 1 can easily understand the inclination angle of the vehicle 1 from the inclination of the first image 100E. Further, the composite image 25E includes the display area 100E1 of the first image 100E, but does not include the non-display area 100E2. Therefore, the passenger of the vehicle 1 can easily understand that the composite image 25E does not include the non-display area 100E2. Therefore, the image processing device 20 according to the present embodiment can easily make the passenger of the vehicle 1 understand the inclination angle of the vehicle 1.

(Embodiment 6)
FIG. 11 shows a functional block diagram showing the configuration of the image processing device 20F according to the present embodiment.

<Configuration of image processing device 20F>
As shown in FIG. 11, the image processing device 20F includes an image acquisition unit 21, a tilt calculation unit 22, an image correction unit 28, a tilt image generation unit 26F, and an image composition unit 25. The tilt image generation unit 26F includes a first image generation unit 23 and a pitch angle image generation unit 27. The image processing device 20F has a configuration in which the second image generation unit 24 provided in the image processing device 20 according to the first embodiment is replaced with a pitch angle image generation unit 27. Therefore, the description of the configuration other than the pitch angle image generation unit 27 will be omitted.

The pitch angle image generation unit 27 receives the pitch angle 22B from the inclination calculation unit 22. The pitch angle image generation unit 27 generates a pitch angle display image 300F by using the received pitch angle 22B. The pitch angle display image 300F shows the pitch angle of the vehicle 1. The pitch angle display image 300F will be described later.

The image composition unit 25 receives the camera image 11B from the image correction unit 28, the first image 100F from the first image generation unit 23, and the pitch angle display image 300F from the pitch angle image generation unit 27. The image synthesizing unit 25 generates a composite image 25F by using the camera image 11B, the first image 100F, and the pitch angle display image 300F.

<Composite image 25F>
FIG. 12 shows an example of the composite image 25F generated by the image synthesis unit 25F. The composite image 25F includes a camera image 11B, a first image 100F, and a pitch angle display image 300F. The first image 100F shown in FIG. 12 is the same as the first image 100A shown in FIG. Therefore, the description of the first image 100F in FIG. 12 will be omitted.

In the description of FIG. 12, the pitch angle 22B at the time T1 is 10 degrees, and the pitch angle 22B at the time T2 is 0 degrees.

The pitch angle display image 300F has a front end surface image 301F and a rear end surface image 302F. The front end surface image 301F corresponds to the front end surface of the vehicle 1. Further, the rear end surface image 302F corresponds to the rear end surface of the vehicle 1. The rear end surface image 302F is fixed in the pitch angle display image 300F. When the vehicle 1 is viewed from the rear, the rear end surface of the vehicle 1 looks larger than the front end surface. Therefore, in the front end face image 301F and the rear end face image 302F, the rear end face image 302F is made larger than the front end face image 301F so that the passenger can intuitively understand which is the front end face image.

The pitch angle image generation unit 27 receives the pitch angle 22B from the inclination calculation unit 22. The pitch angle image generation unit 27 determines the relative positions of the front end surface image 301F and the rear end surface image 302F in the vertical direction based on the received pitch angle 22B. Specifically, in the pitch angle image generation unit 27, as the absolute value of the pitch angle 22B increases, the distance between the vertical center position of the front end surface image 301F and the vertical center position of the rear end surface image 302F increases. Determine to be longer in the vertical direction. The pitch angle image generation unit 27 determines whether the front end face image 301F is located above or below the rear end face image 302F based on the pitch angle 22B.

In the present embodiment, the rear end face image 302F is fixed and the front end face image 301F is moved with reference to the position of the rear end face image 302F, but the front end face image 301F is fixed and the rear end face image 302F is moved as the front end face image. It may be moved with reference to 301F.

When the pitch angle 22B is positive, the pitch angle image generation unit 27 determines that the front end surface image 301F is located above the rear end surface image 302F. When the pitch angle 22B is negative, the pitch angle image generation unit 27 determines that the front end surface image 301F is located below the rear end surface image 302F.

The pitch angle image generation unit 27 arranges the rear end surface image 302F at a position predetermined in the pitch angle display image 300F. The predetermined position is the center in the vertical direction in the pitch angle display image 300F. The pitch angle image generation unit 27 arranges the rear end surface image 302F so that the vertical center of the rear end surface image 302F is located at the center position in the vertical direction of the pitch angle display image 300F.

The pitch angle image generation unit 27 arranges the front end surface image 301F on the pitch angle display image 300F based on the determined relative position. As a result, the pitch angle display image 300F is generated. The pitch angle image generation unit 27 supplies the generated pitch angle display image 300F to the image composition unit 25.

The image synthesizing unit 25 arranges the received pitch angle display image 300F at the right end of the camera image 11B to generate the composite image 25F. The image synthesizing unit 25 arranges the pitch angle display image 300F at a position in the composite image 25F where the vertical center of the pitch angle display image 300F coincides with the vertical center of the composite image 25F.

In the description of FIG. 12, the pitch angle 22B at time T1 is 10 degrees, and the pitch angle 22B at time T2 is 0 degrees. Therefore, in the composite image 25F, the front end face image 301F at time T1 is arranged above the rear end face image 302F. Further, the front end face image 301F at time T2 is arranged so as to overlap the rear end face image 302F.

In the present embodiment, the pitch angle image generation unit 27 changes the relative position between the front end surface image 301F and the rear end surface image 302F based on the pitch angle 22A. Therefore, the passenger of the vehicle 1 can understand the inclination angle of the vehicle 1 in the front-rear direction from the front end surface image 301F and the rear end surface image 302F. The image processing device 20F according to the present embodiment can easily make the passenger of the vehicle 1 understand the inclination angle of the vehicle 1.

(Modification example 1)
In the above embodiment, in the pitch angle display image 300F arranged at the right end portion of the camera image 11B, an example in which the position of the rear end surface image 302F is fixed and the front end surface image 301F is moved has been described. Not limited to. Both the front end face image and the rear end face image may be relatively moved. For example, as in the pitch angle display image 300G shown in FIG. 13, the front end face image 301G and the rear end face image 302G may move relatively. The pitch angle display image 300G shown in FIG. 13 shows that the pitch angle 22B at time T1 is 15 degrees and the pitch angle 22B at time T2 is 0 degrees.

In FIG. 13, the image synthesizing unit 25 displays the pitch angle display image 300G so that the positions of the vertical center of the pitch angle display image 300G and the vertical center of the camera image 11B match in the composite image 25G. Deploy. Further, the pitch angle image generation unit 27 arranges the front end surface image 301G and the rear end surface image 302G symmetrically with respect to the center in the vertical direction of the pitch angle display image 300G. Since the pitch angle 22B at time T1 is 15 degrees, the pitch angle image generation unit 27 arranges the front end surface image 301G and the rear end surface image 302G at the positions when the pitch angle 22B is 15 degrees. Further, since the pitch angle 22B at time T2 is 0 degrees, the pitch angle image generation unit 27 arranges the front end surface image 301G and the rear end surface image 302G in the center of the pitch angle display image 300G in the vertical direction.

In this modification, the image processing device 20F displays the pitch angle 22B by using the front end face image 301G and the rear end face image 302G whose relative positions change according to the pitch angle 22B. Therefore, in this modification, the passenger of the vehicle 1 can easily understand the inclination angle of the vehicle 1.

(Modification 2)
In the above embodiment, an example in which the pitch angle display image 300F is arranged at the right end portion of the camera image 11B and the first image 100F is arranged at the upper end portion of the camera image 11B has been described, but the present invention is not limited thereto. As shown in FIG. 13, the pitch angle display image 300G may be arranged at the right end of the camera image 11B, and the roll angle display image 400G may be arranged at the upper end of the camera image 11B. In this case, the roll angle display image 400G includes a left end face image 401G showing the roll angle of the left end face which is the left end face of the vehicle 1 and a right end face showing the roll angle of the right end face which is the right end face of the vehicle 1. It has an image 402G.

The first image generation unit 23 determines the lateral position of the right end surface image 402G at the center of the roll angle display image 400G in the lateral direction. Based on the roll angle 22A, the first image generation unit 23 determines whether the left end surface image 401G is located to the left or to the right with respect to the right end surface image 402G. When the roll angle 22A is positive, the first image generation unit 23 determines that the left end face image 401G is located to the right with respect to the right end face image 402G. When the roll angle 22A is negative, the first image generation unit 23 determines that the left end face image 401G is located to the left with respect to the right end face image 402G.

The first image generation unit 23 generates a roll angle display image 400G by arranging the right end face image 402G and the left end face image 401G at the determined positions. The image compositing unit 25 arranges the roll angle display image 400G and the camera image 11B so that the horizontal center of the roll angle display image 400G and the horizontal center of the camera image 11B coincide with each other, and the composite image Generate 25G.

In this modification, the roll angle 22A is displayed by using the right end face image 402G and the left end face image 401G whose relative positions change according to the roll angle 22A. Therefore, in this modification, the passenger of the vehicle 1 can easily understand the inclination angle of the vehicle 1.

(Modification 3)
In the above embodiment, a bar-shaped roll angle display area 102A is arranged at the upper end of the camera image 11B, and a bar-shaped pitch angle display area 202A is arranged at the right end of the camera image 11B to generate a composite image 25A. However, the present invention is not limited to this. As shown in FIG. 14, a triangular pitch angle display area 202H may be arranged at the right end of the camera image 11B to generate a composite image 25H. When shown in FIG. 14, the pitch angle display area 202H represents the inclination of the vehicle 1 in the front-rear direction. In this case, the vehicle 1 is tilted forward and the pitch angle 22B is positive. That is, the vehicle 1 is tilted toward the acute angle of the pitch angle display area 202H.

In this modification, the pitch angle display area 202H having an acute angle is used to represent the pitch angle 22B. Therefore, in this modification, the passenger of the vehicle 1 can easily understand the inclination angle of the vehicle 1.

(Modification example 4)
In the above embodiment, an example will be described in which a bar-shaped first image 100A is arranged at the upper end of the camera image 11B and a bar-shaped second image 200A is arranged at the right end of the camera image 11B to generate a composite image 25A. However, the present invention is not limited to this. As shown in FIG. 15, a bar-shaped image may be displayed on all of the upper end portion, the right end portion, the lower end portion, and the left end portion of the camera image 11B.

In the case shown in FIG. 15, a bar-shaped first image 100J is placed at the upper end of the camera image 11B, a bar-shaped second image 200J is placed at the right end of the camera image 11B, and a bar is placed at the lower end of the camera image 11B. A third image 500J having a shape is arranged, and a fourth image 600J having a bar shape is arranged at the left end of the camera image 11B to generate a composite image 25J. In this case, for example, the image in which the vehicle 1 is arranged in the tilted direction is filled with a color corresponding to the tilt, and the color of the image in the other directions is transparent or white.

In this modification, the inclination angle of the vehicle 1 is represented by using the first image 100J, the second image 200J, the third image 500J, and the fourth image 600J. Therefore, in this modification, the passenger of the vehicle 1 can easily understand the inclination angle of the vehicle 1.

(Modification 5)
In the above embodiment, an example has been described in which the transmittance is used as the pixel value of the background area to be changed and the transmittance of the background area is gradually changed as the distance from the roll angle display area 102B or the pitch angle display area 202B is increased. The invention is not limited to this. Color may be used as the pixel value of the background area to be changed. In this case, the color of the left background area and the color of the right background area are gradually changed as the distance from the roll angle display area increases. Further, the color of the upper background area and the color of the lower background area are gradually changed as the distance from the pitch angle display area increases. Further, the brightness may be used as the pixel value of the background region to be changed.

(Modification 6)
In the above embodiment, the image processing device 20 has described an example in which the camera image 11B is acquired from the camera 11 mounted on the vehicle 1 and the generated composite image is displayed on the display 13 mounted on the vehicle 1. The invention is not limited to this. The image processing device 20 may acquire a camera image from a camera of a device that can be carried out from the vehicle 1 such as a smartphone, and display the composite image on the display of the device such as a smartphone. Further, the image processing device 20 acquires a camera image from a camera of a device that can be carried out from the vehicle 1 such as a smartphone, and displays a composite image on both the display of the device such as a smartphone and the display 13 mounted on the vehicle 1. It may be displayed.

In this modification, the image processing device 20 acquires an image from a camera of a device such as a smartphone, and displays a composite image on a display of the device such as a smartphone. Therefore, the composite image can be displayed on the familiar screen of the passenger. Therefore, the passenger of the vehicle 1 can easily understand the inclination angle of the vehicle 1.

Further, in the present modification, the image processing device 20 causes the composite image to be displayed on both the display of the device such as a smartphone and the display 13 mounted on the vehicle 1. Therefore, the image processing device 20 can display the composite image at a position that is easy for the passenger to see, as compared with the case where the composite image is displayed only on the display 13 mounted on the vehicle 1.

(Modification 7)
In the above embodiment, the first character area 107D is arranged outside the first frame area 101D and near the first frame area 101D, and the second character area 207D is outside the second frame area 201D and in the second frame area 201D. Although the example of arranging them close to each other has been described, the present invention is not limited to this. The first character area 107D may be arranged inside the first frame area 101D. Further, the second frame area 201D may be arranged inside the second frame area 201D.

(Modification 8)
In the above embodiment, an example in which the composite image has a bar-shaped first image and a bar-shaped second image has been described, but the present invention is not limited thereto. The shapes of the first image and the second image of the composite image are arbitrary. For example, as shown in FIG. 16, the composite image 25K includes a first image including the left end face image 401K of the triangle and the right end face image 402K of the triangle, and a first image including the front end face image 301K of the triangle and the rear end face image 302K of the triangle. It may have two images. Further, the first image may include the first character region 107K, and the second image may include the second character region 207K.

In the case shown in FIG. 16, the first image generation unit 23 colors the left end face image 401K or the right end face image 402K according to the code of the roll angle 22A. For example, when the roll angle 22A is −10 degrees, the first image generation unit 23 colors the left end face image 401K. The color at the time of coloring is a color corresponding to the absolute value of the roll angle 22A. The first image generation unit 23 is colored yellow when the absolute value of the roll angle 22A is greater than 0 degrees and less than 10 degrees, and orange when the absolute value of the roll angle 22A is 10 degrees or more and less than 20 degrees. When the absolute value of the roll angle 22A is 20 degrees or more, it is colored red. The first image generation unit 23 sets the color of the uncolored side of the left end face image 401K or the right end face image 402K to white.

Further, the second image generation unit 24 colors the front end face image 301K or the rear end face image 302K according to the code of the pitch angle 22B. For example, when the pitch angle 22B is 10 degrees, the second image generation unit 24 colors the upper end surface image 301K. The color at the time of coloring is a color corresponding to the absolute value of the pitch angle 22B. The second image generation unit 24 is colored yellow when the absolute value of the pitch angle 22B is greater than 0 degrees and less than 10 degrees, and orange when the absolute value of the pitch angle 22B is 10 degrees or more and less than 20 degrees. When the absolute value of the pitch angle 22B is 20 degrees or more, it is colored red. The second image generation unit 24 sets the color of the uncolored side of the front end face image 301K or the rear end face image 302K to white.

The first image generation unit 23 arranges the first character region 107K in the vicinity of the colored side of the left end face image 401K or the right end face image 402K. The first character area 107K has a character indicating a roll angle 22A.

The second image generation unit 24 arranges the second character region 207K in the vicinity of the colored side of the front end face image 301K or the rear end face image 302K. The second character region 207K has characters indicating a pitch angle 22B.

In this modification, the left end face image 401K of the triangle, the right end face image 402K of the triangle, the front end face image 301K of the triangle, the rear end face image 302K of the triangle, the first character area 107K, and the second character area 207K are It is used to represent the tilt angle of the vehicle 1. Therefore, in this modification, the passenger of the vehicle 1 can easily understand the inclination angle of the vehicle 1.

(Modification 9)
Further, in the above embodiment, the image processing apparatus 20 may be individually integrated into one chip by a semiconductor device such as an LSI (Large Scale Integration), or may be integrated into one chip so as to include a part or all of the image processing apparatus 20. Good. Although it is referred to as an LSI here, it may be referred to as an IC (Integrated Circuit), a system LSI, a super LSI, or an ultra LSI depending on the degree of integration.

The method of making an integrated circuit is not limited to LSI, and may be realized by a dedicated circuit or a general-purpose processor. An FPGA (Field Programmable Gate Array) that can be programmed after the LSI is manufactured, or a reconfigurable processor that can reconfigure the connection and settings of the circuit cells inside the LSI may be used.

Further, a part or all of the processing executed by the image processing apparatus 20 may be realized by a program. Then, a part or all of the processing of each functional block of each of the above embodiments is performed by the central processing unit (CPU) in the computer. Further, the program for performing each process is stored in a storage device such as a hard disk or a ROM, and is read and executed in the ROM or the RAM.

Further, each process of the above embodiment may be realized by hardware, or may be realized by software (including a case where it is realized together with an OS (operating system), middleware, or a predetermined library). .. Further, it may be realized by mixed processing of software and hardware.

For example, when the image processing device 20 is realized by software, the hardware configuration shown in FIG. 17 (for example, a hardware configuration in which a CPU, ROM, RAM, input unit, output unit, etc. are connected by a bus Bus) is used. , Each functional part may be realized by software processing.

Further, the execution order of the processing methods in the above-described embodiment is not limited to the description of the above-described embodiment, and the execution order may be changed without departing from the gist of the invention.

A computer program that causes a computer to perform the above-mentioned method and a computer-readable recording medium that records the program are included in the scope of the present invention. Here, examples of computer-readable recording media include flexible disks, hard disks, CD-ROMs, MOs, DVDs, DVD-ROMs, DVD-RAMs, large-capacity DVDs, next-generation DVDs, and semiconductor memories. ..

Although the embodiment of the present invention has been described above, the above-described embodiment is merely an example for carrying out the present invention. Therefore, the present invention is not limited to the above-described embodiment, and the above-described embodiment can be appropriately modified and implemented within a range that does not deviate from the gist thereof.

11: Camera 12: Acceleration sensor 12A: Acceleration information 13: Display 20: Image processing device 21: Image acquisition unit 22: Tilt calculation unit 22A: Roll angle 22B: Pitch angle 23: First image generation unit 24: Second image generation Unit 25: Image composition unit 25A: Composite image 26A: Tilt image generation unit 27: Pitch angle image generation unit 28: Image correction unit

Claims (11)

A tilt calculation unit that calculates the roll angle of the vehicle and the pitch angle of the vehicle from the output of the sensor that detects the posture information of the vehicle.
An image acquisition unit that acquires camera images generated by the camera,
A first image generation unit that generates a first image having a first predetermined shape having a display mode corresponding to the roll angle.
A second image generation unit that generates a second image having a second predetermined shape having a display mode corresponding to the pitch angle.
An image in which the first image is arranged at the upper end or the lower end of the camera image acquired by the image acquisition unit, and the second image is arranged at the left end or the right end of the camera image to generate a composite image. Synthetic part and
To prepare
Image processing device. The first image is
The first frame area corresponding to the first predetermined shape and
A roll angle display area arranged inside the first frame area and indicating the roll angle,
The first background area, which is arranged inside the first frame area and is different from the roll angle display area, is included.
The first image generation unit determines the position of the roll angle display area based on the roll angle, and determines the pixel value in the first background area to be a pixel value different from the pixel value in the roll angle display area.
The second image is
The second frame area corresponding to the second predetermined shape and
A pitch angle display area arranged inside the second frame area and showing the pitch angle, and a pitch angle display area.
A second background area, which is arranged inside the second frame area and is different from the pitch angle display area, is included.
The second image generation unit determines the position of the pitch angle display region based on the pitch angle, and determines the pixel value in the second background region to be a pixel value different from the pixel value in the pitch angle display region.
The image processing apparatus according to claim 1. The first image generation unit determines the color of the roll angle display area based on the roll angle, and determines the color of the roll angle display area.
The second image generation unit determines the color of the pitch angle display region based on the pitch angle.
The image processing apparatus according to claim 2. The first image is generated by extending in the lateral direction.
The first image generation unit determines, based on the roll angle, whether or not to arrange the roll angle display area to the right of the reference line indicating the roll angle of 0 degrees.
The second image is formed so as to extend in the vertical direction.
The second image generation unit determines, based on the pitch angle, whether or not to arrange the pitch angle display region above the reference line at which the pitch angle indicates 0 degrees.
The image processing apparatus according to claim 3. The first image is generated by extending in the lateral direction.
The first background area is
A left background area arranged to the left of the roll angle display area and
A right background area arranged to the right of the roll angle display area and
Including
The first image generation unit gradually changes the pixel values of the left background area and the right background area as the distance from the roll angle display area increases.
The second image is generated by extending in the vertical direction.
The second background area is
An upper background area arranged above the pitch angle display area and
A lower background area arranged below the pitch angle display area and
Including
The second image generation unit gradually changes the pixel values of the upper background region and the right background region as the distance from the pitch angle display region increases.
The image processing apparatus according to claim 2. The first image is generated by extending in the lateral direction.
The first image generation unit determines the length of the roll angle display area in the lateral direction based on the roll angle, and extends the roll angle display area to the right from a reference line indicating a roll angle of 0 degrees. Based on the roll angle, it is decided whether to arrange the image on the roll or to extend it to the left.
The second image is formed so as to extend in the vertical direction.
The second image generation unit determines the length of the pitch angle display area in the vertical direction based on the pitch angle, and extends the pitch angle display area upward from the reference line showing the pitch angle of 0 degrees. Whether to arrange or to arrange so as to extend downward is determined based on the pitch angle.
The image processing apparatus according to claim 2. The roll angle display area has a certain length and is arranged at a position based on the roll angle.
The pitch angle display area has a constant length and is arranged at a position based on the pitch angle.
The image processing apparatus according to claim 2. The first image is generated by extending in the lateral direction.
The first image generation unit rotates the first image extending in the lateral direction at an angle corresponding to the roll angle.
The image processing apparatus according to claim 1. The first image generation unit arranges characters representing the roll angle in the vicinity of the first frame area.
The second image generation unit arranges characters representing the pitch angle in the vicinity of the second frame region.
The image processing apparatus according to any one of claims 2 to 8. A tilt calculation unit that calculates the pitch angle of the vehicle from the output of the sensor that detects the posture information of the vehicle,
An image acquisition unit that acquires camera images generated by the camera,
The relative positions of the front end surface image corresponding to the front end surface of the vehicle and the rear end surface image corresponding to the rear end surface of the vehicle are determined based on the pitch angle, and the front end surface image and the rear end surface images arranged at the determined relative positions are determined. A pitch angle image generator that generates a pitch angle display image including an end face image,
An image synthesizing unit that generates a composite image by synthesizing the camera image acquired by the image acquisition unit and the pitch angle display image.
To prepare
Image processing device. A tilt calculation process for calculating the roll angle of the vehicle and the pitch angle of the vehicle from the output of the sensor that detects the posture information of the vehicle, and
The image acquisition process to acquire the camera image generated by the camera,
A first image generation step of generating a first image having a first predetermined shape having a display mode corresponding to the roll angle, and
A second image generation step of generating a second image having a second predetermined shape having a display mode corresponding to the pitch angle, and
An image in which the first image is arranged at the upper end or the lower end of the camera image acquired by the image acquisition step, and the second image is arranged at the left end or the right end of the camera image to generate a composite image. Synthesis process and
To prepare
Image processing method.

Images