JP2013062692A - Calibration device and method for on-vehicle camera - Google Patents

Abstract

An in-vehicle camera calibration apparatus and method capable of calibrating an in-vehicle camera using a calibration index with a simple operation.
An in-vehicle camera calibration device (100) according to the present invention includes an imaging unit (1) that images a calibration index (8) and an instruction unit that indicates a display position of a calibration marker (14) that is superimposed and displayed. (5) and image signal processing means (3) for performing image processing so that the calibration marker is superimposed and displayed at the designated position. Then, the mounting angle of the in-vehicle camera is calculated based on the display position of the calibration marker (14) when the calibration marker (14) is displayed so as to overlap two points of the calibration index (8). Perform calibration.
[Selection] Figure 1

Description

  The present invention relates to a technical field of an in-vehicle camera calibration apparatus and method for performing calibration by calculating an attachment angle of an in-vehicle camera to a vehicle body by imaging a calibration index.

  In recent years, in-vehicle cameras are sometimes attached to various types of vehicles such as automobiles in order to assist driving by assisting drivers to ensure visibility. The captured image of the in-vehicle camera is recognized by the driver by being displayed on a display such as a navigation device, for example, and graphic data indicating a predicted travel route is superimposed on the display screen as necessary. For example, when a back camera is mounted to secure the field of view behind the vehicle, graphic data indicating a predicted travel route may be superimposed and displayed in real time based on the steering angle of the steering when the vehicle is moving backward.

  Such a vehicle-mounted camera is attached to the vehicle body by an operator, but generally there are not a few errors in its attachment accuracy. The display position of the graphic data superimposed on the captured image of the in-vehicle camera is easily affected by the mounting accuracy of the in-vehicle camera. If the mounting error is large, the display position of the graphic data will deviate from the original position, and the driver There is a risk of providing incorrect information. Therefore, in this type of in-vehicle camera, the calibration index prepared outside the vehicle is imaged by the in-vehicle camera, and the display position of the calibration index in the captured image is evaluated to calculate the mounting angle of the in-vehicle camera to the vehicle body, and the calibration Is done.

  Conventionally, a three-dimensional object is generally used as a calibration index for calibration of an in-vehicle camera. However, in view of simplicity and workability in a worker who performs calibration and on-site, Patent Document 1 discloses a three-dimensional object as a calibration index. Instead, an example of a calibration method using a two-dimensional index arranged on a plane is disclosed. In Patent Document 1, when a two-dimensional index is imaged with an in-vehicle camera, a window in which an operator can operate the display position is superimposed on the captured image, and the display position of the window is adjusted so that the calibration index enters the window. By doing so, parameters necessary for calibration (roll, pan, tilt angle of the mounted on-vehicle camera) are calculated. The operation of the display position of the window in Patent Document 1 is performed via a user interface. In this user interface, operation buttons are provided to move the window up, down, left and right, and to rotate left and right. The operator can adjust the display position of the window while viewing the display screen. By the operation, the calibration index is adjusted so as to enter the window.

  Here, FIG. 8 is a schematic diagram briefly showing the adjustment method of the display position of the window in Patent Document 1. FIG. 8 shows a display screen 20 in which a window is superimposed on a captured image when the vehicle-mounted camera is attached to the rear bumper of the vehicle as a back camera. In FIG. 8, reference numeral 21 denotes a rear bumper of the vehicle, reference numeral 22 denotes a linear calibration index previously set on a running surface imaged by the in-vehicle camera, and reference numeral 23 denotes a window superimposed on the captured image of the in-vehicle camera. Is shown. Reference numeral 24 denotes a user interface that allows the operator to operate the superimposed display position of the window 23. By operating each button, the operator can move and rotate left and right and left and right. ing.

  FIG. 8A shows an initial state, and the calibration index 22 is displayed outside the window 23. The operator operates the operation buttons 24 sequentially as shown in FIGS. 8B to 8G to move the display position of the window 23 so that the calibration index 21 is within the window 23. 8B and 8C, the display position of the window 23 is moved in the left direction and the upward direction, respectively, to roughly adjust the position of the calibration index 22 and the window 23, and then as shown in FIG. 8D. In addition, the adjustment is made so that the calibration index 22 falls within the window 23 by rotating the window 23 counterclockwise. Here, it is practically difficult to accurately place the calibration index 22 in the window 23 by performing the operations shown in FIGS. (B) to (d) once. As shown in g), it is necessary to repeat the fine adjustment of the display position of the window 23 and to optimize it again by rotating the window 23 after moving it in the vertical and horizontal directions.

JP 2001-245326 A

  In Patent Document 1, adjustment of the display position of the window 23 is accompanied by left and right rotations in addition to up and down, left and right directions, so that the configuration of the user interface for indicating the display position of the window 23 is complicated, and the operation of the operator The burden is also great. Further, as shown in FIG. 8, it is difficult to accurately place the calibration index 22 in the window 23 by a single adjustment operation, and it is necessary to repeat the adjustment several times. There is a problem that the work burden is large.

  The present invention has been made in view of the above problems, and an object of the present invention is to provide an in-vehicle camera calibration apparatus and method capable of performing calibration of an in-vehicle camera using a calibration index with a simple operation.

  In order to solve the above-mentioned problem, the in-vehicle camera calibration apparatus according to the present invention is a calibration apparatus for an in-vehicle camera that performs calibration by calculating the mounting angle of the in-vehicle camera on the vehicle body by imaging a calibration index, An imaging unit that acquires a captured image including a calibration index having two points separated from each other on a plane; and an image signal processing unit that processes an image signal so that a calibration marker is superimposed on the acquired captured image. Display means for indicating the superimposed display position of the calibration marker in the captured image, and the display position of the calibration marker when the calibration marker is displayed so as to overlap two points of the calibration index. The mounting angle of the in-vehicle camera is calculated based on the above.

  According to the on-vehicle camera calibration apparatus according to the present invention, the display position of the calibration marker is easily adjusted so as to overlap the two points indicated by the calibration index without complicated operations such as repeated fine adjustments. The camera mounting angle can be calculated to calibrate the in-vehicle camera. In particular, the display position of the calibration marker can be adjusted by simply moving it in the vertical and horizontal directions without accompanying a rotation operation. Therefore, calibration can be easily performed with less operator burden.

  In addition, in-vehicle cameras such as a back camera use a wide-angle lens with a wide angle of view in order to ensure a wide imaging range. Such a wide-angle lens has a large distortion, and the subject image may be greatly deformed particularly in the peripheral region of the captured image. In such a case, in Patent Document 1, it is assumed that the image of the calibration index 22 included in the captured image is also greatly deformed and cannot be accommodated in the window 23. In that respect, in the present invention, it is only necessary to move the calibration marker so as to overlap the two points indicated by the calibration index. Therefore, even when a wide-angle lens is used in the imaging unit, the calibration marker is placed at the position of the calibration index. It can be moved easily and accurately.

  In the present invention, it is also possible to expect a reduction in the amount of calculation for displaying the calibration marker on the screen in real time during the calibration operation. In other words, in the present invention, the calibration marker is simply moved in parallel on the image in the vertical and horizontal directions during the calibration operation, and the calibration data is calculated together after the position adjustment is completed. Therefore, as in Patent Document 1, special calculation for displaying the calibration marker on the screen in real time is unnecessary, the processing during the calibration operation is reduced, and the time for displaying the calibration marker on the screen is required. Nothing will happen.

  Preferably, two of the calibration markers are displayed on the captured image so as to correspond to two points separated from each other of the calibration index, and each display position can be independently controlled by the instruction means. Good. According to this, since the display positions of the two calibration markers on the captured image can be controlled independently, the operator can easily operate each calibration marker so as to overlap the two points indicated by the configuration index.

  The calibration marker may be composed of line segments intersecting each other, and the intersection may indicate the display position of the calibration marker. If the calibration marker is configured in this way, a large size of the calibration marker can be ensured, so that the operator can easily visually recognize the calibration marker on a display screen such as a display and improve operability. In particular, since the intersection point indicates the display position of the calibration marker, the operator can sensuously grasp the accurate display position of the calibration marker, so that it overlaps the two points indicated by the calibration index. Thus, the calibration marker can be accurately moved.

  The calibration index may be formed as a line segment connecting the two points separated from each other. In this case, since the two points indicated by the calibration index are at both ends of the line segment, the operator can recognize the two points indicated by the calibration marker sensuously and easily.

  In order to solve the above-mentioned problem, a calibration method for an in-vehicle camera according to the present invention is a calibration method for an in-vehicle camera that performs calibration by calculating the mounting angle of the in-vehicle camera on the vehicle body by imaging a calibration index, An imaging step of acquiring a captured image including a calibration index having two points separated from each other on a plane, a superimposed display step of displaying a calibration marker superimposed on the acquired captured image, and the calibration marker including the calibration index The in-vehicle camera is based on an instruction step for indicating the superimposed display position of the calibration marker so as to overlap the two points shown, and the display position of the calibration marker displayed so as to overlap the two points of the calibration index. And a calculating step for calculating an attachment angle of the lens.

  The on-vehicle camera calibration method according to the present invention can be suitably realized by the above-described calibration apparatus (including the various aspects described above).

  The vehicle-mounted camera according to the present invention is a vehicle-mounted camera that can be calibrated by calculating the mounting angle of the vehicle-mounted camera to the vehicle body by imaging a calibration index, and has two points apart from each other on a plane. An imaging unit that acquires a captured image including an index, an image signal processing unit that processes an image signal so that a calibration marker is superimposed on the acquired captured image, and an overlay of the calibration marker in the captured image And an instruction receiving unit that receives an instruction of a display position, and calculates the attachment angle based on a display position of the calibration marker when the calibration marker is displayed so as to overlap two points of the calibration index. It is characterized by doing.

  The vehicle-mounted camera according to the present invention can be suitably realized by including the above-described calibration device (including the various aspects described above).

It is a block diagram which shows the whole structure of the calibration apparatus of the vehicle-mounted camera which concerns on a present Example. It is an example of the operation part of the instruction device which is a user interface. It is a variation of the shape of the calibration index imaged by the imaging unit during calibration. It is a flowchart which shows the control content in CPU in steps. It is an example of operation of the marker for a calibration in the calibration apparatus which concerns on a present Example. It shows the positional relationship between an imaging unit installed in a vehicle and a calibration index. An example of rotational movement in the world coordinate system is shown. It is a schematic diagram which shows the adjustment method of the window display position of background art.

  FIG. 1 is a block diagram illustrating the overall configuration of a calibration apparatus 100 for an in-vehicle camera according to the present embodiment. An imaging unit 1 that is an in-vehicle camera attached to a vehicle body includes an imaging element such as a CMOS sensor or a CCD, and converts subject light incident through a lens (not shown) into an electrical signal. The video processing unit 2 processes the electrical signal output from the imaging unit 1 to generate a video signal for displaying, for example, a progressive or interlaced digital image. The graphic superimposing circuit 3 is inputted from the video processing unit 2 so that a calibration marker used at the time of performing a calibration operation to be described later is superimposed and displayed as graphic data at a predetermined position on the captured image acquired by the imaging unit 1. Process the video signal.

  The display of the calibration marker displayed superimposed on the captured image is controlled based on an instruction from the CPU 4. In particular, the display position of the calibration marker is configured to be controlled by the CPU 4 via the pointing device 5 which is a user interface operable by the operator. The instruction device 5 includes an operation unit 5a capable of inputting an instruction for moving the display position of the calibration marker on the captured image.

  The configuration of the in-vehicle camera in the in-vehicle camera calibration device of the present invention is not particularly limited, and includes, for example, an imaging unit 1, a video processing unit 2, a graphic superimposing circuit 3, a CPU 4, and a video output circuit 6. Also good.

  FIG. 2 is an example of the operation unit 5a of the pointing device 5 which is a user interface. The operation unit 5a includes a start button 9 that is turned ON at the start of calibration, a move button 10 that moves the calibration marker in the vertical and horizontal directions, a position confirmation button 11 that is turned ON when the calibration marker has been moved, and calibration is performed. An execution button 12 that is sometimes turned on is arranged. Note that the form of the operation unit 5a is not limited to the example of FIG. 2, and may take various forms such as a mouse, a keyboard, a joystick, and a touch panel.

  Returning to FIG. 1 again, the captured image on which the calibration marker is superimposed by the graphic superimposing circuit 3 is converted by the video output circuit 6 into an image signal in a format suitable for display (for example, NTSC). Then, the converted image signal is input to the display 7 which is a display means, and is displayed on the screen.

  Here, FIG. 3 shows variations in the shape of the calibration index 8 imaged by the imaging unit 1 during calibration. The calibration index 8 used in the present embodiment is formed so as to indicate two points A and B that are separated from each other at a predetermined interval on a plane.

  In the example of FIG. 3A, the calibration index 8 is formed as a line segment connecting two points A and B on the plane, and the two points A and B indicated by the calibration index 8 are located at both ends of the line segment. Therefore, it can be easily recognized by the operator when displayed on the display 7. In FIG. 3A, black circle display is provided at both ends of the calibration index 8 to indicate two points A and B. However, such black circle display may not be necessary, and a wide line width is provided. Thus, visibility to the operator may be improved. Further, as shown in FIG. 3B, the calibration index 8 may be provided in a frame shape so that two points A and B are included at both ends.

  As described above, various variations of the calibration index 8 used in this embodiment can be considered. In the following description, the calibration index 8 having the shape shown in FIG. 3A (the shape in which the black circles at both ends are omitted) is used. A case will be described as an example.

  Next, the control contents in the CPU 4 will be specifically described with reference to FIG. FIG. 4 is a flowchart showing the control contents in the CPU 4 step by step.

  First, the CPU 4 determines whether or not the pointing device 5 has been requested to change the position of the calibration marker in order to perform calibration (step S101). The determination regarding the presence or absence of this request is made by detecting that the start button 9 provided on the operation unit 5a of the pointing device 5 has been turned ON by the operator.

When the ON operation of the start button 9 is detected, the CPU 4 detects whether one of the movement buttons 10 corresponding to the up, down, left, and right directions is turned on by the operator (step S101), and the movement button that is turned on is detected. The display position of the calibration marker is moved according to the type (step S102). The calibration marker moving operation is performed in real time while the operator confirms the position of the calibration marker displayed on the screen of the display 7.
If there is no request in step S101 (step S101: NO), the CPU 4 skips step S102 and proceeds to step S103 described later.

  Here, FIG. 5 is an operation example of the calibration marker in the calibration apparatus 100 according to the present embodiment. FIG. 5A shows an initial state, which is provided on the display screen 20 on the captured image acquired by the imaging unit 1 (the rear bumper 21 of the vehicle body on which the calibration device 100 is mounted and the rear road surface of the vehicle). The calibration marker 8 is reflected), and two calibration markers 14a and 14b superimposed on the captured image are displayed.

  Two calibration markers 14 a and 14 b are displayed so as to correspond to the calibration index 8 displayed on the display 7, and each display position thereof is obtained by operating the movement button 10 provided on the pointing device 5. It is configured to be independently controllable. Thereby, since the display positions of the two calibration markers 14a and 14b on the captured image can be controlled independently, the operator can easily set the calibration markers 14a and 14b to the two points A and B indicated by the calibration index 8. It can be operated to overlap.

  First, the operator selects the calibration marker 14a displayed on the left side of the display screen from the two calibration markers 14a and 14b. Then, the movement button 10 corresponding to the left direction is turned on, the calibration marker 14a is moved to the left side as shown in FIG. 5B, and the left end A of the calibration index 8 is aligned with the horizontal coordinate. Subsequently, the operator turns on the operation button 10 corresponding to the upward direction, moves the calibration marker 14a upward as shown in FIG. 5C, and aligns the left end A of the calibration index 8 with the vertical coordinate. .

  Next, the operator selects the calibration marker 14b displayed on the right side on the display screen from the two calibration markers. Then, following the case of the calibration marker 14a described above, the movement button 10 corresponding to the left direction is turned on, and the calibration marker 14b is moved to the left side as shown in FIG. And the horizontal coordinate. Subsequently, the operator turns on the operation button 10 corresponding to the upward direction, moves the calibration marker 14b upward as shown in FIG. 5E, and aligns the right end B of the calibration index with the vertical coordinate.

  As described above, the calibration device 100 according to the present embodiment can be adjusted so as to overlap the two points A and B indicated by the calibration index 8 simply by moving the calibration markers 14a and 14b in the vertical and horizontal directions on the captured image. That is, since it is not necessary to perform complicated adjustments such as rotating the calibration marker as in the background art (see FIG. 8), there is no need to make adjustments by repeating the position of the calibration marker. It can be effectively reduced. In addition, the movement button 10 arranged on the operation unit 5a of the pointing device 5 can be simplified by the absence of the rotation operation, and is excellent in cost.

Returning to FIG. 4 again, the CPU 4 determines whether or not the change of the display positions of the calibration markers 14a and 14b has been completed (step S103). This determination is made based on whether or not the confirmation button 11 provided on the operation unit 5a has been turned ON by the operator. When the execution button 12 is turned on by the operator, calibration parameters (roll, pan, and tilt angles of the in-vehicle camera) are calculated based on the determined display positions of the calibration markers 14a and 14b, and calibration is performed. (Step S104).
If the positions of the calibration markers 14a and 14b are not determined in step S103 (step S103: NO), the CPU 4 returns the process to step S101 and repeats the above process.

  Next, the calibration parameter calculation method performed in step S104 will be described. FIG. 6 shows the positional relationship between the imaging unit 1 (on-vehicle camera) installed in the vehicle and the calibration index 8. The imaging unit 1 is installed at the rear of the vehicle, and the installation position of the imaging unit 1 is represented by a point (tx, ty, tz) in the world coordinate system (Xw, Yw, Zw) that is an absolute coordinate. The origin of the world coordinate system (Xw, Yw, Zw) is shown in the example of FIG. 6 when it is set at the end center 0 of the rear bumper of the vehicle, but the position of the entire system is uniquely defined. Needless to say, it may be set at any point in the space as much as possible.

で表される。 The relative positional relationship of the system with the position of the imaging unit 1 as a reference can be expressed by a camera coordinate system (Xc, Yc, Zc). This camera coordinate system (Xc, Yc, Zc) is represented as a relative coordinate system (parallel movement and rotational movement with respect to the world coordinate system) with the world coordinates (tx, ty, tz) of the imaging unit 1 as the origin. Here, FIG. 7 shows an example of rotational movement of the world coordinate system (Xw, Yw, Zw), and FIGS. 7A to 7C respectively show rotation (panning), Yw about Zw as an axis. Rotation (roll) around the axis and rotation (tilt) around the Xw axis. When the rotation angle of the pan is α, the rotation angle of the roll is β, and the rotation angle of the tilt is γ, the relationship between the camera coordinate system (Xc, Yc, Zc) and the world coordinate system (Xw, Yw, Zw) is

It is represented by

で表される。尚、式（２）において、ｆはレンズの焦点距離を表わしている。また、レンズの入射角θ_ｉは次式

と算出される。 Here, the relationship between the incident angle θ _i of the subject light to the lens of the imaging unit 1 and the image height y on the imaging element included in the imaging unit 1 is expressed by the following equation, for example, when a pinhole lens is used.

It is represented by In equation (2), f represents the focal length of the lens. The incident angle θ _{i of the} lens is

Is calculated.

で表される。ここで、Xmax、Ymaxは左右方向及び上下方向の画像サイズ、pix_x、pix_yは左右方向及び上下方向のセンサピクセルサイズであり、

である。較正点のワールド座標系での座標は既知であるので、図５に示す一連の指示装置５の操作によって２つの較正点Ａ、Ｂの画像上の座標が決まれば、式（１）―（５）より、パン・ロール・チルトのそれぞれの設置角度を算出することができる。 On the other hand, the image coordinate system (xi, yi) based on the image height y is

It is represented by Here, Xmax and Ymax are the horizontal and vertical image sizes, pix_x and pix_y are the horizontal and vertical sensor pixel sizes,

It is. Since the coordinates of the calibration points in the world coordinate system are known, if the coordinates on the images of the two calibration points A and B are determined by the operation of the series of pointing devices 5 shown in FIG. 5, the equations (1)-(5 ), The installation angles of pan, roll, and tilt can be calculated.

  As described above, according to the in-vehicle camera calibration device of the present embodiment, the calibration index 8 can easily display the display positions of the calibration markers 14a and 14b without complicated operations such as repeated fine adjustments. Calibration can be performed by adjusting so as to overlap two points A and B shown, and calculating the mounting angle (pan, roll, tilt) of the imaging unit 1 (vehicle camera). In particular, the display positions of the calibration markers 14a and 14b can be adjusted by simply moving them up and down and left and right without any rotational movement, so that the operation burden on the operator is small and calibration can be performed easily.

  INDUSTRIAL APPLICABILITY The present invention can be used in a vehicle-mounted camera calibration apparatus and method for performing calibration by calculating an attachment angle of a vehicle-mounted camera to a vehicle body by imaging a calibration index.

DESCRIPTION OF SYMBOLS 1 Imaging part 2 Image processing part 3 Graphic superimposition circuit 4 CPU
DESCRIPTION OF SYMBOLS 5 Instruction apparatus 6 Image | video output circuit 7 Display 8 Calibration index 9 Start button 10 Movement button 11 Position determination button 12 Execution button 14 Calibration marker 20 Display screen 21 Rear bumper 100 Calibration apparatus

Claims (6)

An in-vehicle camera calibration device that performs calibration by calculating the mounting angle of the in-vehicle camera to the vehicle body by imaging a calibration index,
An imaging unit for acquiring a captured image including a calibration index having two points separated from each other on a plane;
Image signal processing means for processing an image signal so that a calibration marker is superimposed and displayed on the acquired captured image;
Instructing means for indicating the superimposed display position of the calibration marker in the captured image,
A calibration of the in-vehicle camera, wherein the mounting angle of the in-vehicle camera is calculated based on a display position of the calibration marker when the calibration marker is superimposed and displayed so as to overlap two points of the calibration index. apparatus.   Two of the calibration markers are displayed on the captured image so as to correspond to two points separated from each other of the calibration index, and each display position thereof can be independently controlled by the instruction means. The in-vehicle camera calibration device according to claim 1.   The in-vehicle camera calibration apparatus according to claim 1, wherein the calibration marker includes line segments intersecting each other, and an intersection thereof indicates a display position of the calibration marker.   4. The on-vehicle camera calibration device according to claim 1, wherein the calibration index is formed as a line segment connecting the two points separated from each other. 5. An in-vehicle camera calibration method for performing calibration by calculating the mounting angle of the in-vehicle camera to the vehicle body by imaging a calibration index,
An imaging step of acquiring a captured image including a calibration index having two points separated from each other on a plane;
A superimposed display step of superimposing and displaying a calibration marker on the acquired captured image;
An instruction step for indicating a superimposed display position of the calibration marker so that the calibration marker overlaps two points indicated by the calibration index;
A vehicle-mounted camera calibration method comprising: a calculation step of calculating an attachment angle of the vehicle-mounted camera based on a display position of the calibration marker displayed so as to overlap two points of the calibration index. An in-vehicle camera that can be calibrated by calculating the mounting angle of the in-vehicle camera to the vehicle body by imaging a calibration index,
An imaging unit for acquiring a captured image including a calibration index having two points separated from each other on a plane;
Image signal processing means for processing an image signal so that a calibration marker is superimposed and displayed on the acquired captured image;
Instruction receiving means for receiving an instruction of the superimposed display position of the calibration marker in the captured image,
The in-vehicle camera, wherein the mounting angle is calculated based on a display position of the calibration marker when the calibration marker is superimposed and displayed so as to overlap two points of the calibration index.

Images