JP2024080208A - Image processing apparatus, image processing method, and program - Google Patents

Abstract

To make it easy to determine data for use in calibration of an imaging apparatus regardless of the type of the imaging device.SOLUTION: An image processing apparatus 1 includes: an image data acquisition unit 131 which acquires first image data generated by a first imaging apparatus 2 imaging a calibration image, and second image data different from the first image data; a feature information group extraction unit 132 which extracts a first feature information group indicating gradient of brightness of the calibration image included in the first image data and a second feature information group indicating gradient of brightness of the calibration image included in the second image data; a reference point specifying unit 133 which specifies a plurality of reference points of the calibration image included in the second image data; a coordinate transformation unit 134 which specifies coordinates of multiple correction reference points by performing coordinate transformation on the coordinates of the reference points on the basis of a result of collating the first feature information group with the second feature information group; and a data generation unit 135 which generates calibration data of the first imaging apparatus 2 on the basis of the coordinates of the correction reference points.SELECTED DRAWING: Figure 2

Description

The present invention relates to an image processing device, an image processing method, and a program.

Conventionally, a method is known in which parameters used for calibrating a camera are measured by comparing feature points in captured images generated by photographing a camera calibration board with multiple types of cameras (see, for example, Patent Document 1).

International Publication No. 2017/056473

Conventional methods are expected to use Harris' corner detection method to detect feature points on a camera calibration board. However, depending on the type of camera, there are cases where pixel values do not change significantly at the edge positions of the pattern contained in the camera calibration board. In such cases, it is not possible to detect feature points on the camera calibration board, which creates the problem of difficulty in determining the parameters used to calibrate the camera.

The present invention has been made in consideration of these points, and aims to make it easier to generate data to be used for calibrating an imaging device, regardless of the type of imaging device.

The image processing device of the first aspect of the present invention has an image data acquisition unit that acquires first image data generated by capturing a calibration image of a predetermined pattern with a first imaging device, and second image data corresponding to an image having the same characteristics as the calibration image and different from the first image data, a feature information group extraction unit that extracts a first feature information group indicating the brightness gradient of the calibration image contained in the first image data and a second feature information group indicating the brightness gradient of the calibration image contained in the second image data, a reference point identification unit that identifies multiple reference points of the pattern of the calibration image contained in the second image data, a coordinate conversion unit that identifies the coordinates of multiple correction reference points by coordinate conversion based on the result of comparing the coordinates of the multiple reference points with the first feature information group and the second feature information group, and a data generation unit that generates calibration data for the first imaging device based on the coordinates of the multiple correction reference points identified by the coordinate conversion unit.

The first feature information group and the second feature information group may be information including directional information of a brightness gradient, and the coordinate conversion unit may compare first directional information corresponding to the first feature information group with second directional information corresponding to the second feature information group.

The coordinate conversion unit may transform at least one of the first image data and the second image data to facilitate matching of the first image data and the second image data, and then match the first image data and the second image data.

The coordinate conversion unit may determine parameters of a conversion function in which the content of the coordinate conversion varies depending on one or more parameters so that the result of the comparison satisfies a predetermined condition, and convert the coordinates of the multiple reference points based on the conversion function that uses the parameters.

In order to measure a first specific area in the first image data, the reference point identification unit may identify a plurality of reference points constituting a second specific area corresponding to the first specific area, which is included in the second image data, and the data generation unit may output measurement data of the first specific area in the first image data based on the coordinates of the plurality of corrected reference points determined by the coordinate conversion unit by converting the plurality of reference points constituting the second specific area.

The first image data is image data generated by capturing an image of a plate-like object including the calibration image, and the pattern of the calibration image may also include an opening formed in the plate-like object.

The first imaging device may be an infrared camera that captures the calibration image using infrared light, and the second image data may be image data generated by capturing the calibration image using a second imaging device that is a visible light camera that captures images using visible light, or image data generated by computer calculations.

The image processing method of the second aspect of the present invention includes the steps of acquiring first image data generated by capturing a calibration image of a predetermined pattern with a first imaging device and second image data corresponding to an image having the same characteristics as the calibration image and different from the first image data, extracting a first group of feature information indicating the brightness gradient of the calibration image contained in the first image data and a second group of feature information indicating the brightness gradient of the calibration image contained in the second image data, identifying a plurality of reference points of the pattern of the calibration image contained in the second image data, identifying the coordinates of a plurality of correction reference points by coordinate conversion based on the result of comparing the coordinates of the plurality of reference points with the first group of feature information and the second group of feature information, and generating calibration data for the first imaging device based on the identified coordinates of the plurality of correction reference points.

The program of the third aspect of the present invention causes a computer to execute the steps of acquiring first image data generated by capturing a calibration image of a predetermined pattern with a first imaging device, and second image data corresponding to an image having the same characteristics as the calibration image and different from the first image data, extracting a first group of feature information indicating the brightness gradient of the calibration image contained in the first image data and a second group of feature information indicating the brightness gradient of the calibration image contained in the second image data, identifying a plurality of reference points of the pattern of the calibration image contained in the second image data, identifying the coordinates of a plurality of correction reference points by coordinate conversion based on the result of comparing the coordinates of the plurality of reference points with the first group of feature information and the second group of feature information, and generating calibration data for the first imaging device based on the identified coordinates of the plurality of correction reference points.

The present invention has the advantage of making it easier to generate data used to calibrate an imaging device, regardless of the type of imaging device.

FIG. 1 is a diagram for explaining an overview of an image processing device 1. 1 is a diagram showing a configuration of an image processing device 1. FIG. 13 is a diagram for explaining an overview of a feature information group. FIG. 13 is a diagram showing an image in which a part of the first image data and a part of the second image data are cut out and arranged in a tiled pattern after being transformed by a warp function. 13 is a diagram showing a state in which a marker indicating the position of a correction reference point calculated by a coordinate conversion unit 134 is superimposed on the first image data. FIG. FIG. 2 is a diagram showing a processing flow in the image processing device 1. FIG. 13 is a diagram showing another example of the calibration image. FIG. 13 is a diagram showing another example of the calibration board.

[Overview of image processing device 1]
Fig. 1 is a diagram for explaining an overview of an image processing device 1. The image processing device 1 is a device for creating calibration data used for calibrating an imaging device, and is, for example, a computer. Fig. 1 shows a flow in which the image processing device 1 creates calibration data for the first imaging device 2 using first image data generated by the first imaging device 2 photographing a calibration board including a calibration image and second image data generated by the second imaging device 3 photographing a calibration board including a calibration image.

The first imaging device 2 has different performance from the second imaging device 3. The first imaging device 2 is an imaging device that cannot clearly capture the pattern of the calibration image, such as an infrared camera or a camera equipped with an image filter. The second imaging device 3 is an imaging device that can capture the pattern of the calibration image more clearly than the first imaging device 2, such as a visible light camera. If the contour of the pattern of the calibration image in the first image data generated by the first imaging device 2 is unclear, it is difficult to accurately identify the position of the feature points of the calibration image in the first image data, and it is also difficult to generate calibration data for the first imaging device 2.

To solve this problem, the image processing device 1 according to this embodiment is characterized in that it identifies the relationship between multiple positions in the first image data and multiple positions in the second image data by comparing a first group of feature information indicating the brightness gradient of the calibration image contained in the first image data with a second group of feature information indicating the brightness gradient of the calibration image contained in the second image data.

The brightness gradient is represented by the difference between the brightness of a pixel of interest and the brightness of multiple pixels in a specified direction within a specified distance from the pixel. The brightness of multiple pixels is represented by a statistical value such as the average or median brightness of the multiple pixels. The specified distance is, for example, a distance corresponding to a number of consecutive pixels set by the user, and is, for example, a distance of five pixels. Details will be described later, but the first feature information group and the second feature information group are information that include directional information of the brightness gradient at each pixel of the first image data and the second image data, respectively.

Based on the result of identifying the relationship between the multiple positions in the first image data and the multiple positions in the second image data, the image processing device 1 identifies which positions in the first image data correspond to the positions of the reference points (e.g., lattice points) of the calibration image included in the second image data. That is, the image processing device 1 determines the coordinates of multiple correction reference points corresponding to the multiple reference points in the first image data by performing coordinate conversion of the coordinates of the multiple reference points in the second image data based on the result of comparing the first feature information group with the second feature information group. The image processing device 1 can generate calibration data based on the coordinates of the multiple correction reference points determined in this manner. In this way, the image processing device 1 can generate calibration data even if the imaging camera is an imaging device that cannot clearly capture the contour lines of the pattern in the calibration image, such as an infrared camera.

In the above description, the second image data is image data generated by the second imaging device 3, but the method of generating the second image data is arbitrary as long as the pattern of the calibration image is more clearly expressed than the first image data. As an example, the second image data may be image data that is generated by computer calculation and indicates the calibration image captured by the first imaging device 2.

[Configuration of image processing device 1]
The configuration and operation of the image processing device 1 will now be described in detail.
2 is a diagram showing the configuration of the image processing device 1. The image processing device 1 has a communication unit 11, a storage unit 12, and a control unit 13. The control unit 13 has an image data acquisition unit 131, a feature information group extraction unit 132, a reference point identification unit 133, a coordinate conversion unit 134, and a data generation unit 135.

The communication unit 11 is a communication interface for transmitting and receiving data to and from other devices. For example, the communication unit 11 receives first image data from the first imaging device 2 and receives second image data from the second imaging device 3. The communication unit 11 may receive at least one of the first image data and the second image data from a device (e.g., a computer) different from the first imaging device 2 or the second imaging device 3. The communication unit 11 inputs the received first image data and second image data to the image data acquisition unit 131. The communication unit 11 may input the first image data and second image data to the image data acquisition unit 131 by storing the received first image data and second image data in the storage unit 12.

The communication unit 11 may transmit the calibration data generated by the data generation unit 135 to an external device. For example, the communication unit 11 transmits the calibration data to a computer used by a user who performed an operation to generate the calibration data. The communication unit 11 may transmit the calibration data to a display connected to the image processing device 1.

The storage unit 12 has storage media such as a ROM (Read Only Memory), a RAM (Random Access Memory), and an SSD (Solid State Drive). The storage unit 12 stores programs executed by the control unit 13. The storage unit 12 also stores various data used by the control unit 13 to generate calibration data. The storage unit 12 stores, for example, a group of feature information indicating the brightness gradient of the calibration image identified in the first image data and the second image data by the feature information group extraction unit 132.

Figure 3 is a diagram for explaining an overview of the feature information group. The feature information group is composed of multiple directional codes determined for each of multiple pixels included in the image data. The directional code is data that indicates the direction of the brightness gradient at the target pixel. The directional code is data that indicates, for example, the brightness gradient in multiple directions centered on the target pixel.

Figure 3(a) is a diagram showing multiple directions in which the directional code includes a brightness gradient. In Figure 3(a), directions (1) to (16) are shown for one pixel indicated by a black circle in the center, but the number of directions included in the directional code is arbitrary. The directional code may include a brightness gradient for each direction that differs by, for example, one degree.

Figure 3(b) is a diagram showing an example of data included in a direction code. In a direction code, data indicating a direction is associated with data indicating the magnitude of the brightness gradient. In Figure 3(b), the magnitude of the brightness gradient is shown in eight stages from 1 to 8, but the brightness gradient may be shown in more stages. The memory unit 12 stores a group of feature information in which such a direction code is associated with the coordinates of each of a plurality of pixels.

The directional code is not limited to the example shown in FIG. 3, but may be a code obtained by quantizing a value indicating the direction or orientation in which the gradient of brightness in the target pixel is at its maximum. In this case, the code of a pixel that has the same gradient in all directions is assigned a value indicating, for example, that there is no gradient.

The control unit 13 has, for example, a CPU (Central Processing Unit). The control unit 13 executes the programs stored in the storage unit 12 to function as an image data acquisition unit 131, a feature information group extraction unit 132, a reference point identification unit 133, a coordinate conversion unit 134, and a data generation unit 135.

The image data acquisition unit 131 acquires, via the communication unit 11, first image data generated by capturing an image for calibration of a predetermined pattern with the first imaging device 2, and second image data that corresponds to an image having the same characteristics as the image for calibration captured by the first imaging device 2 and differs from the first image data. The image data acquisition unit 131 inputs the acquired first image data and second image data to the feature information group extraction unit 132 and the reference point identification unit 133.

As described above, the second image data may be image data generated by the second imaging device 3 capturing a calibration image captured by the first imaging device 2, or may be image data generated by a computer and corresponding to an image having the same characteristics as the calibration image captured by the first imaging device 2. An image having the same characteristics as the calibration image may be identical to the calibration image, or may be an image having the same positions of feature points but different colors. When the calibration image is composed of multiple black and white squares as shown in FIG. 1, the calibration image may be an image in which the black and white of the calibration image shown in FIG. 1 are reversed.

The feature information group extraction unit 132 extracts a first feature information group indicating the brightness gradient of the calibration image included in the first image data, and a second feature information group indicating the brightness gradient of the calibration image included in the second image data. To extract the first feature information group and the second feature information group, the feature information group extraction unit 132 first identifies the brightness of multiple pixels included in each of the first image data and the second image data. Next, the feature information group extraction unit 132 selects one pixel, and calculates the average brightness (e.g., the average value or the median value) of pixels within a predetermined distance from the selected pixel in each of a predetermined number of directions.

The feature information group extraction unit 132 identifies the gradient of brightness corresponding to the selected pixel by dividing the difference between the brightness of the selected pixel and the calculated average brightness by a predetermined distance. The feature information group extraction unit 132 quantizes the identified gradient value to identify a value corresponding to the identified gradient, for example, from among eight gradient values. The feature information group extraction unit 132 performs this process on all pixels to generate a direction code as shown in FIG. 3(b). The feature information group extraction unit 132 stores a feature information group including direction codes corresponding to each of the multiple pixels in the memory unit 12, and notifies the feature information group to the reference point identification unit 133 and the coordinate conversion unit 134.

The reference point identification unit 133 identifies multiple reference points of the pattern of the calibration image included in the second image data. The reference point identification unit 133 identifies reference points, which are characteristic points included in the pattern of the calibration image, for example, by identifying the positions of pixels having a gradient equal to or greater than a threshold in the second feature group information group generated by the feature information group extraction unit 132. The reference points are, for example, the vertices of polygons or the centers of circles included in the calibration image. When the calibration image is an image of a lattice pattern as shown in FIG. 1, the reference points are lattice points. The reference point identification unit 133 notifies the coordinate conversion unit 134 of the coordinates of the identified multiple reference points.

The coordinate conversion unit 134 determines the coordinates of the multiple correction reference points by converting the coordinates of the multiple reference points included in the second feature information group based on the result of comparing the first feature information group with the second feature information group. In order to determine the coordinates of the multiple correction reference points, the coordinate conversion unit 134 first compares the first direction information corresponding to the first feature information group with the second direction information corresponding to the second feature information group. The coordinate conversion unit 134 determines the parameters of a conversion function whose contents of the coordinate conversion change depending on one or more parameters so that the result of the comparison satisfies a predetermined condition. This conversion function is a function that indicates a global geometric correspondence between the first image data and the second image data, and is, for example, a parametric quadratic warp function. The coordinate conversion unit 134 determines the coordinates of the multiple correction reference points by converting the coordinates of the multiple reference points based on the conversion function that uses the parameters.

Specifically, the coordinate transformation unit 134 first calculates parameters of a transformation function for transforming at least one of the values of the first direction information or the second direction information so that the difference between the first direction information of the multiple pixels indicated by the direction code included in the first feature information group and the second direction information of the multiple pixels indicated by the direction code included in the second feature information group is minimized. That is, the coordinate transformation unit 134 calculates parameters of a warp function corresponding to the change in the brightness gradient that occurs in the first image data due to the lens distortion of the image processing device 1.

The coordinate transformation unit 134 determines the parameters by solving the minimization problem of the objective function with the parameters of the warp function as control variables. Specifically, the coordinate transformation unit 134 transforms at least one of the values of the first direction information or the second direction information using the provisionally determined parameters, and then compares the transformed values of the first direction information and the second direction information, and if the difference is smaller than the previous comparison result, updates the provisionally determined parameters to the latest parameters. The coordinate transformation unit 134 repeats this process a predetermined number of times to determine parameters that will sufficiently reduce the difference between the transformed values of the first direction information and the second direction information.

As an example, the coordinate conversion unit 134 defines a quadratic warp function W(X;P) as shown in the following equation (1) with image coordinates X=(x, y) ^T and parameters P=(p1, p2, p3, p4, p5, p6, p7, p8) ^T .

Figure 4 shows an image in which a portion of the first image data and a portion of the second image data are cut out and arranged in a tiled pattern after transformation using a warp function. In Figure 4, it can be seen that the positions of the grid points in the calibration image in the transformed first image data match the positions of the grid points in the calibration image in the second image data. Note that color correction has been applied to the first image data in Figure 4 to make it easier to see.

Next, the coordinate transformation unit 134 transforms the positions of the multiple reference points of the pattern of the calibration image included in the second image data identified by the reference point identification unit 133 using the calculated warp function to identify the positions of the multiple correction reference points. Specifically, the coordinate transformation unit 134 inputs x and y in the coordinates of the multiple reference points identified by the reference point identification unit 133 into the warp function shown in, for example, equation (1), and sets the output x and y coordinates as the coordinates of the correction reference points.

Figure 5 is a diagram showing a state in which markers indicating the positions of correction reference points calculated by the coordinate conversion unit 134 are superimposed on the first image data. In Figure 5, the positions of the correction reference points are indicated by "+" markers. Although the positions of the lattice points are not clear in the first image data, it can be seen that the correction reference points are indicated around the dark and light areas, and the positions of the lattice points in the first image data are appropriately identified.

However, if the shape of the first image data and the shape of the second image data are different, the coordinate transformation unit 134 cannot calculate an appropriate warp function. Therefore, in order to make it easier to compare the first image data and the second image data, the coordinate transformation unit 134 may deform at least one of the image corresponding to the first image data or the image corresponding to the second image data before comparing the first image data and the second image data.

The coordinate transformation unit 134 performs image transformation processing, such as a predetermined affine transformation, on at least one of the first image data or the second image data, to deform the first image data and the second image data so that their shapes are approximately equal. The transformation includes at least one of changing the shape, changing the size, or rotating. The parameters of the affine transformation are determined in advance based on a known relationship between at least one of the shape or size of the image data generated by the first imaging device 2 and at least one of the shape or size of the image data generated by the second imaging device 3.

The coordinate conversion unit 134 may compare the shapes of the contour lines of the first image data and the second image data (e.g., the contour lines of a calibration board) and deform at least one of the first image data or the second image data so that the contour lines of the first image data and the second image data have the same shape. By operating in this manner, the coordinate conversion unit 134 can compare the first image data and the second image data of similar shapes to calculate a warp function, thereby improving the accuracy of the warp function.

The data generation unit 135 generates calibration data for the first imaging device 2 based on the coordinates of the multiple correction reference points identified by the coordinate conversion unit 134. For example, the data generation unit 135 generates corrected coordinates for the coordinates of positions between the multiple reference points based on the coordinates of the multiple correction reference points, and generates calibration data indicating multiple corrected coordinates for the multiple coordinates in the first image data generated by the first imaging device. The data generation unit 135 outputs the calibration data via the communication unit 11 to an external device such as a computer that performs a process of calibrating the first image data.

[Measurement based on image data]
There are cases where it is required to measure the shape, such as the length or size, of an object included in the first image data. However, if the contour line of the object is not clear in the first image data, it is difficult to measure the shape of the object.

The image processing device 1 may therefore use the second image data to measure the shape of a specific area included in the first image data. Specifically, in order for the image processing device 1 to measure the first specific area in the first image data, the reference point identification unit 133 identifies a plurality of reference points constituting a second specific area included in the second image data and corresponding to the first specific area. The data generation unit 135 generates measurement data of the first specific area in the first image data based on the coordinates of a plurality of corrected reference points determined by coordinate conversion of a plurality of reference points constituting the second specific area.

As an example, when the image processing device 1 measures the distance between the lattice points of the calibration image shown in FIG. 1 included in the first specific region in the first image data, the reference point identification unit 133 identifies multiple lattice points in the region including the calibration image, which is the second specific region in the second image data, as multiple reference points. The data generation unit 135 calculates the coordinates of multiple corrected reference points obtained by correcting the positions of the multiple lattice points using a warp function, and measures the distance between the calculated multiple corrected reference points, thereby measuring the distance between the lattice points of the calibration image in the first image data. In this way, the image processing device 1 can measure the shape, etc. of an object included in the image data generated by the first imaging device 2 even if the contour line of the object, etc. is unclear.

[Processing flow in image processing device 1]
6 is a diagram showing a flow of processing in the image processing device 1. The image data acquisition unit 131 acquires, for example, first image data from the first imaging device 2 (S1) and acquires second image data from the second imaging device 3 (S2). The image data acquisition unit 131 may acquire the first image data and the second image data at a predetermined time interval, or may acquire the first image data and the second image data at the time when an instruction to create calibration data is received.

Next, if the image corresponding to the first image data and the image corresponding to the second image data are not of the same shape (YES in S3), the coordinate transformation unit 134 deforms at least one of the image corresponding to the first image data or the image corresponding to the second image data (S4). If the image corresponding to the first image data and the image corresponding to the second image data are of the same shape (NO in S3), the coordinate transformation unit 134 does not deform the image corresponding to the first image data and the image corresponding to the second image data. The coordinate transformation unit 134 identifies coordinate transformation parameters, i.e., parameters of the transformation function, based on a predetermined condition (e.g., a condition that the result of comparing the first directional information corresponding to the first feature information group and the second directional information corresponding to the second feature information group is minimized) (S5).

Before processing S3 to S5, after processing S3 to S5, or in parallel with processing S3 to S5, the reference point identification unit 133 identifies multiple reference points of the pattern of the calibration image included in the second image data (S6).

Next, the coordinate conversion unit 134 converts the coordinates of the multiple reference points identified by the reference point identification unit 133 using a conversion function of the parameters identified in S5, thereby identifying the coordinates of multiple correction reference points in the first image data (S7). The data generation unit 135 generates calibration data for the first imaging device 2 based on the coordinates of the multiple correction reference points identified by the coordinate conversion unit 134 (S8).

[Modification]
In addition to the examples described above, various modified examples are conceivable. Although the calibration image in which black and white squares are arranged alternately is shown in Fig. 1, the calibration image and the calibration board may have any shape.

Figure 7 shows another example of a calibration image. In the calibration image of Figure 7(a), black circles are arranged at equal intervals in the up, down, left and right directions. In the calibration image of Figure 7(b), squares, each with a different pattern, are arranged at equal intervals in the up, down, left and right directions. In this way, the form of the calibration image is arbitrary, as long as it contains a pattern from which reference points can be extracted.

Figure 8 is a diagram showing another example of a calibration board. The calibration board shown in Figure 8 is characterized by having an opening. That is, the first image data is image data generated by capturing an image of a plate-like object including a calibration image, and the pattern of the calibration image also includes an opening formed in the plate-like object. The calibration board shown in Figure 8 has two circular openings, a square opening, and a star-shaped opening, and the background is visible through the openings. The difference between the color of the calibration board and the color of the background produces the same effect as when a calibration image is drawn on the calibration board.

[Effects of image processing device 1]
As described above, the feature information group extraction unit 132 extracts a first feature information group indicating the brightness gradient of the calibration image included in the first image data and a second feature information group indicating the brightness gradient of the calibration image included in the second image data. The coordinate conversion unit 134 then performs coordinate conversion on the coordinates of the multiple reference points included in the second image data identified by the reference point identification unit 133 based on the result of comparing the first feature information group with the second feature information group, thereby identifying the coordinates of multiple correction reference points. The data generation unit 135 then generates calibration data for the first imaging device 2 based on the coordinates of the multiple correction reference points.

By configuring the image processing device 1 in this manner, even if the position of the reference point in the calibration image is unclear, such as when the first image data is infrared image data, the image processing device 1 can identify the position of the reference point in the first image data. As a result, the image processing device 1 can generate calibration data for an imaging device that cannot clearly capture the contour lines of the pattern in the calibration image, such as an infrared camera. In other words, the image processing device 1 makes it easier to generate data to be used for calibrating an imaging device, regardless of the type of imaging device.

Although the present invention has been described above using embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments, and various modifications and changes are possible within the scope of the gist of the invention. For example, all or part of the device can be configured by distributing or integrating functionally or physically in any unit. In addition, new embodiments resulting from any combination of multiple embodiments are also included in the embodiments of the present invention. The effect of the new embodiment resulting from the combination also has the effect of the original embodiment.

REFERENCE SIGNS LIST 1 Image processing device 2 First imaging device 3 Second imaging device 11 Communication unit 12 Storage unit 13 Control unit 131 Image data acquisition unit 132 Feature information group extraction unit 133 Reference point identification unit 134 Coordinate conversion unit 135 Data generation unit

Claims (9)

an image data acquisition unit that acquires first image data generated by capturing an image for calibration of a predetermined pattern with a first imaging device, and second image data corresponding to an image having the same characteristics as the image for calibration and different from the first image data;
a feature information group extraction unit that extracts a first feature information group indicating a brightness gradient of the calibration image included in the first image data and a second feature information group indicating a brightness gradient of the calibration image included in the second image data;
a reference point specifying unit that specifies a plurality of reference points of a pattern of the calibration image included in the second image data;
a coordinate conversion unit that converts coordinates of the plurality of reference points based on a result of comparing the first feature information group with the second feature information group to identify coordinates of a plurality of correction reference points;
a data generating unit that generates calibration data for the first imaging device based on the coordinates of the plurality of correction reference points identified by the coordinate conversion unit;
An image processing device comprising: the first feature information group and the second feature information group are information including directional information of a brightness gradient, and the coordinate conversion unit compares first directional information corresponding to the first feature information group with second directional information corresponding to the second feature information group.
The image processing device according to claim 1 . the coordinate transformation unit transforms at least one of the first image data and the second image data so as to facilitate matching of the first image data and the second image data, and then matches the first image data and the second image data.
The image processing device according to claim 1 . the coordinate conversion unit determines one or more parameters of a conversion function in which the content of the coordinate conversion changes depending on the one or more parameters so that the result of the comparison satisfies a predetermined condition, and converts the coordinates of the plurality of reference points based on the conversion function using the determined parameters.
The image processing device according to claim 1 . In order to measure a first specific area in the first image data, the reference point identification unit identifies a plurality of the reference points constituting a second specific area included in the second image data and corresponding to the first specific area;
The data generation unit outputs the measurement data of the first specific area in the first image data based on the coordinates of the plurality of corrected reference points determined by the coordinate conversion unit through coordinate conversion of the plurality of reference points constituting the second specific area.
The image processing device according to claim 1 . The first image data is image data generated by capturing an image of a plate-like object including the calibration image, and a pattern of the calibration image also includes an opening formed in the plate-like object.
The image processing device according to claim 1 . the first imaging device is an infrared camera that captures the calibration image using infrared rays,
The second image data is image data generated by capturing the calibration image by a second imaging device that is a visible light camera that captures an image using visible light, or image data generated by a computer.
The image processing device according to claim 1 . The computer executes
acquiring first image data generated by capturing an image of a calibration pattern by a first imaging device, and second image data corresponding to an image having the same characteristics as the calibration image and different from the first image data;
extracting a first set of feature information indicating a gradient of brightness of the calibration image included in the first image data, and a second set of feature information indicating a gradient of brightness of the calibration image included in the second image data;
identifying a plurality of reference points of a pattern of the calibration image included in the second image data;
determining coordinates of a plurality of correction reference points by performing coordinate transformation on the plurality of reference points based on a result of matching the first feature information group with the second feature information group;
generating calibration data for the first imaging device based on the coordinates of the identified correction reference points;
An image processing method comprising the steps of: On the computer,
acquiring first image data generated by capturing an image of a calibration pattern by a first imaging device, and second image data corresponding to an image having the same characteristics as the calibration image and different from the first image data;
extracting a first set of feature information indicating a gradient of brightness of the calibration image included in the first image data, and a second set of feature information indicating a gradient of brightness of the calibration image included in the second image data;
identifying a plurality of reference points of a pattern of the calibration image included in the second image data;
determining coordinates of a plurality of correction reference points by performing coordinate transformation on the plurality of reference points based on a result of matching the first feature information group with the second feature information group;
generating calibration data for the first imaging device based on the coordinates of the identified correction reference points;
A program for executing the above.

Images