JP2002042125A - Image synthesizing device, image synthesizing method, and computer readable recording medium recording image synthesizing program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To synthesize an image by preventing the deterioration of image quality to the utmost. SOLUTION: This image synthesizing device synthesizes at least three split images having overlapping portions on each other. The image synthesizing device conducts a coordinate transformation quantity calculating process S4 for detecting the relative positional relations of at least three split images based on the images of the overlapping portions, a reference image determining process S6 for determining a reference split image among at least three split images based on the detected positional relations, and an overlapping section synthesizing process S8 for synthesizing the other split images, with the determined split image used as a reference.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image synthesizing apparatus, an image synthesizing method, and a computer-readable recording medium storing an image synthesizing program, and more particularly, to dividing an original or a natural scenery into a plurality of areas. The present invention relates to an image synthesizing apparatus, an image synthesizing method, and a computer-readable recording medium on which an image synthesizing program is recorded.

[0002]

2. Description of the Related Art Conventionally, there is an apparatus which divides a region to be imaged into a plurality of parts, images the image, and combines a plurality of divided images obtained by the divisional photographing to obtain one high-definition or wide-angle image. Proposed.

(1) Japanese Patent Application Laid-Open No. Hei 6-141246 discloses a method in which a plurality of images obtained by dividing an image by an optical path dividing means so as to have an area overlapping each other are picked up by using image signals of the overlapping area. A technique for detecting and synthesizing a positional relationship is described. According to this, two images selected from a plurality of images or a combined image are to be combined, and all the images are attached by repeating the combination. Further, JP-A-6-1
In 41246, in synthesizing three or more divided images that are continuous in the same direction, the amount of rotation is threshold-processed when a composite image obtained by synthesizing two divided images and a third image are synthesized, and the amount of rotation is determined. A technique for rotating and converting a composite image instead of a third image when the image is large is described. This prevents a displacement amount, particularly a rotation amount, obtained from a correlation between two images to be combined from being cumulatively increased when three or more divided images continuous in the same direction are combined. It is like that. As a result, it is possible to prevent the correlation between the two images to be combined from being undetectable or from being mismatched.

(2) Japanese Patent Application Laid-Open No. Hei 9-74522 discloses a method in which four divided images taken by a camera having four image pickup devices are output to a personal computer together with predetermined relative position information, and the personal computer outputs the predetermined images. A technique is described in which four divided images are synthesized based on relative position information with reference to the divided images obtained.

[0005]

However, in JP-A-6-141246, the first divided image, that is,
When the images are combined, the leftmost divided image is used as a reference image, and another image is geometrically transformed in accordance with the reference image, so that the second and subsequent divided images do not become reference images. For this reason, unnecessary geometric transformation may have to be performed on the second and subsequent images, and there is a problem that the image quality of the synthesized image is deteriorated.

In addition, when the combined image obtained by combining the first and second divided images and the third image are combined, the amount of rotation is subjected to threshold processing, and when the amount of rotation is large, the third image is processed. Instead, the combined image is rotationally converted, but the relative displacement between the third image and the fourth image is not always small. When the displacement amount is large, it is necessary to further convert the image obtained by synthesizing the first to third images.

Japanese Patent Application Laid-Open No. 9-74522 synthesizes four divided images based on predetermined relative position information. Therefore, when a relative positional relationship shifts, images are synthesized accurately. Can not do it. Further, since the reference divided image is predetermined, another divided image is not used as a reference.

An object of the present invention is to solve the above-mentioned problems, and an object of the present invention is to provide an image synthesizing apparatus capable of synthesizing an image while minimizing deterioration of image quality. It is to provide.

[0009]

According to one aspect of the present invention, there is provided an image synthesizing apparatus for synthesizing at least three divided images having portions overlapping each other. Position detecting means for detecting a relative positional relationship between at least three divided images based on the image of the overlapping portion; and a reference from among the at least three divided images based on the detected positional relationship. Reference image determining means for determining a divided image, and image synthesizing means for synthesizing another divided image based on the determined divided image are provided.

According to the present invention, a reference divided image is determined from at least three divided images based on a relative positional relationship between at least three divided images, and the determined divided image is used as a reference. , And another divided image. As a result, it is possible to provide an image synthesizing apparatus capable of synthesizing an image while minimizing image quality deterioration.

Preferably, the position detecting means of the image synthesizing device calculates a relative rotational shift amount of at least three divided images.

According to the present invention, a relative rotational shift amount of at least three divided images is calculated. For this reason, the reference divided image is determined based on the conversion amount for rotationally converting the divided image, and it is possible to prevent the image quality from deteriorating.

[0013] Preferably, the reference image determining means of the image synthesizing apparatus determines a reference divided image according to the number of divided images having a relative rotation shift amount equal to or less than a predetermined value.

According to the present invention, a reference divided image is determined according to the number of divided images for which the relative rotational shift amount is equal to or less than a predetermined value. For this reason, a reference divided image can be determined so that the number of divided images to be subjected to the rotation conversion is reduced.

Preferably, the image synthesizing means of the image synthesizing apparatus includes a rotation converting means for rotationally converting the divided image, and the rotation converting means does not rotationally convert the divided image having a relative rotation shift amount equal to or less than a predetermined value. It is characterized by the following.

According to the present invention, since a divided image having a relative rotation shift amount equal to or smaller than a predetermined value is not rotationally converted, it is possible to prevent deterioration in image quality due to rotationally converting the image.

According to another aspect of the present invention, an image synthesizing method is an image synthesizing method for synthesizing at least three divided images having mutually overlapping portions, wherein at least three divided images having overlapping portions are synthesized based on an image of the overlapping portion. Detecting a relative positional relationship between the three divided images;
The method includes a step of determining a reference divided image from at least three divided images based on the detected positional relationship, and a step of combining another divided image based on the determined divided image.

According to the present invention, a reference divided image is determined from at least three divided images based on a relative positional relationship between the at least three divided images, and the determined divided image is used as a reference. , And another divided image. As a result, it is possible to provide an image synthesizing method capable of synthesizing images while minimizing image quality deterioration.

According to still another aspect of the present invention, a computer readable recording medium having recorded thereon an image synthesizing program is a computer having recorded thereon an image synthesizing program for synthesizing at least three divided images having portions overlapping each other. Detecting a relative positional relationship between the at least three divided images on the basis of the image of the overlapping portion, and detecting the relative positional relationship between the at least three divided images based on the detected positional relationship. Determining a reference divided image from among them;
Recording an image synthesizing program for causing a computer to execute the step of synthesizing another divided image based on the determined divided image.

According to the present invention, a reference divided image is determined from at least three divided images based on a relative positional relationship between the at least three divided images, and the determined divided image is used as a reference. , And another divided image. As a result, it is possible to provide a computer-readable recording medium on which an image synthesizing program capable of synthesizing images while minimizing image quality deterioration is recorded.

[0021]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An image synthesizing apparatus according to an embodiment of the present invention will be described below using a camera as an example. The same reference numerals in the drawings denote the same or corresponding members, and description thereof will not be repeated.

FIG. 1 is a perspective view from the front of the camera according to the present embodiment. Referring to FIG.
Are the release switch 109 and the imaging lens unit 121
And a recording medium insertion slot 123.

In the camera 100 according to the present embodiment,
It is possible to improve the image quality of images obtained by imaging paper documents such as materials, catalogs, magazines, and research records distributed at conferences, landscapes, and persons.

FIG. 2 shows a camera 10 according to the present embodiment.
FIG. 2 is a block diagram showing a schematic configuration of a block 0. Referring to FIG. 2, camera 100 includes a central processing unit (hereinafter, referred to as “CPU”) 101 that controls the entire camera 100, and a CPU 1.
01 and an image sensor 1 including a charge coupled device (hereinafter referred to as a “CCD”) and the like.
13 and an analog / digital converter 1 for converting analog data output by the image sensor 113 into digital data
15 (hereinafter, referred to as an “A / D converter 115”), an image memory 117 for storing the converted digital data, a removable recording medium 119, and an actuator 105 for moving the lens of the imaging lens unit 121. , A position sensor 107 for detecting the position of the lens, and an optical axis direction controller 103 connected to the actuator 105, the position sensor 107 and the release switch 109.

The program memory 111 includes a CPU 101
Is a random access memory for recording a program to be executed by the CPU and for storing variables and the like necessary for executing the program. Further, the program memory 111 can also store a program read from the removable recording medium 119.

The optical axis direction control section 103 controls the actuator 105 in response to the release switch 109 being turned on. The actuator 105 moves the lens of the imaging lens unit 121 in two axial directions orthogonal to each other. The lens of the imaging lens unit 121 is moved by the actuator 105, and the optical axis of the imaging lens unit 121 is moved. Therefore, the optical axis is determined by the position of the lens. The position of the lens is detected by the position sensor 107, and the optical axis direction control unit 103
Is output to The optical axis direction control unit 103 includes the position sensor 1
The actuator 105 is controlled until the output of 07 reaches a predetermined value, that is, until the optical axis reaches a predetermined position. The position of the optical axis is determined at a position at which the imaging area of the camera 100 is divided into four parts, that is, up, down, left, and right.

The image sensor 113 photoelectrically converts the received light and outputs analog signal image data.

The removable recording medium 119 is a recording medium that can be mounted on the camera 100 through the recording medium insertion slot 123. The removable recording medium 119 is, for example, P
CMCIA (personal computer memory and interface
A hard disk card or the like based on an association may be used, or a memory card or the like may be used instead. Further, it may be a mini disk (MD), a magneto-optical disk, a digital video disk, a floppy (registered trademark) disk, a flash memory card, or the like. Further, in addition to the removable recording medium 119,
For example, SCSI (small computer system interfac
e) The camera 100 may be directly connected to a printer or the like by a cable or the like.

The removable recording medium 119 records an image recorded in the image memory 117,
Record the program for the control of. In this case,
After the program recorded on the removable recording medium 119 is read, the program is stored in the program memory 111 and executed by the CPU 101.

Next, the camera 100 according to the present embodiment
Will be described. Here, the four-division imaging will be described. The four-division imaging is an operation of dividing an imaging region into four regions, that is, upper, lower, left, and right, which overlap each other, and sequentially imaging four parts. In the camera in the present embodiment, the optical axis is moved in the order of upper left, upper right, lower left, and lower right, and an image is taken.

When the operator presses the release switch 109 provided on the upper surface of the camera 100, the divisional imaging is started. When the optical axis direction control unit 103 controls the actuator 105 in response to an instruction from the CPU 101, the optical axis of the imaging lens unit 121 moves to a position where the divided image corresponding to the upper left portion is captured.

When the movement of the optical axis of the imaging lens unit 121 is completed, the CPU causes the image sensor 113 to output an image, and the output image is converted from analog data to digital data. Then, the converted image is stored in the image memory 117 as a divided image of the upper left portion.

Next, in response to an instruction from the CPU 101, the optical axis direction control unit 103 controls the actuator 105 to move the optical axis of the imaging lens unit 121 to a position where a divided image corresponding to the upper right portion is captured. Then, after the image output from the image sensor 113 is converted into digital data, the image memory 11
7 is stored. Movement of the optical axis of the imaging lens unit 121, output of an image by the imaging sensor 113, A / D conversion unit 115
Is repeated, and the divided image at the lower right part and the divided image at the lower left part are stored in the image memory 117.

FIG. 3 is a diagram showing an example of four divided images obtained by dividing and photographing with the camera according to the present embodiment. FIG. 3A shows an upper left divided image obtained by imaging the upper left portion of the imaging region. FIG. 3B shows an upper right divided image obtained by imaging the upper right portion of the imaging region. FIG.
(C) shows a lower left divided image obtained by imaging the lower left portion of the imaging region. FIG. 3D shows a lower right divided image obtained by imaging the lower right portion of the imaging region.

Referring to FIGS. 3A to 3D, at least a part of standard overlap area OLab of upper left divided image 210 overlaps with at least a part of standard overlap area OLba of upper right divided image 220. I do.
Standard overlap area OLac of upper left divided image 210
At least partially overlaps the standard overlap area OLca of the lower left divided image 230. Similarly, at least a part of the standard overlap area OLdc of the lower right divided image 240 is
At least one of the 30 standard overlap regions OLcd
Overlap with the department. At least a part of the standard overlap area OLdb of the lower right divided image 240 overlaps with at least a part of the standard overlap area OLbd of the upper right divided image 220.

The upper left divided image 210, the upper right divided image 220, the lower left divided image 230, and the lower right divided image 240
Partially include the subject image 201. In the present embodiment, the subject image 201 is a document image.

When the divisional imaging is performed by hand-held imaging in which the camera 100 is supported by the hand, the relative positional relationship between the four divisional images is not constant because the camera 100 shakes due to camera shake. In addition, even when the camera 100 is fixedly photographed by a tripod or the like to perform divided photographing, the relative positional relationship between the four divided images is not constant due to the mechanical positional accuracy of the imaging system or the optical system of the camera 100.

For this reason, the camera 1 according to the present embodiment
At 00, when performing image synthesis processing on the four divided images, the amount of positional deviation is detected from the four divided images obtained by imaging, and the positional deviation between the divided images is corrected to perform the bonding. The detection of the amount of displacement is performed by associating the feature points existing in the overlapping portions between the divided images. The image synthesis processing described below may be performed inside the camera 100, or may be performed by transferring the four divided images to another camera connected to the camera 100 or a terminal of a personal computer.

FIG. 4 shows a camera 10 according to the present embodiment.
9 is a flowchart showing the flow of an image synthesis process performed at 0. Referring to FIG. 4, the image synthesizing process is an image correcting process for correcting each of the divided images (step S1).
And a feature point detection process for detecting a feature point from an overlapped portion of each divided image (step S2), and a correspondence check for detecting a corresponding feature point in an overlapping portion of the detected feature points Out processing (step S
3), coordinate conversion amount calculation processing (step S4), coordinate conversion amount comparison processing (step S5), and reference image determination processing for determining a reference image in image synthesis (step S6)
And a geometric transformation process for geometrically transforming the divided images other than the reference image (step S7), an overlap portion combining process for combining the geometrically transformed divided images in the overlap portion (step S8), and combining the four divided images. An effective image area extraction process (step S9) for extracting an effective image area from the synthesized image.

(Image Correction Processing) This processing is performed for each of the four divided images recorded in the image memory 117, and is a processing for correcting the distortion of the divided images themselves during photographing. More specifically, this is a process of correcting image distortion caused by the lens of the imaging lens unit 121 and image distortion caused by camera tilt. Image processing for removing these distortions from the divided image includes correction processing using the characteristic values of the lens, and determining the angle between the subject at the time of shooting and the optical axis of the imaging lens unit 121 as the tilt angle, and determining the tilt angle obtained. A known technique, such as correcting the distortion by geometrically transforming the image from the image, can be used.

(Feature Point Detection Processing) A feature point is detected in a standard overlap area of one of the divided images overlapping each other. A feature point is a point or area serving as a reference for aligning one divided image with another divided image to be combined. The feature point can be an end point of a line segment included in the standard overlap region, a vertex angle of a polygon, or the like. In these cases, detection is easy.

Here, referring to FIG. 3, upper left divided image 2
From the 10 standard overlapping areas OLab to the feature points P
1 and P2 are detected.

At least two feature points are detected.
This is because the entire movement of the divided image can be specified by the movement of at least two feature points. That is, by moving at least two feature points, parallel movement, rotation movement, and magnification conversion of the divided image can be specified.

(Corresponding Point Detection Processing) A point having the highest correlation with the detected feature point is detected as a corresponding point from the standard overlap area of the other divided image to be synthesized.
For example, a point or area having the smallest difference from the feature point may be set as a point or area having a high correlation. Specifically, the corresponding points corresponding to the feature points P1 and P2 of the standard overlap region OLab of the upper left divided image 210 exist in the standard overlap region OLba of the upper right divided image 220. A corresponding point P4 corresponding to the feature point P1 and a corresponding point P5 corresponding to the feature point P2 are detected.

Similarly, the characteristic points P2 and P2 of the standard overlap area OLac of the upper left divided image 210
The corresponding point corresponding to 3 exists in the standard overlap region OLca of the lower left divided image 230. A corresponding point P8 corresponding to the feature point P2 and a corresponding point P7 corresponding to the feature point P3 are detected.

The corresponding points corresponding to the feature points P5 and P6 of the standard overlap area OLbd of the upper right divided image 220 exist in the standard overlap area OLdb of the lower right divided image 240. A corresponding point P10 corresponding to the feature point P5 and a corresponding point P11 corresponding to the feature point P6 are detected.

The corresponding points corresponding to the feature points P8 and P9 of the standard overlap area OLcd of the lower left divided image 230 exist in the standard overlap area OLdc of the lower right divided image 240. A corresponding point P10 corresponding to the feature point P8 and a corresponding point P12 corresponding to the feature point P9 are detected.

(Coordinate Conversion Amount Calculation Processing) A conversion amount is calculated when one of the divided images is used as a reference image and geometric transformation is performed to combine a divided image other than the reference image with the reference image. Upper left divided image 210, upper right divided image 220,
The conversion amount is calculated for each divided image by changing the divided image serving as the reference image in the order of the lower left divided image 230 and the lower right divided image 240.

The conversion amount is calculated using the corresponding points included in the standard overlap area of the divided image, and is represented by the coordinate conversion amount with respect to the reference image. The coordinate conversion amount includes a rotation amount by rotation conversion, a parallel movement amount by parallel movement conversion, and a magnification by magnification conversion. Since the rotation conversion is a conversion in which coordinates are rotated and moved around a predetermined point, a pixel after conversion and a pixel before conversion do not match in pixel units. For this reason, interpolation processing is required to obtain the pixel value of the pixel after the conversion. As a result, the image quality of the rotation conversion is significantly degraded as compared with the translation conversion or the magnification conversion. On the other hand, the magnification conversion can be ignored if the imaging distance does not change between the divided images. Further, since the parallel translation conversion is coordinate-transformed on a pixel-by-pixel basis, it is not visually noticeable, and high-speed, high-quality conversion processing is possible. In the present embodiment, a rotation amount that most affects the image quality is used as the coordinate conversion amount.

The coordinate conversion calculation processing will be specifically described. FIG. 5 is a flowchart showing the flow of the coordinate conversion amount calculation process performed in step S4 of FIG. Referring to FIG. 5, first, a reference image is set (step S11). The four divided images are divided into an upper left divided image 210,
The upper right divided image 220, the lower left divided image 230, and the lower right divided image 240 are set as the reference images in this order. Here, a case where the upper left divided image 210 is set as the reference image will be described.

Next, the coordinate transformation amount Rh of the upper right divided image 220 overlapping the upper left divided image 210 in the horizontal direction is calculated (step S12). The coordinate transformation amount Rh is calculated using the feature points P1 and P2 of the standard overlap region OLab of the upper left divided image 210, and the corresponding points P4 and P5 of the standard overlap region OLba of the upper right divided image 220. You. The coordinate transformation amount Rh is represented by an angle formed by a vector starting from the feature point P1 and ending at the feature point P2, and a vector starting from the corresponding point P4 and ending at the corresponding point P5.

Next, the coordinate transformation amount Rv of the lower left divided image 230 vertically overlapping the upper left divided image 210 is calculated (step S13). The coordinate transformation amount Rv is calculated using the feature points P3 and P2 of the standard overlap area OLac of the upper left divided image 210, and the corresponding points P7 and P8 of the standard overlap area OLca of the lower left divided image 230. You. The coordinate transformation amount Rv is represented by an angle formed by a vector starting from the feature point P2 and ending at the feature point P3, and a vector starting from the corresponding point P8 and ending at the corresponding point P7.

Then, the coordinate transformation amount R of the lower right divided image 240 that overlaps the upper left divided image 210 in the oblique direction
d is calculated (step S14). Coordinate transformation amount Rd
Is obtained as follows. First, the feature point P5 of the standard overlap area OLbd of the upper right divided image 220
Is rotated around the feature point P6 by the coordinate conversion amount Rh obtained in step S12 to obtain a point P6 ′.
Then, the characteristic points P5 and P6 ′ and the lower right divided image 240
The first conversion amount Rd1 is calculated using the corresponding points P10 and P11 of the standard overlap region OLdb. The first conversion amount Rd1 is calculated based on the point P5 starting from the feature point P5.
It is represented by an angle between a vector ending at 6 'and a vector ending at corresponding point P10 and ending at corresponding point P8.

Second, the characteristic point P9 is rotated around the characteristic point P8 of the standard overlap area OLcd of the lower left divided image 230 by the coordinate transformation amount Rv obtained in step S13 to obtain a point P9 '. Ask for. Then, the feature points P8 and P9 ′, and the corresponding points P10 and P10 of the standard overlap area OLdc of the lower right divided image 240
12, the second conversion amount Rd2 is calculated. The second transformation amount Rd2 includes a vector having the feature point P8 as a start point and a point P9 ′ as an end point, and a corresponding point P10 as a start point and a corresponding point P12.
With the vector ending in.

Thirdly, the average of the first conversion amount Rd1 and the second conversion amount Rd2 is obtained, and the obtained average value is used as the coordinate conversion amount Rd.

In the next step S15, all the divided images are used as reference images, and steps S12 to S1 are performed.
It is determined whether the processes up to 4 have been performed. When the processing has been completed for all the divided images, the coordinate conversion amount calculation processing ends. Otherwise, the above processing is repeated until the processing is completed for all the divided images.

As described above, in the coordinate conversion amount calculation processing, the coordinate conversion amounts Rh, Rv, and Rd of the other three divided images are obtained for each of the four divided images.

(Coordinate conversion amount comparison processing and reference image determination processing) Next, coordinate conversion amount comparison processing and reference image determination processing will be described. FIG. 6 is a flowchart showing a flow of the coordinate conversion amount comparison processing and the reference image determination processing performed in steps S5 and S6 of FIG. Referring to FIG. 6, first, setting of a processed image is performed (step S21). The processed image is a divided image to be compared with the coordinate transformation amount, and includes four divided images 210 and 22.
0, 230, and 240 are sequentially set to the processed image. Here, the upper left divided image 210, the upper right divided image 220, the lower left divided image 230, and the lower right divided image 240 are set in the order of the processed image.

It is determined whether all three coordinate transformation amounts Rh, Rv, and Rd when the divided image set as the processed image is used as the reference image are equal to or smaller than the threshold value (step S2).
2). If all three coordinate transformation amounts Rh, Rv, and Rd are equal to or smaller than the threshold value, the process proceeds to step S23; otherwise, the process proceeds to step S24.

The threshold value is a conversion amount at which the rotation of the image can be visually ignored. In step S23, the divided image set as the processed image is set as the reference image, and the process ends. In step S24, for all the divided images,
It is determined whether the determination in step S22 has been made.
If the determination has been made for all the divided images, the process proceeds to step S25; otherwise, the process proceeds to step S2.
Proceeding to 1, the determination in step S22 is repeatedly performed for all the divided images.

The four divided images 210, 220, 230,
In all of 240, if all three coordinate conversion amounts when each is set as a reference image are equal to or less than the threshold value, 4
The relative positional relationship between the divided images is a positional relationship that does not require rotation conversion for synthesis. In this case, since the rotation conversion is unnecessary even if any of the divided images is used as a reference, any of the divided images may be set as the reference image. In the present embodiment, first, the divided image satisfying the condition of step S22, that is, the upper left divided image 210 is the reference image.
Note that the divided image having the smallest sum of the three coordinate transformation amounts Rh, Rv, and Rd may be used as the reference image.

In step S25, a process image is set (step S21). Four divided images 210,
220, 230, and 240 are sequentially set to the processed image.
Here, the upper left divided image 210, the upper right divided image 220,
It is assumed that the processing image is set in the order of the lower left divided image 230 and the lower right divided image 240.

It is determined whether two of the three coordinate conversion amounts Rh, Rv, and Rd when the divided image set as the processed image is used as the reference image are equal to or smaller than the threshold value (step S).
26). If two of the three coordinate transformation amounts Rh, Rv, and Rd are equal to or smaller than the threshold value, the process proceeds to step S27; otherwise, the process proceeds to step S28.

The threshold value is a conversion amount in which the rotation of the image can be visually ignored, and is the same as the threshold value used in step S24.

When two of the three coordinate transformation amounts Rh, Rv, and Rd are equal to or smaller than the threshold value, the three divided images have a positional relationship that does not require rotation transformation when they are combined.
The remaining one divided image has a positional relationship that requires rotation conversion. A divided image at a position that requires rotational movement is called a rotated image. When one of the four divided images is located at a position that requires rotation conversion, any one of the three divided images excluding the rotated image can be used as the reference image. In the present embodiment, a divided image that is diagonal to the first detected rotated image is set as the reference image. Note that among the three divided images except the rotated image, the divided image having the smallest sum of the coordinate conversion amounts equal to or smaller than the threshold value can be set as the reference image.

Here, referring to FIG.
0, 220, 230, 240, upper left divided image 210
Only at the position rotated relative to the other three divided images 220, 230, 240. When the divided image shown in FIG. 3 is input to the camera 100 in the present embodiment, at the stage where the upper right divided image 220 is set as the processing image, in step S26, two coordinate transformation amounts R
It is determined that v and Rd are equal to or less than the threshold, and step S27
Proceed to. Then, in step S27, the upper left divided image 210 corresponding to the coordinate conversion amount Rh larger than the threshold value is set as a rotated image, and the lower right divided image 240 diagonal to the upper left divided image 210 is set as the reference image. .

In step S28, it is determined whether or not the determination in step S26 has been made for all the divided images. If the determination has been made for all the divided images, the process proceeds to step S29; otherwise, the process proceeds to step S25, and the determination in step S22 is repeatedly performed for all the divided images.

In step S29, setting of a processed image is performed. Four divided images 210, 220, 230, 2
40 are sequentially set to the processed image. Here, the upper left divided image 210, the upper right divided image 220, and the lower left divided image 23
It is assumed that 0 is set to the processed image in the order of the lower right divided image 240.

It is determined whether one of the three coordinate transformation amounts Rh, Rv, and Rd when the divided image set as the processed image is used as the reference image is equal to or smaller than a threshold value (step S3).
0). If one of the three coordinate transformation amounts Rh, Rv, and Rd is equal to or smaller than the threshold value, the process proceeds to step S31; otherwise, the process proceeds to step S32.

The threshold value is a conversion amount by which the rotation of the image can be visually ignored, and is the same as the threshold value used in step S24.

When one of the three coordinate transformation amounts Rh, Rv, and Rd is equal to or smaller than the threshold value, the four divided images 210, 220, 230, and 240 are paired with the upper and lower or left and right divided images in the best condition. And there is no relative rotation between the upper pair and the lower pair, or between the left pair and the right pair. In this case, no matter which of the divided images is set as the reference image, the deterioration of the image quality due to the rotation conversion is almost the same. Therefore, any of the divided images may be used as the reference image. In the present embodiment, first, a processed image satisfying the condition of step S30 is set as a reference image. Note that a divided image having the smallest sum of coordinate conversion amounts equal to or smaller than the threshold value may be set as the reference image.

In step S31, the divided image set as the processed image is set as the reference image, and the process ends.
In step S32, it is determined whether or not the determination in step S30 has been made for all the divided images. If the determination has been made for all the divided images, the process proceeds to step S33; otherwise, the process proceeds to step S29, and the determination in step S30 is repeatedly performed for all the divided images.

In step S33, the divided image having the smallest sum of the coordinate conversion amounts is set as the reference image. Step S3
In No. 3, the coordinate conversion amount does not fall below the threshold value in any of the divided images. In this case, when four divided images are combined, the positional relationship requires rotation conversion. Therefore, the divided image with the smallest rotation conversion amount is set as the reference image.

(Geometric Transformation Processing) The three divided images other than the reference image are geometrically transformed according to the coordinate transformation amounts Rh, Rv, and Rd for the reference image determined in the reference image determination processing.
The geometric transformation process is a process of transforming the coordinates of pixels by rotating or scaling the image. If the converted coordinates are at a position where no pixel exists, interpolation processing is required using the pixel values of adjacent pixels in the converted pixel. Known techniques such as cubic convolution and bilinear method are used for the interpolation processing.

(Overlapping part synthesis processing) Two images are synthesized by aligning the positions of the feature points and the corresponding points corresponding to the feature points on the divided image subjected to the geometric transformation processing. This is done by translating the image. The area where the two images overlap is
The correction is made smoothly so that the joint formed by combining the two images is visually invisible. The correction can be performed, for example, by using the weighting according to the distance from the connection point, and making the pixel value of the combined image a weighted average of the pixel values of the two combined images.

(Effective Image Area Extraction Processing) An effective image area is extracted from a combined image obtained by combining four divided images. The effective image area is the largest area in the composite image where each divided image cannot be lost, and is a rectangular area surrounded by a line parallel to the coordinate axis of the reference image.

FIG. 7 shows a camera 10 according to the present embodiment.
FIG. 4 is a diagram showing a result of performing a combining process on the divided image shown in FIG. 3 at 0. The divided image shown in FIG. 3 is an image obtained by rotating only the upper left divided image 210 among the four divided images 210, 220, 230, and 230 with respect to the other divided images. Therefore, in the coordinate conversion amount comparison processing and the reference image removal processing shown in FIG.
At 27, the lower right divided image 240 is set as the reference image.

Referring to FIG. 7, upper right divided image 220 and lower left divided image 230 are combined with lower right divided image 240 as a reference image without being rotated. The outline of the lower right divided image 240 is parallel to the outline of the upper right divided image 220 and the lower left divided image 230, respectively. On the other hand, the contour of the upper left divided image 210 is not parallel to the contour of the lower right divided image 240. Only the corners formed by the vertical or horizontal contour lines of the divided image are upper left divided images 2
This is because 10 has been rotationally converted.

The four divided images 210, 220, 230,
The subject image 201 is represented as an image having a rectangular outline in the combined image by the combining process of combining the image 240 and the image 240. Further, an effective image area 250 extracted so as not to give a loss to the four divided images is shown.

FIG. 8 is a diagram showing a result of performing a synthesizing process on the divided image shown in FIG. 3 using the upper left divided image 210 as a reference image. Referring to FIG. 8, upper right divided image 2
20, lower left divided image 230 and lower right divided image 240
Has been rotationally transformed by the same amount with respect to the upper left divided image 210 as the reference image. Further, the composite image shown in FIG. 8 is similar to the composite image shown in FIG.
0 and the subject image 201 are shown. However, the composite image shown in FIG. 8 has three divided images 220, 230, and 24.
0 is rotationally converted. For this reason, the three divided images 220, 230, 2
As a result, the quality of the image included in the effective image area 250 is reduced in a wide area.

On the other hand, the composite image shown in FIG.
Since only the upper left divided image 210 is rotationally converted, the area where the image quality is reduced is small, and the image quality of the image included in the effective image area 250 is prevented from being reduced.

The camera 100 according to the present embodiment
Is configured to capture the four divided images by moving the optical axis of the optical system with the imaging lens unit 121. However, a plurality of divided images captured by a normal camera or a plurality of divided images read by a scanner or the like are captured. You may make it input.

The camera 100 according to the present embodiment
In the stage before combining the divided images, the relative positional relationship of the divided images including the overlapping portions is calculated. Therefore, the relative position of the divided images including the overlapping portions by the combining of the images is calculated. Relationships do not change. Therefore, when detecting the relative positional relationship, it is not necessary to prevent the failure of the correlation detection or the mismatching.

Further, in the present embodiment, an example has been described in which image synthesis processing for synthesizing a divided image is performed by camera 100, but the divided image may be input to a personal computer and the processing may be performed on the personal computer. . In this case, the software for the above-described image synthesis processing is read from a recording medium such as a CD-ROM into a personal computer and executed.

The embodiments disclosed this time are to be considered in all respects as illustrative and not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

[Brief description of the drawings]

FIG. 1 is a front perspective view of a camera according to an embodiment.

FIG. 2 is a block diagram illustrating a schematic configuration of a camera according to the present embodiment.

FIG. 3 is a diagram illustrating an example of four divided images obtained by dividing and imaging with a camera according to the present embodiment.

FIG. 4 is a flowchart showing a flow of an image synthesizing process performed by the camera in the present embodiment.

FIG. 5 is a flowchart showing a flow of a coordinate conversion amount calculation process performed in step S4 of FIG. 4;

FIG. 6 is a flowchart showing a flow of coordinate conversion amount comparison processing and reference image determination processing performed in steps S5 and S6 of FIG. 4;

FIG. 7 is a diagram showing a result of performing a synthesizing process on the divided images shown in FIG. 3 by the camera according to the present embodiment.

8 is a diagram illustrating a result obtained by performing a combining process on the divided image illustrated in FIG. 3 using an upper left divided image 210 as a reference image.

[Explanation of symbols]

Reference Signs List 100 camera, 103 optical axis direction control unit, 105 actuator, 107 position sensor, 109 release switch, 111 program memory, 113 imaging sensor, 115 A / D conversion unit, 117 image memory, 11
9 removable recording medium, 121 imaging lens unit, 1
23 Recording media insertion slot, 201 subject image, 210
Upper left split image, 220 Upper right split image, 230 Lower left split image, 240 Lower right split image, 250 effective image area, OLab, OLac, OLba, OLbd, OLc
a, OLcd, OLdb, OLdc Standard overlap area.

Continued on the front page F-term (reference) 5B057 AA20 BA02 CA12 CA16 CB12 CB16 CC03 CD03 CE09 CE10 DA07 DB02 DC32 5C022 AA13 AB45 AB62 AB68 AC32 AC51 AC54 AC69 5C023 AA03 AA04 AA10 AA11 AA37 BA02 BA11 CA03 DA04 A08 AA ABA A

Claims (6)

[Claims] 1. An image synthesizing apparatus for synthesizing at least three divided images having overlapping portions, wherein a relative positional relationship between the at least three divided images is detected based on an image of the overlapping portion. Based on the detected positional relationship, the at least 3
An image synthesizing apparatus, comprising: a reference image determining unit that determines a reference divided image from one divided image; and an image combining unit that synthesizes another divided image based on the determined divided image. 2. The apparatus according to claim 1, wherein said position detecting means includes:
2. The image synthesizing apparatus according to claim 1, wherein a relative rotation shift amount between the two divided images is calculated. 3. The reference image determining means determines a reference divided image according to the number of divided images in which the relative rotational shift amount is equal to or less than a predetermined value.
The image synthesizing device according to claim 2. 4. The image synthesizing means includes a rotation converting means for rotationally converting the divided image, wherein the rotation converting means does not rotationally convert the divided image having the relative rotation shift amount equal to or less than the predetermined value. The image synthesizing apparatus according to claim 3, wherein: 5. An image synthesizing method for synthesizing at least three divided images having portions overlapping each other, wherein a relative positional relationship between the at least three divided images is detected based on an image of the overlapping portion. And at least 3 based on the detected positional relationship.
An image synthesizing method, comprising: determining a reference divided image from one divided image; and synthesizing another divided image based on the determined divided image. 6. A computer-readable recording medium storing an image synthesizing program for synthesizing at least three divided images having portions overlapping each other, wherein the at least three divided images are divided based on an image of the overlapping portion. Detecting a relative positional relationship between the images; and
An image synthesizing program for causing a computer to execute a step of determining a reference divided image from the one divided image and a step of synthesizing another divided image based on the determined divided image as a reference. Computer readable recording medium.