JP2003101690A - Image processing method, digital camera, and recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image-processing method for representing the impression of a three-dimensional world that a human vision would recognize, a digital camera using the method, and a record medium that has recorded an image processing program for achieving a function for performing image processing using the method. SOLUTION: Digital images acquired optically and mechanically via an optical lens are deformed by image-processing information that is preset based on human visual psychological characteristics, and generate digital images close to an impression when a person sees the subject. The image processing information, based on the human vision psychological characteristics is preset based on data that are measured by a visual psychological experiment.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing method, and more particularly to an image processing method for processing digital image information obtained optically, a digital camera using the method, and image processing using the method. The present invention relates to a recording medium recording an image processing program for realizing the function to be performed.

[0002]

2. Description of the Related Art Generally, a digital camera spatially samples image information through an optical lens using a two-dimensional photoelectric conversion element such as a charge coupled device (CCD) and stores it as digital data. Although it has the function of saving and displaying it on a liquid crystal display, etc., the image acquired through the optical lens is an optical and mechanical image, has distortion aberration due to the optical lens, and is recognized by humans. It may be very different from the image.

Distortion aberration caused by an optical lens becomes particularly large in a small-sized popular digital camera in which the aperture of the optical lens used is small, but a known technique (for example, Japanese Unexamined Patent Application Publication No. 2000-242242)
2000-004391, JP 09-294225, JP
08-305841 gazette) can be used for calibration. Usually, this calibration is performed based on the data obtained by the calibration experiment of the optical lens. For example, a flat plate on which a grid pattern of a known size is drawn is used, and the pattern on this flat plate is photographed at different depths. Then, the distortion characteristic of the lens can be estimated from the measured image information.

An image recognized by a human is an image in which a human brain restores a three-dimensional world in real time from two pieces of two-dimensional image information captured by both eyes, and is an optical image obtained through an optical lens. It can be very different from a mechanical image. Therefore, various prior arts of digital cameras have been disclosed for the purpose of bringing the image closer to an image recognized by humans by applying the technique of digital image processing. For example, a technology (Japanese Unexamined Patent Publication No. 2000-090233) that conveys a texture, a three-dimensional effect, etc. by performing processing including movement and rotation of an object in consideration of color, shape of illumination light, illumination angle, arrangement, etc. While acquiring the depth information of the subject from multiple image information taken continuously,
A technique for creating a wide-angle image of a target direction or a pan-focus image (Japanese Patent Laid-Open No. 10-229566), a plurality of pieces of image information about the same image pickup target, and a plurality of pieces of image information are integrated by image differentiating means or the like. A technique for obtaining one integrated image (Japanese Patent Laid-Open No. 09-116855) is disclosed.

However, it cannot be said that these conventional techniques have sufficiently achieved the purpose of approaching an image recognized by humans. That is, no technique has been proposed for generating a digital image (photograph) close to the impression perceived by a person by digital image processing in consideration of human visual psychological characteristics. Furthermore, by image conversion processing considering visual psychological characteristics,
There is no technical attempt to express the impression of a three-dimensional scene perceived by humans.

As a typical phenomenon of human visual psychology, there is size constancy with respect to an object placed in a three-dimensional world. The size constancy is, for example, a characteristic that even if the same object is placed at a deep depth (far from the observer), it looks small, and if it is near, it looks large. Various clues such as three-dimensional information (stereo vision, motion vision, shadow, perspective projection expression,
(Knowledge, etc.) is acquired, and it is caused by adjusting the appearance of the size of an object in the brain. Related to this, the horizontal-vertical illusion (the characteristic of the visual perception that the horizontal direction is shorter and the vertical direction is longer than the actual length), and the homeostasis illusion of size (even for a figure drawn on a two-dimensional plane, a three-dimensional When depth cues are given around a figure, the constancy of visual size works, and even figures of the same size are perceived as different sizes).

These visual psychological characteristics are known in the field of cognitive science and visual psychology, and the homeostatic illusion of size and the Ponzo illusion (the length of a line segment in the area between two diagonal lines). Experiments have also been conducted to quantitatively measure the amount of optical illusion, such as the optical illusion. However, the comparison experiment with the monocular case where the size constancy illusion depends on the visual field direction and the depth cues is not sufficient. Moreover, the problem of horizontal and vertical illusions and the amount of illusions related to the apparent size of the full moon near the horizon, known as the so-called "moon illusion", has not been solved.

[0008]

SUMMARY OF THE INVENTION The present invention has been made in view of the above situation, and an image processing method capable of expressing an impression of a three-dimensional world perceived by human vision, a digital camera using the method, and the same. An object of the present invention is to provide a recording medium recording an image processing program for realizing a function of performing image processing using the method.

[0009]

In order to achieve the above object, an image processing method of the present invention has an image processing information based on a preset human visual psychological characteristic, and an optically acquired digital image. This is an image processing method for transforming information into image information based on the image processing information, and generating an image that approximates the impression when a human sees the subject.

Further, the image processing method of the present invention is an image processing method in which the image processing information based on the human visual psychological characteristic is image processing information set based on data measured in a visual psychological experiment.

Further, the visual psychological characteristic of the human is an image processing method which is a characteristic including at least one of a constancy illusion of size, a horizontal / vertical illusion of a visual field, and a direction dependency of depth perception at least.

The image processing method of the present invention also transforms the optically acquired digital image information into an image that approximates the impression when a human sees the subject based on the data measured in the visual psychology experiment. For presetting image processing information based on human visual psychological characteristics, optically acquiring digital image information of a subject, and preset image of digital image information of the subject optically acquired It is an image processing method including a step of transforming the image information into image information based on the processing information and generating an image that approximates the impression that a human sees the subject.

A digital camera according to the present invention is a digital camera using the image processing method according to the present invention, which comprises input means for acquiring image information as digital data via an optical lens, and an image for storing the digital image information. Information storage means, image processing information storage means for storing image processing information, and image processing is performed on the primary image information stored in the image information storage means based on the image processing information stored in the image processing information storage means. Image processing means for creating image information,
A digital camera having an output unit that outputs the image information stored in the image information storage unit in a desired format, and image processing information based on human visual psychological characteristics is preset in the image processing information storage unit. It is a thing.

Further, the image processing information has an operating means including a function of setting a correction parameter for correcting the strength of deformation, and when the correction parameter is set by the operating means, image processing information based on a preset human visual psychological characteristic. It is a digital camera having a function of correcting the standard deformation strength according to (1) to the deformation strength corrected by the set correction parameter.

The recording medium of the present invention is a computer-readable recording medium recording an image processing program for realizing the function of performing image processing using the image processing method of the present invention.

[0016]

BEST MODE FOR CARRYING OUT THE INVENTION According to the present invention, a digital image optically or mechanically acquired through an optical lens is transformed by image processing information preset based on human visual psychological characteristics, and the human subject An image processing method for generating a digital image closer to the impression when viewed, a digital camera using the method, and a recording medium recording an image processing program for realizing a function of performing image processing using the method. However, there are various kinds of human visual psychological characteristics. First, the human visual psychological characteristics intended by the present invention will be described.

The human brain recognizes the three-dimensional world by restoring the three-dimensional world in real time from two pieces of two-dimensional image information captured by both eyes. At that time, the human brain 1) converts an inverted retinal image into an erect image, and 2) the retinal image may have an area in which pillars and the like are obliquely reflected, but a three-dimensional image of a pillar standing vertically. We also perform image processing such as converting to, but these reproduce the actual physical three-dimensional world,
The image processing and image modification for recognition are not modifications based on the human visual psychological characteristics intended by the present invention.

The human visual psychological characteristics intended by the present invention are:
It is a visual psychological characteristic of human beings who perceive differently from the actual physical three-dimensional world, and although this is also various, for example, one of them is the distance from the eye if the physical world has the same size. It is a characteristic that the size of the retinal image recovers and is recognized in the same size (absolute size) in the perceptual world even if the size becomes large and the size of the retina becomes small. That is, the perception of the size by the visual sense is not only determined by the visual angle captured through the lens of the lens of the eye and the size on the retina, but also by other stimulation variables (the structure of the visual space such as depth information). It means that

Specific examples of the structure of the visual space that influences the homeostasis are 1) the difference in whether the space is homogeneous or has a high degree of segmentation, 2) whether the observation direction is horizontal or vertical, and 3). The three-dimensional positional relationship between an object (such as the moon) at the center of interest and an object to be compared.

A specific example of the inhomogeneity of the visual space is the horizontal and vertical illusion. As described above, the horizontal-vertical illusion is a visual characteristic that the horizontal length is shorter and the vertical length is longer than the actual length, and is also related to the next direction dependency of depth perception.

The direction dependency of depth perception is such that the world in which a person lives can look over 100 km or more in the horizontal (horizontal) direction of the line of sight, while it is approximately 10 km (clouds) in the vertical direction. It is said that the anisotropy of the depth perception characteristic comes from the fact that we have only recognized the world up to the tropospheric zone where it occurs. In other words, the world that people feel is considered to be a flat world such as an ellipsoid that is long in the horizontal direction and short in the vertical direction (the geometrical world view that humans perceive). It is considered to bring about the direction dependency of perception.

Other human visual psychological characteristics intended by the present invention include, for example, the illusion of the moon in which the moon rising to the horizon or the mountain edge looks large, the Ponzo illusion, the Mueller-Lear illusion (which is sandwiched between outward arrow blades). The axis (line segment) looks short, the axis sandwiched by the inward arrow wings looks long, and the perspective illusion (on a screen with strong perspective projection, the size of an object located far away is greatly recognized). is there.

As described above, the human visual psychological characteristics intended by the present invention are various, but the present invention is not required to have all the characteristics as image processing information. It is possible to set and execute the image processing information based on the selected visual psychological characteristics of the human, such as only the horizontal and vertical illusions, or the direction dependency of the horizontal and vertical illusions and depth perception.

Next, the outline of the embodiment of the image processing method of the present invention will be described step by step.

In the present invention, first, it is necessary to preset image processing information based on human visual psychological characteristics, but as described above, the characteristics are not absolutely or numerically fixed.
In the present invention, data measured in advance for a plurality of people in visual psychological experiments is used. At that time, human visual psychological characteristics include age, nationality, growing environment, sex, whether or not eyeglasses are used,
Since there is an element that changes depending on the blood type, etc., data containing some human parameters such as age is used, and for example, a form in which this human parameter can be set by the operating means of the digital camera of the present invention is used. It may be adopted, or all average data not including human parameters such as age may be used.

In any case, it is unavoidable that an individual difference remains, so that, for example, the operating means of the digital camera of the present invention,
It is desirable to have a function of setting a correction parameter for correcting the strength of deformation so that a preset standard deformation strength can be corrected by the set correction parameter. With this configuration, the photographer can appropriately adjust the deformation strength according to his / her impression.

In the visual psychology experiment, for example, in a size constancy illusion experiment, the depth at which the object is placed, the angle from the view line (upper and lower), the color of the object (hue, saturation, lightness), the object The experiment is performed by changing the shape of the object, the presence / absence of the comparison object, the depth of the comparison object, etc., and one or more main stimulus variables that affect the human visual psychological characteristics are extracted, and the data having the stimulus variables are imaged. Set as processing information. That is, the image transformation based on the human visual psychological characteristic of the present invention is performed by using the stimulus variable obtained from the digital image information obtained through the optical lens or the like or set by the operating means as a parameter.

As described above, the image processing method of the present invention is
As the human visual psychological characteristics, it is possible to set and implement image processing information based on the selected human visual psychological characteristics, such as horizontal and vertical illusions only, or horizontal and vertical illusions and the direction dependence of depth perception. When setting a plurality of characteristics, for example, in the operating means of the digital camera of the present invention,
It is desirable to have a function of selecting whether or not to use the plurality of characteristics so that it is possible to select and use arbitrary characteristics. With this configuration, the photographer can obtain an image that is closer to his or her impression.

It is desirable that the optically acquired digital image information is subjected to image processing for calibrating the distortion aberration based on the optical lens before the image deformation based on the human visual psychological characteristic of the present invention. For example, it is desirable that the distortion aberration calibration is performed as the first image processing, and the image information subjected to the distortion aberration calibration is subjected to the image deformation based on the human visual psychological characteristic as the second image processing. Since the correction of the distortion aberration based on the optical lens can be performed by the above-mentioned conventional technique or the like and does not limit the present invention, detailed description thereof will be omitted.

The stimulus variables for image deformation differ depending on the specific characteristics of visual psychology, but the most common stimulus variable is depth information (disparity information). The depth information may be manually input from an operating means based on human knowledge and experience, but the principle of binocular stereoscopic vision or motion stereoscopic vision (which is a known technique) from a plurality of image information ( The form extracted by the above-mentioned JP-A-10-229566, etc.) is preferable. It should be noted that the operation of converting the size of the target object in the visual field based on the depth information can be rephrased as photographing using a lens having a different focal length depending on the three-dimensional position (depth) of the target. In other words, this corresponds to an operation of selecting a telephoto ratio from a large number of images taken by a camera having a continuously variable telephoto ratio according to the depth information and reconstructing into one image.

In the case of the manual input mode, for example, the depth information of each pixel in the image is manually specified in 16 steps for the input image, and the depth information of each pixel input by this manual is specified. Is used to refer to image processing information set in advance on the basis of human visual psychological characteristics, and image processing such as enlargement / reduction of each pixel is executed.

In the case of extracting from a plurality of image information, for example, a template matching method (a region similar to an image of an appropriate (characteristic) local region in one image is selected from the other image). It is preferable to use a method of searching) or a method of hierarchically changing the image resolution (image size), repeating the template matching method sequentially from an image with a low resolution, and then globally matching and then locally matching. Can be used for Furthermore, it is preferable to use a regularization method (March's method and Yokoya's method) that simultaneously optimizes the matching error and the continuity of the parallax obtained by calculation. At this time, the smoothing digital filter is used. , Resample (resample) the smoothed image with reduced resolution
By doing so, the image size can be reduced and the calculation time can be shortened.

A plurality of image information for extracting the depth information can be input in various forms. For example, a stereo digital camera having a plurality of input means including an optical lens or a multi-lens digital camera can be input at the same time. It is also possible to perform continuous shooting while moving with a video camera and input. Further, in a monaural digital camera, for example, as in the above-described conventional technique, continuous shooting may be performed while inputting, or the optical lens may be rotatable in a plane perpendicular to the optical axis, or the optical axis may be changed. It is also possible to perform translational movement in the vertical direction, and it is also possible to input as a plurality of continuous digital image information by an automatic shutter function linked with rotation or translation.

That is, the digital camera of the present invention can be embodied in the form of a monaural digital camera, a stereo digital camera, a multi-lens digital camera, a video camera, or the like.

The embodiment has been described above with respect to the method of extracting depth information (disparity information), which is a stimulus variable that is almost common to image deformation, but the stimulus variable of image deformation is, as described above, visual psychology. It is different for each specific characteristic.For example, in the optical illusion of the moon, it is desirable to use the elevation angle of the optical camera as one of the stimulation variables. It is desirable to be attached. That is, in the digital camera of the present invention, naturally, the input means of the stimulus variable which needs to be attached may differ depending on the specific visual psychological characteristics to be selected and used.

Next, regarding the digital camera of the present invention,
The embodiment will be specifically described.

As described above, the digital camera of the present invention transforms a digital image optically or mechanically acquired through the optical lens by the image processing information set in advance based on the human visual psychological characteristic, It is a digital camera that has the function of generating a digital image that approximates the impression that a human sees the subject, and preferably has a mechanism such as an optical lens, an imaging plate (CCD, etc.), focus adjustment, aperture adjustment, and shutter. An image pickup unit (input unit) including the image information storage unit (image information storage unit) for storing digital image information captured by the image pickup unit, a processing information storage unit (image processing information storage) for storing image processing information Means), based on the image processing information stored in the processing information storage unit, image processing is performed on the primary image information stored in the image information storage unit to create secondary image information, and the control of each unit is performed. A control unit (image processing unit) for performing the image display, a display unit (output unit) including a liquid crystal display for outputting the image information stored in the image information storage unit in a desired format, a shutter operation by a photographer, and photographing It can be configured to include an operation unit (operation means) for performing an operation such as mode switching. Note that the image information storage unit and the processing information storage unit may use the same memory and divide the usage area to be shared, but for example, the image information storage unit is RA
M, the processing information storage unit is a ROM, and it is desirable to provide them separately.

In the processing information storage unit, for example, image processing information for performing distortion aberration correction is preset as first image processing information, and image processing for performing image deformation based on human visual psychological characteristics. As information, image processing information based on visual psychological characteristics of a human selected from a database in which data measured in advance for a plurality of humans in visual psychological experiments is appropriately set as second image processing information in advance.

The digital image information acquired by the image pickup section is stored in the image information storage section as original image information, and distortion correction is performed based on the original image information based on the first image processing information stored in the processing information storage section. Then, the calibration image information that has been subjected to this distortion aberration calibration is stored in the image information storage unit, and subsequently, based on the second image processing information and the like stored in the processing information storage unit, the calibration image information Deformation image information is created by performing image transformation based on visual psychological characteristics. In addition,
The calibration image information may be stored by overwriting the original image information, or the original image information may be left and stored in another area. Similarly, the modified image information can be overwritten or stored in another area.

In the case of a monaural digital camera having an optical lens rotatable in a plane perpendicular to the optical axis or capable of translational movement in a direction perpendicular to the optical axis, for example, it is interlocked with the rotation or translation of the optical lens. With the automatic shutter function, the continuous digital image information of multiple sheets is input as the original image information, and the calibration image information of the plurality of original image information that has been subjected to distortion aberration calibration is used for binocular stereoscopic vision and motion stereoscopic Based on the principle of vision, the depth information of the object in the image is estimated, and the estimated depth information of each pixel is used as a parameter based on the second image processing information. Perform image processing. The generated deformed image is output and displayed on a liquid crystal display, for example.

In a stereo digital camera equipped with a stereo optical lens or the like, for example, two image pickup units including two optical lenses, two color CCD two-dimensional photoelectric conversion devices, two analog / digital converters, etc. are provided. Two-dimensional imaging (sampling) of video at the same time with two imaging units
In addition, with a built-in small CPU (control unit),
The parallax of each pixel is calculated by the above-mentioned parallax information extraction method, etc., and the absolute value of the depth is estimated by taking into consideration the device constants (focal length of optical lens, absolute size of 1 pixel of CCD, etc.), and each obtained pixel Can be configured to execute the enlargement / reduction processing of each pixel size using the depth estimation value of the.

At this time, the processing for changing the object size in the image is executed for either one or both of the two images obtained through the two optical lenses. When performing on one image, only one geometrically deformed image is output. For example, the geometrically deformed image based on human visual psychological characteristics is displayed on the liquid crystal display in one screen. In the case of executing both images, for example, the processed results are displayed side by side on the liquid crystal display as two converted images, and stereo is obtained by utilizing the ability of human's binocular stereoscopic vision (3D vision). It can also be visible.

The display unit of the digital camera of the present invention includes, for example, a liquid crystal display and the like, and the control unit corresponds to the setting from the operation unit, for example, and the original image, the calibration image, and the modified image are processed according to the processing procedure. It is preferable that the images are sequentially displayed on the display unit, or corresponding to the setting from the operation unit, so that the images can be sequentially displayed retroactively.

The present invention also uses a plurality of images captured by rotating or translating a stereo camera in a plane perpendicular to the optical axis, and using the spatial coordinates (ie, viewpoint) of the stereo lens, By acquiring the three-dimensional information from that position, various forms of images can be generated.

For example, in the case of a landscape photograph, this is a close view,
It is possible to obtain a multi-view stereo image that adopts the technique of landscape painting, which is divided into a middle view and a distant view, with the near view looking up and the distant view looking down. That is, a multi-view stereo image can be obtained by selecting the human visual psychological characteristics to be applied from the stereo images having different shooting heights according to the difference in depth.

Further, it is also possible to synthesize a multi-view CG image by synthesizing images from a stereo image captured from a plurality of viewpoints with reference to depth information. Furthermore, a stereo moving image can be obtained by using the information obtained by the stereo matching of each of the continuous stereo images and sequentially expressing it as a three-dimensional CG on the display.

According to such an embodiment, in the image processing method of the present invention, a digital image optically or mechanically acquired through an optical lens is transformed by image processing information preset based on human visual psychological characteristics. However, the present invention provides an image processing method for generating a digital image that resembles the impression when a human sees the subject, and provides an image processing method that can express the impression of the three-dimensional world recognized by human vision.

The digital camera of the present invention is an example of an apparatus capable of suitably carrying out the image processing method of the present invention, and the image processing method of the present invention having the above-described embodiment is used to detect a human image. The recording medium of the present invention is a digital camera capable of expressing the impression of a three-dimensional world that is visually perceived, and the recording medium of the present invention is perceived by human vision by using the image processing method of the present invention having the above-described embodiment. It is a computer-readable recording medium recording an image processing program for realizing a function of expressing an impression of a three-dimensional world.

[0049]

Embodiments of the present invention will be described below with reference to the drawings.

First, regarding an image based on human visual psychological characteristics that can be used in the image processing method of the present invention,
The concept will be described by taking various illusions as examples. Figure 1
FIG. 3 is a diagram conceptually showing a typical illusion, which is a three-dimensional view showing (a) vertical and horizontal illusions, (b) Mueller-Rear illusion, and (c) Ponzo illusion. It is an illusion that is also associated with the illusion or upward hypertropia. These optical illusions can be considered to be the information processing characteristics of human beings obtained as a result of living on the earth while recognizing the three-dimensional world. Note that each specific illusion amount is about 20 to 100%, although there are individual differences as described above.

Next, as a first embodiment, an image processing method for transforming into an image based on human visual psychological characteristics will be described.
A case in which the homeostasis illusion of size and the horizontal and vertical illusions are related to each other will be specifically described step by step.

First, it was carried out in order to determine the image processing information for transforming into an image based on human visual psychological characteristics.
The results of the visual psychology experiment will be described.

FIG. 2 shows three types of presentation figure sizes (S,
(M, L) is the result of a visual psychological experiment on the depth dependence of the apparent size, where the horizontal axis is the distance (cm) from the subject and the vertical axis is the size at the reference position. It is a relative size based on the standard. This experiment is the result when the presentation position of the figure to be compared in the bright field experiment is brought close to from the distance, and the reference position (196.5
It was done by having the figure size on the other monitor at a different depth position adjusted so that it would be the same size as the figure displayed on the monitor (CRT) placed at a distance of (cm). FIG. 2 shows that a far object recognizes relatively large (120 to 140%) and a near object small (60 to 80%) with respect to the object placed at the reference position. That is, even within the range of this experiment, human beings perceive that the size of the object is different from the size of the object at least twice.

Next, a method of acquiring depth information will be described.

In the case of two images taken by a stereo camera, 1) parallax information d (pixels) is obtained by using a stereo matching method which is an improved template matching method.
In this case, it is assumed that the parallax lines of the left and right images match each other (epipolar constraint), and the correspondence of the template images is searched only on the parallax line, whereby the parallax information can be obtained by a simple method. . 2) The parallax information is basically in units of pixels (pixels), but it is also possible to determine the minimum value of the parallax information with accuracy of sub-pixel unit by performing curve interpolation of the evaluation value for determining the degree of coincidence with the template. Is. 3)
The three-dimensional depth information D (m) is given by the following equation.

[0056]

[Equation 1]

Here, h (m) is the focal length of the camera, L
(M) is the distance between camera lenses (baseline length), I (pixel /
m) is a device constant (the reciprocal of the size of one pixel on the photoelectric element).

When the depth information is obtained from the viewpoint of motion stereoscopic vision using a monaural camera, instead of the parallax information d, the apparent speed (optical flow) v in the moving image is obtained.
It is given by equation (2) using (pixels / frame).

[0059]

[Equation 2]

Here, v ₀ (m / s) is the moving speed of the camera, and F (frames / s) is the sampling frequency of the moving image.

Next, a method of deforming an image based on the depth information will be described.

The pixel enlargement ratio M (x, y) can be expressed as a function of the depth information D, for example, by the following equation.

[0063]

[Equation 3]

Here, P ₁ is the minimum value of the enlargement ratio, P ₂ is the width of the enlargement ratio, and P ₃ is a parameter representing the change of the enlargement ratio,
Each parameter value is determined with reference to the result of the visual psychological experiment.

In the expression (3), for convenience of image conversion expression, the conversion expression is given only by the expansion expression without using the reduction expression. Further, α is an amount calculated from the depth information, and is +1 in the part where the distance is the maximum and − in the part where the distance is the minimum.
The value is linearly converted (normalized) to be 1.

The image is reconstructed based on the obtained pixel enlargement ratio. At that time, it is desirable to draw the luminance value of the conversion target pixel (x, y) at the coordinates obtained by the following equation.

[0067]

[Equation 4]

Here, (C _x , C _y ) is the coordinates of the vanishing point on the image, and when it is difficult to automatically obtain the vanishing point from the image, for example, the optical axis of the camera and the image plane are It can be easily obtained by selecting the intersection.

From the above relational expressions, the effect of the horizontal and vertical illusions can be regarded mainly as upward hypervision, and can be expressed by the following expression, for example. In the following equation, V is the vertical size of the image, and P ₄ is the enlargement ratio (1.0 or more) of the image top line.

[0070]

[Equation 5]

Next, as a second embodiment, a method of realizing the lunar illusion, which is one of the main illusions that can be used in the image processing method of the present invention, will be described.

The illusion of the moon is as shown in the conceptual diagram of FIG.
This is an illusion that the rising moon (or sun) and the setting sun (sun) look larger than the moon (sun) near the zenith, and these are known for a long time. This is a typical example in which the impression of the size when shooting is greatly different. There have been many theories about this problem, such as comparison theory with ground objects, constancy of size, vertical and horizontal illusion, Ponzo illusion, and even perceived sky shape, but they have not been solved yet.

In the image processing method of the present invention of the present invention, the illusion of the moon is determined based on, for example, the result of a visual psychological experiment as shown in FIG. ,
This can be achieved by assuming the equation (5) and performing geometric transformation of the image. That is, the image conversion of the illusion of the moon can be realized by changing the enlargement ratio according to the difference in the depth information and pasting the enlarged and reduced images in each pixel unit. In this case, based on the theory of sky shape, an ellipsoidal shape whose distance is closer to the top is empirically (for example, empirically about 10 km directly above (cloud height in the troposphere), about 100 km in the horizontal direction (horizontal direction when the weather is good) It is possible to assume a view)). Also, if there is occlusion,
Although there is a portion where the depth information cannot be calculated, this portion can be solved by referring to the depth information of neighboring pixels and adding the texture of the shielded portion.

Next, as a third embodiment, using the conversion formula shown in formula (3), the enlargement / reduction ratio of each pixel size is calculated from the depth information at each point in the image, and the geometrical ratio is calculated. A specific example in which such image conversion processing is executed will be described.

FIG. 4 is a stereo original image (simulation image, (a) left image, (b) right image), theoretical value of depth information ((c) obtained from the original image using the stereo matching method). FIG. 6 is a six-sided view showing a parallax image) and a transformed image ((d), (e), (f)) deformed based on visual psychological characteristics in comparison. Incidentally, (d), (e), numerical values under (f) is, (minimum value of the enlargement ratio) P ₁ (3) below, (the width of the enlargement ratio) P _2, P ₃ (magnification), And the numerical value of P ₄ (magnification ratio of the top line of the image) in the equation (7) is shown.

The transformed image (d) is a transformed image in consideration of the homeostatic illusion of the size in the depth direction, and the deeper the depth (smaller parallax) is, the larger the size of the object is represented. ing. The transformed image (e) is a geometrical image that considers the anisotropy (upper hypertrophic) of the homeostatic illusion of the size in the vertical direction of the camera's viewing line along with the homeostatic illusion of the size in the depth direction. It has been converted. Converted image (f) is similar to the transformed image (e), is a transformed image in consideration of the above exaggerated with homeostasis illusion depth direction size, (7) Expansion of P ₄ (image top line in formula Since the distortion of the screen tends to increase as the ratio increases,
The results obtained by performing the conversion by the following formula instead of the formula (6) are shown by comparison.

[0077]

[Equation 6]

Incidentally, regarding these conversion characteristics, in consideration of the difference in visual psychological characteristics due to individual differences, the operating means is provided with a function for the photographer to select whether or not to use the plurality of characteristics, and It is desirable to have a function to adjust the strength of deformation.

According to the embodiment described in detail above, for example, depth information (disparity information) acquired from a stereo image.
Based on the depth of the image, human vision (images recognized by the brain)
It is possible to provide an image close to the impression captured by. In a normal telephoto lens, all the objects in the image captured by the lens are magnified, whereas in the image conversion of the present invention, the magnification amount is changed according to the depth amount. In addition, by applying the sky shape theory and the concept of upward hyperparalysis (visual psychology field), the shape of the rising moon and the shapes of mountains and buildings in the distant view and near view are transformed into images that approximate the visual impression. can do. Although the embodiments of the present invention have been described above, it is obvious that various changes may be made in the form and details without departing from the spirit and scope of the present invention defined in the claims. . For example, in the above-described embodiment, the explanation was centered on the image conversion technique based on the depth information of the illusion as the human visual characteristic (such as size constancy and upward hypervision) and the stereoscopic binocular depth information. This is one approach for converting to a close image, and various variations are conceivable in the depth information acquisition method and the impression image conversion method, and the present invention is not particularly limited thereto.

[0080]

As described above in detail, the present invention is
A digital image obtained by transforming a digital image obtained optically and mechanically through an optical lens by image processing information set in advance based on human visual psychological characteristics to bring the human to the impression when looking at the subject. Is a recording medium storing an image processing method for generating an image, a digital camera using the method, and an image processing program for realizing a function of performing image processing using the method, and a three-dimensional world recognized by human vision. There is an effect that can express the impression of. Also,
By obtaining 3D information in the form of a stereo digital camera, etc., it is possible to obtain multi-view stereo images, multi-view CG images, stereo moving images, etc.

[Brief description of drawings]

FIG. 1 is a conceptual diagram for explaining an example of an illusion based on human visual psychological characteristics, and is a three-dimensional view of (a) vertical and horizontal illusions, (b) Mueller-Rier illusion, and (c) Ponzo illusion. is there.

FIG. 2 is a result of a visual psychological experiment performed to determine image processing information for transforming into an image based on human visual psychological characteristics, showing the visual psychological experiment regarding the depth dependence of apparent size. It is an example of a result.

FIG. 3 is a conceptual diagram for explaining an illusion of the moon.

FIG. 4 is an embodiment of the present invention, which is a stereo original image ((a) left image, (b) right image), depth information ((c) parallax image) acquired from the original image, and visual psychological characteristics. FIG. 6 is a six-sided view showing a comparison of converted images ((d), (e), (f)) deformed based on FIG.

─────────────────────────────────────────────────── ─── Continuation of front page (51) Int.Cl. ⁷ Identification code FI theme code (reference) // H04N 101: 00 H04N 101: 00 (72) Inventor Takeshi Kinoshita 2-16-1 Tokiwadai, Ube City, Yamaguchi Prefecture Issue Yamaguchi University Faculty of Engineering (72) Inventor Makoto Ichikawa 2-16-1 Tokiwadai, Ube City Yamaguchi Prefecture Yamaguchi University Faculty of Engineering F-term (reference) 5B050 AA09 BA06 BA15 CA07 DA02 DA04 EA13 FA02 5B057 BA02 CA08 CA12 CA16 CB08 CB12 CB16 CC01 CD11 CH08 5C022 AA13 AC69 5C061 AA06 AA21 AB03 AB04 AB08 5C062 AA01 AB17 AB40 AC21 AE01 BB03

Claims (7)

[Claims] 1. Having image processing information based on a preset human visual psychological characteristic, digitally acquired digital image information is transformed into image information based on the image processing information, and a human sees the subject. An image processing method, which is characterized in that an image close to the impression when it is generated is generated. 2. The image processing information based on the human visual psychological characteristics is image processing information set based on data measured in a visual psychological experiment.
The described image processing method. 3. The human visual psychological characteristic is a characteristic including at least one of constancy illusion of size, horizontal / vertical illusion, and direction dependency of depth perception at least. The image processing method according to claim 2. 4. For transforming optically acquired digital image information into an image that approximates the impression when a human sees the subject, based on the data measured in a visual psychology experiment,
A step of presetting image processing information based on human visual psychological characteristics; a step of optically acquiring digital image information of a subject; and a preset image processing of the digital image information of the subject acquired optically. 4. The image processing method according to claim 1, further comprising a step of transforming the image information into image information based on the information, and generating an image that approximates an impression when a human sees the subject. . 5. An input means for acquiring image information as digital data via an optical lens, an image information storage means for storing digital image information, an image processing information storage means for storing image processing information, and the image processing. An image processing unit that applies image processing to the primary image information stored in the image information storage unit based on the image processing information stored in the information storage unit to create secondary image information; and an image processing unit stored in the image information storage unit. 2. An output unit for outputting image information in a desired format, wherein image processing information based on human visual psychological characteristics is preset in the image processing information storage unit. A digital camera using the image processing method according to any one of 4 above. 6. An image based on a preset human visual psychological characteristic, which has an operating means including a function of setting a correction parameter for correcting the deformation intensity, and when the correction parameter is set by the operating means. 6. The digital camera according to claim 5, further comprising a function of correcting a standard deformation strength according to the processing information to a deformation strength corrected by the set correction parameter. 7. A computer-readable recording medium recording an image processing program for realizing a function of performing image processing by using the method according to claim 1.