JP2006030014A - Color estimation system and color estimation method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To estimate a color with high precision when estimating the color of a subject using the spectral sensitivity characteristics of a camera measured in advance. <P>SOLUTION: The color estimation system comprises an image photographing unit 10 for photographing a color chart of which the spectral reflectance is known and the subject of which the spectral reflectance is unknown by a color image photographing device, a color-chart selection unit 18 for designating what color chart is used at the time of color-chart photographing, a color-chart spectral reflectance database 20 which stores spectral reflectance information of a plurality of color charts, and an image processing unit 22 for estimating the spectral reflectance or the color of the subject from a color image signal obtained by photographing the subject by the image photographing unit 10 using a signal value of the image of the color chart photographed by the image photographing unit 10, the spectral reflectance information of the color chart acquired from the color-chart spectral reflectance database 20 on the basis of the information on the color chart designated by the color-chart selection unit 18, and the spectral sensitivity characteristics of the color image photographing device. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a color estimation system and a color estimation method for estimating the spectral reflectance and color of a subject from an image taken by a multiband camera.

  For the purpose of acquiring an accurate colorimetric value of an object as an image, a method for estimating the spectral reflectance of an object from an image photographed by a multiband camera and performing color estimation has been proposed (for example, Patent Documents). 1). Here, when estimating the spectral reflectance of the subject from the image obtained by the multiband camera, the estimation is performed using the spectral sensitivity characteristics of each band of the camera and the spectral statistical information of the subject.

  Further, in a spectral radiance meter and a colorimeter that perform spot photometry, a method of measuring a white plate with a known spectral reflectance in advance and correcting the spectral sensitivity of the measuring device from the output signal value is performed. (For example, refer to Patent Document 2).

Furthermore, a method for estimating the spectral sensitivity of a camera by photographing a multicolor color chart whose spectral reflectance is already known has been proposed (see, for example, Patent Document 3).
Japanese Patent Application Laid-Open No. 11-85952 JP 2003-90761 A JP 11-96333 A

  However, in the color estimation method disclosed in Patent Document 1, if an error occurs in camera characteristics, for example, the spectral sensitivity characteristics of each band due to changes over time, an accurate color cannot be obtained.

  Therefore, if the spectral sensitivity correction method disclosed in Patent Document 2 is applied to the camera in the color estimation method disclosed in Patent Document 1, it is highly accurate for an object having a spectral reflectance close to that of the white plate. Color estimation is possible. However, the estimation accuracy often deteriorates for a subject that is significantly different from the above-mentioned white plate, and therefore high-accuracy data cannot be obtained depending on the subject.

  In the method disclosed in Patent Document 3 as well, accuracy is not guaranteed for a subject having spectral characteristics different from the photographed color chart. Therefore, in order to increase the accuracy of a fine color range, an enormous amount of color is required. It was difficult to implement a vote.

  The present invention has been made in view of the above points, and a color estimation system and a color estimation method capable of performing color estimation with high accuracy when performing color estimation of a subject using a spectral sensitivity characteristic of a camera measured in advance. The purpose is to provide.

  One aspect of the color estimation system according to the present invention includes a color image capturing device that captures a color chart with a known spectral reflectance and a subject with an unknown spectral reflectance, and spectral reflectance information of the color chart. Using the input means for inputting, the signal value of the color chart image taken by the photographing means, the spectral reflectance information of the color chart inputted by the input means, and the spectral sensitivity characteristics of the color image photographing apparatus And estimating means for estimating the spectral reflectance or color of the subject from the color image signal obtained by photographing the subject with the photographing means.

  According to another aspect of the color estimation system of the present invention, a color image capturing device uses a color chart with a known spectral reflectance and a photographing means for photographing a subject with an unknown spectral reflectance, Selection means for selecting or specifying spectral reflectance information of a color chart photographed by the photographing means from among a plurality of color charts stored in the storage means, and storing means for storing spectral reflectance information of the color chart Means, the signal value of the color chart image photographed by the photographing means, the spectral reflectance information selected or designated by the selection designation means, and the spectral sensitivity characteristic information of the color image photographing device. And estimating means for estimating the spectral reflectance or color of the subject from the color image signal obtained by photographing the subject with the photographing means.

  Still another aspect of the color estimation system of the present invention is that a color image photographing device uses a color image with a known spectral reflectance and a photographing means for photographing an object with a known spectral reflectance, Storage means for storing spectral reflectance information of the color chart, and a color chart to be photographed from among a plurality of color charts stored in the storage means based on the signal value of the subject image photographed by the photographing means Selection designation means for selecting or designating, a signal value of an image of the color chart obtained by photographing the color chart selected or designated by the selection designation means by the photographing means, and the above-mentioned stored in the storage means Obtained by photographing the subject by the photographing means using the spectral reflectance information of the color chart selected or designated by the selection designation means and the spectral sensitivity characteristic information of the color image photographing device. Characterized by comprising an estimating means for estimating the spectral reflectance or color of an object from a color image signal.

  In addition, according to one aspect of the color estimation method of the present invention, a color chart with a known spectral reflectance and a subject with a known spectral reflectance are photographed by a color image photographing device, and spectral reflectance information of the color chart is input. The color image using the signal value of the color chart image captured by the color image capturing apparatus, the spectral reflectance information of the input color chart, and the spectral sensitivity characteristics of the color image capturing apparatus. The spectral reflectance or color of the subject is estimated from a color image signal obtained by photographing the subject with the photographing device.

  Another aspect of the color estimation method of the present invention is that a color image capturing device captures a color chart with a known spectral reflectance and a subject with an unknown spectral reflectance, and uses a plurality of color charts prepared in advance. Spectral reflectance information is stored, and spectral reflectance information of the color chart photographed by the color image photographing apparatus is selected or designated from the stored plurality of color charts, and the color image photographing apparatus is used. A subject is photographed by the color image photographing device using the signal value of the photographed color chart image, the selected or designated spectral reflectance information, and the spectral sensitivity characteristic information of the color image photographing device. The spectral reflectance or color of the subject is estimated from the color image signal obtained in this way.

  According to still another aspect of the color estimation method of the present invention, a color image with a known spectral reflectance and a subject with an unknown spectral reflectance are photographed by a color image photographing device, and a plurality of color charts prepared in advance are obtained. The color chart to be photographed by the color image photographing device from among the plurality of stored color charts based on the signal value of the subject image photographed by the color image photographing device. And the signal value of the color chart image obtained by photographing the selected or designated color chart by the color image photographing device, and the stored spectrum of the selected or designated color chart. Using the reflectance information and the spectral sensitivity characteristic information of the color image photographing device, the spectral reflectance of the subject from the color image signal obtained by photographing the subject by the color image photographing device or Estimating a, characterized in that.

  According to the present invention, it is possible to provide a color estimation system and a color estimation method capable of performing color estimation with high accuracy when performing color estimation of a subject using spectral sensitivity characteristics of a camera measured in advance.

  The best mode for carrying out the present invention will be described below with reference to the drawings.

[First Embodiment]
FIG. 1A is a block diagram showing a functional configuration of the color estimation system according to the first embodiment of the present invention. This color estimation system is a system that takes a multispectral image of a subject and a color chart and outputs an image that reproduces the exact color of the subject under a desired observation illumination.

  That is, the color estimation system according to the present embodiment includes an image capturing unit 10, an image / data storage unit 12, an image / data output unit 14, an image / data storage unit 16, a color chart selection unit 18, a color chart spectral reflectance database. 20 and an image processing unit 22.

  Here, the image capturing unit 10 captures a multiband image of a subject whose spectral reflectance is not known or a color chart whose spectral reflectance is known. The image / data storage unit 12 stores a multiband image photographed by the image photographing unit 10 and a color reproduction image color-converted by an image processing unit 22 described later in detail. The image / data output unit 14 outputs the multiband image and the color reproduction image stored in the image / data storage unit 12 to an output device such as a monitor. The image / data storage unit 16 records / saves the multiband image and the color reproduction image stored in the image / data storage unit 12 in a predetermined format.

  The color chart selection unit 18 designates what color chart was used by the user interface when the image photographing unit 10 photographed the color chart. The color chart spectral reflectance database 20 stores spectral reflectance data of a plurality of color charts measured in advance by a spectral radiance meter or the like. Then, the image processing unit 22 performs the color chart based on the multiband image of the subject and the color chart stored in the image / data storage section 12 and the color chart information specified by the color chart selection section 18. Using the spectral reflectance data of the color chart acquired from the spectral reflectance database 20, the subject color is calculated or converted.

  2A and 2B are diagrams showing a specific configuration of the color estimation system according to the present embodiment. Here, as an example, a dental system is assumed, and FIG. 2A shows a state in which a camera 24 takes a picture of a person's vital teeth as a subject 26. A camera 24 in the figure corresponds to the image photographing unit 10 in FIG. 1 constitutes the image processing unit 22, the image / data output unit 14, the image / data storage unit 16, the color chart selection unit 18, and the color chart spectral reflectance database 20 in FIG. The image captured by the camera 24 is transferred to the PC 28 and stored, processed, output, or saved in the PC 28. FIG. 2B shows the state of color chart shooting. The image of the color chart 30 input by the camera 24 is transferred to the PC 28 and used for color estimation in the image processing section 22. .

FIG. 3 is a diagram showing the configuration of the camera 24 used in the present embodiment. That is, the camera 24 shown in the figure is a contact-type multiband camera, and the subject 26 is sequentially illuminated by the built-in multi-color LED 24A, and the light reflected from the subject 26 is received by the CCD 24B. The CCD control unit 24C acquires images in a frame sequential manner. However, normally, the built-in multicolor LED emits all colors to emit white light, and the image signal of the subject illuminated with white light is displayed live on the LCD monitor 24E via the image memory 24D. The user adjusts the position and focus of the subject 26 and the camera 24 while watching the image displayed live on the LCD monitor 24E, and releases the shutter. Then, the LED 24A sequentially turns on each color under the control of the LED control unit 24F, the subject 26 or the color chart 30 is photographed for each illumination of each color LED, and the acquired image signals are respectively stored in the image memory 24D and the I / F 24G. Is transferred to the PC 28 through the multi-band shooting of the subject or color chart. Note that the spectral sensitivity characteristic S _k (λ) (k = 1 to M, M: number of bands) of the contact type camera 24 as described above is the P (λ) sensitivity characteristic of the CCD 24B, and the emission spectrum of each color of the LED 24A. Is E _k (λ) (k = 1 to M),

It is represented by

  FIG. 1B is a diagram illustrating a detailed configuration of the image processing unit 22 in FIG. As shown in the figure, the image processing unit 22 includes a subject multiband image storage unit 22A, a camera spectral sensitivity data storage unit 22B, a color chart multiband image storage section 22C, a color chart spectral reflectance data storage section 22D, a spectral chart. It comprises a sensitivity data correction unit 22E, a subject spectral reflectance calculation unit 22F, an observation illumination light data storage unit 22G, a subject colorimetric value calculation unit 22H, a monitor color characteristic data storage unit 22I, and a monitor output signal value calculation unit 22J.

  That is, the subject multiband image storage unit 22A and the color chart multiband image storage unit 22C store the multiband photographed images of the subject 26 and the color chart 30 stored in the image / data storage unit 12 in FIG. Acquire and memorize. The camera spectral sensitivity data storage unit 22B stores the spectral sensitivity information of the camera 24 measured in advance. The color chart spectral reflectance data storage unit 22D stores the color chart 30 from the color chart spectral reflectance database 20 based on the information on the color chart 30 designated by the color chart selection unit 18 in FIG. Spectral reflectance data is acquired and stored. In the spectral sensitivity data correction unit 22E, the color chart multiband image stored in the color chart multiband image storage section 22C and the color chart spectrum stored in the color chart spectral reflectance data storage section 22D. Using the reflectance data, the camera spectral sensitivity data stored in the camera spectral sensitivity data storage unit 22B is corrected.

  Thereafter, in the subject spectral reflectance calculation unit 22F, from the camera spectral sensitivity data corrected by the spectral sensitivity data correction unit 22E and the subject multiband image stored in the subject multiband image storage unit 22A, the subject 26 spectral reflectance is calculated.

  Next, in the subject colorimetric value calculation unit 22H, the spectral reflectance of the subject 26 calculated by the subject spectral reflectance calculation unit 22F and the observation stored in advance in the observation illumination light data storage unit 22G are observed. The colorimetric value of the subject 26 under the illumination light is calculated using the spectrum data of the illumination light under the environment. Further, in the monitor output signal value calculation unit 22J, the colorimetric value of the subject 26 calculated by the subject colorimetric value calculation unit 22H and the monitor primary color stored in advance in the monitor color characteristic data storage unit 22I. A signal value for subject monitor output is calculated using information on chromaticity characteristics and gradation characteristics, and is stored in the image / data storage unit 12 as a monitor output image.

  Note that as the intermediate data in the above series of processing, the spectral reflectance of the subject 26 calculated by the subject spectral reflectance calculation unit 22F and the colorimetric value of the subject 26 calculated by the subject colorimetric value calculation unit 22H. Can also be stored in the image / data storage unit 12.

  FIG. 4 is a diagram showing an overall processing flow of the color estimation system according to the present embodiment.

  That is, first, in the color chart selection unit 18, the user selects a color chart 30 suitable for the subject 26 to be photographed thereafter, and designates information on the selected color chart 30 to the processing system (step S10). At this time, the user may visually select and select the color chart 30 of the color closest to the subject 26 of interest from among the plurality of color charts 30 prepared in advance, or as shown in FIG. In addition, if the subject 26 is limited in advance to living teeth, a coated plate, human skin, etc., a color chart designed exclusively for that subject may always be used. The designed color chart is, for example, standardly used as a color chart designed with an average value or a color sample of vital teeth from spectral reflectance measurement data of a large number of vital teeth. It is a color chart of a typical spectral reflectance such as a shade guide.

  Using the color chart 30 selected as described above, the selected color chart 30 is photographed by the camera 24 (image photographing unit 10) as shown in FIG. 2B (step S12). At this time, when shooting the color chart, the distance from the color chart to the camera is set to be the same as the distance from the subject to the camera at the time of subject shooting described later. It is better to match the illumination intensity with color chart photography. Next, after the color chart 30 is photographed, the image processing section 22 obtains it from the color chart spectral reflectance database 20 based on the photographed color chart image and the information specified by the color chart selection section 18. The spectral reflectance characteristic data of the color chart 30 is used to correct the camera spectral sensitivity characteristic data by a method described later (step S14). At this time, the spectral reflectance data registered in advance in the color chart spectral reflectance database 20 is not used, but the spectral reflectance design data or measurement data of the color chart 30 directly input from the user is used. Also good.

  Next, as shown in FIG. 2A, the subject 26 is photographed by the camera 24 (image photographing unit 10) (step S16). After photographing the subject, the image processing unit 22 estimates the spectral reflectance of the subject 26 using the photographed image of the subject 26 and the corrected spectral sensitivity characteristic data (step S18). Colorimetric value data of the subject 26 is calculated by multiplying the estimated spectral reflectance by a predetermined illumination light spectrum (step S20). As a method for estimating the spectral reflectance, for example, the method disclosed in Patent Document 1 is used. At this time, as the illumination light spectrum, spectrum data of a standard light source such as D65, A light source or the like determined by the CIE (International Lighting Commission) may be used, or spectrum data of typical illumination light measured in advance. Alternatively, spectrum data of actual illumination light separately measured by the user under the observation environment may be used.

  Next, the image processing unit 22 calculates a signal value for monitor output from the calculated colorimetric value of the subject 26 using information on the chromaticity characteristics and gradation characteristics of each primary color in the monitor for image observation. Then, it is stored in the image / data storage unit 12 as a monitor output image (step S22). Finally, from the monitor output image stored in the image / data storage unit 12, the image / data output unit 14 displays the subject 26 on the monitor (step S24). Note that the monitor output signal value may be calculated by performing color conversion based on the color characteristics of the monitor to be output, for example, by the method disclosed in Patent Document 1 described above.

  Here, the spectral sensitivity characteristic data correction method described above will be described in detail using mathematical expressions.

The spectral sensitivity characteristics of the k-th band (k = 1 to M, M: number of bands) of the multiband camera before correction are S _k (λ) (λ = 380 to 780, λ: wavelength), and the color chart used for photographing If the spectral reflectance of 30 is c (λ), and the signal value of the k-th band in the corresponding pixel in the multiband image of the photographed color chart 30 is g _k , the corrected spectral sensitivity characteristic is ,

It becomes. here,

It is.

  The above description is a spectral sensitivity correction method in the case of a multi-band camera for close-up photography as shown in FIGS. 2B and 3, but the present embodiment is not limited to this.

For example, as shown in FIG. 6, the color chart 30 (and the subject 26) is exposed to external illumination light 32, and a filter turret 34 capable of switching various types of filters is attached to the front surface of the camera 24 '. In the case of shooting with a multiband camera, the same correction can be made. In this case, the spectral sensitivity characteristic S _k (λ) of the camera 24 ′ as defined below is used instead of the above-described equation (1), and the correction according to the above equations (2) and (3) is performed. Just do it.

Here, E _OUTSIDE (λ) is the spectrum of the external illumination light 32, F _k (λ) is the spectral transmittance characteristic of the kth filter among the plurality of filters attached to the front of the camera, and P (λ) is the camera. It is a sensitivity characteristic in an internal CCD. Note that the multiband camera shown in FIG. 6 may use a wavelength tunable filter using liquid crystal instead of the filter turret 34.

  In the first embodiment described above, the number of bands M of the multiband camera is not particularly limited. However, the greater the number of bands, the higher the color estimation accuracy. Of course, the method described above can also be applied to a conventional RGB camera (M = 3).

  As described above, according to the first embodiment, a plurality of color charts 30 having different spectral reflectances are prepared in advance, and for each subject 26 to be photographed, a color chart having a spectral reflectance close to that subject 26. By selecting and photographing 30 and correcting the spectral sensitivity characteristics, it is possible to realize highly accurate color estimation for various subjects 26. Alternatively, the spectral sensitivity is corrected using a color chart designed with characteristics close to the spectral reflectance of the subject (life teeth, paint plate, human skin, etc.) that is assumed in advance. Color estimation can be performed with high accuracy. As described above, the spectral sensitivity can be corrected to include absolute brightness by matching the distance between the subject shooting and color chart shooting and matching the illumination intensity. As a result, the color estimation of the subject can be obtained with high accuracy as an absolute amount.

[Second Embodiment]
Next, a color estimation system according to the second embodiment of the present invention will be described with reference to FIGS.

  FIG. 7 is a functional configuration diagram of the color estimation system according to the second embodiment. That is, the color estimation system of the present embodiment is obtained by adding an area designating unit 36 to the configuration shown in FIG. 1A in the first embodiment described above, as shown in FIG.

  The area designation unit 36 is a user interface for designating a partial area in the multiband image of the subject 26 photographed by the image photographing unit 10. The user can designate a partial area of the subject 26 to be measured through the area designation unit 36. At this time, in the present embodiment, the image processing unit 22 performs colorimetry only within the designated area designated by the user in the multiband image of the subject 26.

  FIG. 8 shows a functional configuration of the image processing unit 22 in the present embodiment. As shown in the figure, the image processing unit 22 in the present embodiment is obtained by adding an in-region pixel average value calculation unit 22K to the configuration shown in FIG. 1B in the first embodiment described above. . For simplification of the drawing, the monitor color characteristic data storage unit 22I and the monitor output signal value calculation unit 22J are not shown in FIG.

  The intra-region pixel average value calculation unit 22K is based on the attention region information specified by the region specification unit 36 described above from the multiband image of the subject 26 stored in the subject multiband image storage unit 22A. A region is cut out, and an average value of multiband pixel values is obtained in the cut out region. The average value calculated by the in-region pixel average value calculation unit 22K is then used as a value for calculating the spectral reflectance in the subject spectral reflectance calculation unit 22F. The spectral reflectance data calculated by the subject spectral reflectance calculation unit 22F is output to the image / data storage unit 12 as it is, or based on the illumination light spectrum in the observation environment. Colorimetric value data is calculated and output to the image / data storage unit 12.

  FIG. 9 is a diagram showing a flow of processing in the color estimation system according to the present embodiment. Here, the process (step S16) from the step (step S10) in which the user selects and specifies the color chart 30 in the color chart selection unit 18 until the subject 26 is photographed by the camera 24 (image photographing unit 10) is described above. This is the same as the process shown in FIG. 4 in the first embodiment.

  In this embodiment, after photographing the subject 26 in step S16, the image / data output unit 14 displays a multiband image of the photographed subject 26 on the monitor (step S30). The user refers to the displayed multiband image and designates the area of the subject 26 to be measured by the area designation unit 36 (step S32).

  Thereafter, the image processing unit 22 estimates the spectral reflectance of the subject 26 in the region based on the pixel value (average value) of the multiband image based on the designated region (step S18), and also observes illumination. The colorimetric values under light are calculated (step S20), and the estimation / calculation results are output as data or displayed in a graph by the image / data output unit 14 (step S34).

  As described above, the type of the subject 26 can be categorized in more detail by designating the user's attention region and calculating the spectral reflectance and the colorimetric value by the region designating unit 36, so that it is close to the characteristics of the subject. When the color chart is selected, the optimum color chart 30 can be selected, and the calculation accuracy can be improved.

[Third Embodiment]
Next, a color estimation system according to a third embodiment of the present invention will be described with reference to FIGS.

  As shown in FIG. 10, the color estimation system of the present embodiment includes a subject specifying unit 38 and a subject basis function database 40 in addition to the system configuration of FIG. 6 in the second embodiment described above.

  Here, the subject designation unit 38 is a user interface for designating information (life teeth, dentures, etc.) regarding the type of the subject 26 photographed by the image photographing unit 10. The subject basis function database 40 stores spectral reflectance basis functions corresponding to the type of the subject 26. Then, the image processing unit 22 in the present embodiment acquires the basis function of the subject 26 from the subject basis function database 40 by designating specific information of the subject 26 to be measured through the subject designation unit 38 by the user. Then, by using it as statistical information in calculating the spectral reflectance, the colorimetric accuracy can be improved.

  FIG. 11 shows a detailed configuration of the image processing unit 22 in the present embodiment. That is, in the image processing unit 22 of the present embodiment, a subject basis function data storage unit 22L is added to the configuration of the image processing unit 22 in the second embodiment described above. In order to simplify the drawing, the monitor color characteristic data storage unit 22I and the monitor output signal value calculation unit 22J are not shown in FIG.

  Here, the subject basis function data storage unit 22L acquires and stores the basis function of the subject 26 from the subject basis function database 40 based on the category information of the subject 26 specified by the subject specifying unit 38 described above. is there. Using the stored basis function of the subject 26 as statistical information, the subject spectral reflectance calculation unit 22F calculates the spectral reflectance based on, for example, the method described in Patent Document 1.

  FIG. 12 is a diagram showing a specific processing flow in the color estimation system according to the present embodiment. Here, the process from the user selecting and specifying the color chart 30 in the color chart selection unit 18 (step S10) to the process of selecting the attention area of the subject 26 (step S32) are the same as those in the second embodiment described above. This is the same as the process shown in FIG.

  In this embodiment, after designating the attention area of the subject 26 in step S32, the subject designation unit 38 designates specific information regarding the subject 26 in the attention area (step S40). Then, the image processing unit 22 calculates the spectral reflectance of the subject 26 in the region of interest based on the specified information (step S18), and calculates the colorimetric value under the observation illumination light ( In step S20), the image / data output unit 14 outputs the estimation and calculation results as data or displays a graph (step S34).

  As described above, the user can specify more specific statistical information when calculating the spectral reflectance of the subject 26 by designating the target region of the subject 26 to be measured and specific information on the subject 26 in the target region. Therefore, the colorimetric accuracy can be improved.

[Fourth Embodiment]
Next, a color estimation system according to the fourth embodiment of the present invention will be described with reference to FIGS.

  In the present embodiment, as shown in FIG. 13, the color chart 30 is photographed using the color chart 30 having uniform color characteristics in the range of the photographing range 42 or larger photographed by the multiband camera. Using the photographed color chart image 44, the sensitivity unevenness and illumination unevenness of the camera 24 are corrected.

  The basic configuration of the color estimation system according to this embodiment is the same as the configuration shown in FIG. 1A in the first embodiment described above. However, in this embodiment, the functional configuration of the image processing unit 22 is different.

  FIG. 14 is a diagram illustrating a detailed functional configuration of the image processing unit 22 in the present embodiment. As shown in the figure, in this embodiment, in addition to the configuration shown in FIG. 1B in the first embodiment described above, a color unevenness correction unit 22M and a corrected multiband image storage unit 22N are provided.

  Here, the color unevenness correction unit 22M performs color unevenness of the multiband image of the subject 26 based on the multiband image obtained by photographing the color chart 30 having a size larger than the angle of view as described above. Is to correct. The corrected multiband image storage unit 22N stores a multiband image of the subject 26 corrected by the color unevenness correction unit 22M. In the present embodiment, the subject spectral reflectance calculation unit 22F calculates the spectral reflectance of the subject 26 using the multiband image stored in the corrected multiband image storage unit 22N. .

  FIG. 15A is a diagram showing the flow of processing in the color estimation system according to this embodiment. Here, the process (step S16) from the step (step S10) in which the user selects and specifies the color chart 30 in the color chart selection unit 18 until the subject 26 is photographed by the camera 24 (image photographing unit 10) is described above. This is the same as the process shown in FIG. 4 in the first embodiment.

In this embodiment, after photographing the subject 26 in step S16, the image processing unit 22 uses the color chart image 44 photographed in advance in step S12 to cause uneven color of the photographed image of the subject 26. Correct. That is, the signal value of the k band of the subject image at the j th pixel position is o (j, k), and the signal value of the k band of the color chart image 44 at the j th pixel position is c (j, k). If the signal value of the k-th band of the color chart image 44 in the pixel at the center of the image is c ₀ (k), the subject image o ′ (j, k) after color unevenness correction is

It becomes.

  As described above, according to the color estimation system according to the present embodiment, the color chart 30 having uniform color characteristics as a calibration color chart in the range of the photographing range 42 or larger photographed by the multiband camera. By using this, it is possible to correct the color unevenness of the subject image caused by the shading of the camera 24 and the unevenness of the illumination light, and perform accurate color estimation.

[Fifth Embodiment]
Next, a color estimation system according to the fifth embodiment of the present invention will be described with reference to FIGS.

  The color estimation system according to the present embodiment captures the subject 26 before photographing the color chart 30 with a multiband camera, pre-estimates the spectral reflectance of the subject 26 in advance, and calculates the spectral reflectance estimated before that. Color chart calibration is performed by selecting the optimum color chart 30 again.

  The overall functional configuration of the color estimation system according to the present embodiment is the same as the configuration illustrated in FIG. 7 in the second embodiment described above.

  FIG. 15B shows the overall processing flow in the color estimation system according to this embodiment.

  That is, in the present embodiment, first, the subject 26 is photographed by the image photographing unit 10 (step S16), and the multiband image of the photographed subject 26 is displayed on the monitor by the image / data output unit 14 (step S30). ). Then, after the user designates the attention area by the area designation section 36 (step S32), the image processing section 22 once estimates the spectral reflectance in the attention area of the subject 26 as preprocessing (this is referred to as pre-estimation). (Step S60). Note that the spectral sensitivity characteristic data used at this time is spectral sensitivity characteristic data before correction stored in the image processing unit 22 in advance. For example, the method disclosed in Patent Document 1 may be used as the spectral reflectance estimation method.

  Next, based on the spectral reflectance data estimated in this way, the color chart selection unit 18 performs spectral reflection of a plurality of color charts 30 (color charts “1” to “4” in the figure) as shown in FIG. Of the rates, the color chart 30 (color chart "2" in the example shown) having the spectral reflectance closest to the previously estimated spectral reflectance 46 is automatically selected (step S62). The selected color chart number is presented in the image / data output unit 14 (step S64), and the user sets the color chart 30 of the presented number in front of the camera 24 (image capturing unit 10). Then, photographing is performed (step S66).

  Thus, after photographing the color chart 30 of the automatically selected / presented number, the camera spectral sensitivity characteristic data is corrected in the image processing unit 22 (step S14), and the spectral reflectance of the subject 26 is estimated (step S18). Colorimetric value data is calculated (step S20). Then, the image / data output unit 14 outputs the estimation and calculation results as data or displays a graph (step S34).

  FIG. 17 shows the detailed flow of processing for the color chart selection processing in step S62.

  That is, first, “1” is initially set to the variable n indicating the color chart number (step S62A). Then, the color chart spectral reflectance data for the nth color chart 30 is acquired from the color chart spectral reflectance database 20 (step S62B). Next, a mean square error (RMSE) between the acquired color chart spectral reflectance and the spectral reflectance 46 previously estimated of the subject 26 is calculated (step S62C). Then, it is determined whether or not the calculated mean square error is within a predetermined value range (step S62D).

  If it is determined in step S62D that the calculated mean square error is not within the predetermined value range, whether or not the color chart number n is equal to N, that is, is stored in the color chart spectral reflectance database 20. It is determined whether or not the processing has been completed for all N color chart spectral reflectance data that are present (step S62E). If there is still an unprocessed one, the color chart number n is incremented by “+1” (step S62F), and the process returns to step S62B to process the color chart spectral reflectance data in the next color chart. repeat.

  If it is determined in step S62D that the calculated mean square error is within a predetermined value range, this process is terminated. As a result, the nth color chart 30 is selected.

  If it is determined in step S62E that the color chart number n is equal to N, that is, there is no unprocessed color chart spectral reflectance data, it is determined that there is no corresponding color chart spectral reflectance data. Exit.

  As described above, the color chart 30 whose mean square error with respect to the spectral reflectance of the subject 26 is within a predetermined range is automatically selected. Here, if there is no color chart 30 prepared in advance that satisfies the above conditions, the one with the smallest mean square error is obtained from the mean square errors obtained for all the color charts 30. Use it. Alternatively, the predetermined value used for the determination in step S62D may be changed, and the corresponding color chart spectral reflectance data may be selected by repeating the process in step S62.

  FIG. 18 is a diagram showing a detailed functional configuration of the image processing unit 22 in the present embodiment. As shown in the figure, in this embodiment, in addition to the configuration of the image processing unit 22 shown in FIG. 8 in the second embodiment described above, a subject spectral reflectance pre-estimation unit 22O and a color chart evaluation unit 22P are provided. It is a thing.

  Here, the subject spectral reflectance pre-estimation unit 22O estimates the spectral reflectance of the region of interest of the subject 26 as described above. The color chart evaluation unit 22P compares the pre-estimated spectral reflectance with the color chart spectral reflectance acquired from the color chart spectral reflectance database 20, and evaluates the color chart 30. Then, the color chart evaluation unit 22P sends information about the color chart 30 that is optimum as a result of the evaluation to the color chart selection unit 18.

  As described above, the subject 26 is photographed first, and then the color chart 30 closest to the spectral reflectance of the subject 26 is selected and pre-estimated, and the color chart is photographed, so that the optimum color chart for the characteristics of the subject 26 is obtained. 30 can be used for calibration, so highly accurate color estimation can be realized. Further, it is possible to make the selection more accurately and easily by the automatic selection than the color chart selection by the user's visual observation.

  Although the above description is based on the premise that the color chart 30 is selected from a plurality of color charts 30 prepared in advance, the present embodiment is not limited to this.

  For example, as shown in FIG. 19, based on the previously estimated spectral reflectance, a color chart of a color having the spectral reflectance characteristic closest to the spectral reflectance is output by a printer 48 or the like, and this is used as a color chart 30. It may be used. In this case, in order to output a color chart having a desired spectral reflectance, color adjustment of the printer 48 is performed by a method disclosed in, for example, Japanese Patent Laid-Open Nos. 10-157206 and 2002-98590. Need to do. Alternatively, a color chart that can obtain similar spectral characteristics may be displayed on the monitor instead of the printer 48, and the displayed color chart may be used as a color chart 30 for photographing.

  Although the present invention has been described above based on the embodiments, the present invention is not limited to the above-described embodiments, and various modifications and applications are naturally possible within the scope of the gist of the present invention.

  For example, in the first to fifth embodiments, the spectral reflectance of the subject 26 is calculated after correcting the camera spectral sensitivity data based on the photographed color chart image 44 and the stored color chart spectral reflectance data. However, the captured subject image itself is corrected based on the photographed color chart image 44 and the stored color chart spectral reflectance data, and the spectral reflectance of the subject 26 is calculated. Also good.

  That is, as shown in FIG. 20, as the image processing unit 22, in the configuration shown in FIG. 1B in the first embodiment described above, a multiband image correction unit 22Q is provided instead of the spectral sensitivity data correction unit 22E. . The multiband image correcting unit 22Q includes a color chart multiband image stored in the color chart multiband image storage section 22C and a color chart spectral reflectance stored in the color chart spectral reflectance data storage section 22D. Using the data, the subject multiband image stored in the subject multiband image storage unit 22A is corrected. In this case, the subject spectral reflectance calculation unit 22F includes the subject multiband image corrected by the multiband image correction unit 22Q and the camera spectral sensitivity data stored in the camera spectral sensitivity data storage unit 22B. From the above, the spectral reflectance of the subject 26 is calculated.

  FIG. 20 corresponds to the first embodiment, but it is needless to say that the present invention can be applied to the second to fifth embodiments in the same manner.

(A) is a block diagram showing a functional configuration of the color estimation system according to the first embodiment of the present invention, (B) is a detailed functional configuration of the image processing unit in the color estimation system according to the first embodiment. FIG. (A) is a figure which shows the mode of object imaging | photography in the color estimation system which concerns on 1st Embodiment, (B) is a figure which similarly shows the mode of color chart imaging | photography. It is a figure which shows the structure of the camera used in the color estimation system which concerns on 1st Embodiment. It is a figure which shows the flowchart for demonstrating the flow of the whole process of the color estimation system which concerns on 1st Embodiment. It is a figure which shows the spectral reflectance distribution of the color chart designed for dentistry used in the color estimation system which concerns on 1st Embodiment. It is the figure which showed the mode of the multiband imaging | photography of the color chart in case there exists external illumination light in the color estimation system which concerns on 1st Embodiment. It is a block diagram which shows the functional structure of the color estimation system which concerns on 2nd Embodiment of this invention. It is a block diagram which shows the detailed function structure of the image process part in the color estimation system which concerns on 2nd Embodiment. It is a figure which shows the flowchart for demonstrating the flow of the whole process of the color estimation system which concerns on 2nd Embodiment. It is a block diagram which shows the functional structure of the color estimation system which concerns on 3rd Embodiment of this invention. It is a block diagram which shows the detailed function structure of the image process part in the color estimation system which concerns on 3rd Embodiment. It is a figure which shows the flowchart for demonstrating the flow of the whole process of the color estimation system which concerns on 3rd Embodiment. It is a figure which shows the relationship between the color chart used for the color estimation system which concerns on 4th Embodiment of this invention, and the image | photographed color chart image. It is a block diagram which shows the detailed function structure of the image process part in the color estimation system which concerns on 4th Embodiment. (A) is a figure which shows the flowchart for demonstrating the flow of the whole process of the color estimation system which concerns on 4th Embodiment, (B) is the whole color estimation system which concerns on 5th Embodiment of this invention. It is a figure which shows the flowchart for demonstrating the flow of a typical process. It is a figure which shows the spectral reflectance distribution of the object pre-estimated in 5th Embodiment, and the spectral reflectance of the several color chart used as a use candidate. FIG. 16 is a diagram illustrating a flowchart for explaining a detailed processing flow of a color chart selection step in FIG. It is a block diagram which shows the detailed function structure of the image process part in the color estimation system which concerns on 5th Embodiment. It is a figure which shows the usage example of the color estimation system which concerns on 5th Embodiment. It is a block diagram for demonstrating the modification of the image process part in the color estimation system which concerns on 1st Embodiment.

Explanation of symbols

    DESCRIPTION OF SYMBOLS 10 ... Image pick-up part, 12 ... Data storage part, 14 ... Data output part, 16 ... Data preservation | save part, 18 ... Color chart selection part, 20 ... Color chart spectral reflectance database, 22 ... Image processing part, 22A ... Subject multi Band image storage unit, 22B ... Camera spectral sensitivity data storage unit, 22C ... Color chart multiband image storage unit, 22D ... Color chart spectral reflectance data storage unit, 22E ... Spectral sensitivity data correction unit, 22F ... Subject spectral reflectance calculation 22G ... Observation illumination light data storage unit, 22H ... Subject colorimetric value calculation unit, 22I ... Monitor color characteristic data storage unit, 22J ... Monitor output signal value calculation unit, 22K ... Intra-region pixel average value calculation unit, 22L ... Subject basis function data storage unit, 22M ... color unevenness correction unit, 22N ... post-correction multiband image storage unit, 22O ... subject spectral reflectance presumption , 22P ... color chart evaluation unit, 22Q ... multiband image correction unit, 24, 24 '... camera, 24A ... LED, 24B ... CCD, 24C ... CCD control unit, 24D ... image memory, 24E ... LCD monitor, 24F ... LED control unit, 24G ... I / F, 26 ... subject, 28 ... PC, 30 ... color chart, 32 ... external illumination light, 34 ... filter turret, 36 ... area designation unit, 38 ... subject designation unit, 40 ... subject base Function database, 42 ... shooting range, 44 ... color chart image, 46 ... previously estimated spectral reflectance, 48 ... printer.

Claims (18)

A color image photographing device for photographing a color chart whose spectral reflectance is known and a subject whose spectral reflectance is unknown;
Input means for inputting spectral reflectance information of the color chart;
Using the signal value of the color chart image photographed by the photographing means, the spectral reflectance information of the color chart inputted by the input means, and the spectral sensitivity characteristics of the color image photographing device, the photographing means An estimation means for estimating the spectral reflectance or color of a subject from a color image signal obtained by photographing the subject;
A color estimation system comprising: The estimation means is
Correction means for correcting spectral sensitivity characteristics of the color image photographing device from the signal value of the color chart image photographed by the photographing means and the spectral reflectance information of the color chart inputted by the input means;
Color estimation means for estimating the spectral reflectance or color of the subject from the color image signal obtained by photographing the subject by the photographing means based on the spectral sensitivity characteristic of the color image photographing device corrected by the correcting means. When,
The color estimation system according to claim 1, further comprising: The estimation means is
A color image signal obtained by photographing the subject by the photographing means based on the signal value of the color chart image photographed by the photographing means and the spectral reflectance information of the color chart inputted by the input means. Correction means for correcting
Color estimation means for estimating the spectral reflectance or color of the subject from the color image signal corrected by the correction means based on the spectral sensitivity characteristics of the color photographing device;
The color estimation system according to claim 1, further comprising: A color image photographing device for photographing a color chart whose spectral reflectance is known and a subject whose spectral reflectance is unknown;
Storage means for storing spectral reflectance information of a plurality of color charts prepared in advance;
A selection / designating means for selecting or designating spectral reflectance information of the color chart photographed by the photographing means from a plurality of color charts stored in the storage means;
Using the signal value of the color chart image photographed by the photographing means, the spectral reflectance information selected or designated by the selection designation means, and the spectral sensitivity characteristic information of the color image photographing device, the photographing is performed. Estimating means for estimating the spectral reflectance or color of a subject from a color image signal obtained by photographing the subject by means;
A color estimation system comprising: The estimation means is
Correction means for correcting the spectral sensitivity characteristic information of the color image photographing device from the signal value of the color chart image photographed by the photographing means and the spectral reflectance information selected or designated by the selection / designating means; ,
A color for estimating the spectral reflectance or color of the subject from the color image signal obtained by photographing the subject by the photographing unit based on the spectral sensitivity characteristic information of the color image photographing device corrected by the correcting unit. An estimation means;
The color estimation system according to claim 4, further comprising: The estimation means is
A color image signal obtained by photographing the subject by the photographing means based on the signal value of the color chart image photographed by the photographing means and the spectral reflectance information selected or designated by the selection / designating means. Correction means for correcting
Color estimation means for estimating the spectral reflectance or color of the subject from the color image signal corrected by the correction means based on the spectral sensitivity characteristic information of the color photographing device;
The color estimation system according to claim 4, further comprising: A color image photographing device for photographing a color chart whose spectral reflectance is known and a subject whose spectral reflectance is unknown;
Storage means for storing spectral reflectance information of a plurality of color charts prepared in advance;
A selection designating unit for selecting or designating a color chart to be photographed from among a plurality of color charts stored in the storage unit based on the signal value of the subject image photographed by the photographing unit;
The signal value of the color chart image obtained by photographing the color chart selected or designated by the selection designation means by the photographing means, and the color selected or designated by the selection designation means stored in the storage means Using the spectral reflectance information of the vote and the spectral sensitivity characteristic information of the color image photographing device, the spectral reflectance or color of the subject is estimated from the color image signal obtained by photographing the subject by the photographing means. An estimation means;
A color estimation system comprising: The estimation means is
The spectral sensitivity characteristic information of the color image photographing apparatus is corrected from the signal value of the color chart image photographed by the photographing means and the spectral reflectance information of the color chart selected or designated by the selection / designating means. Correction means;
A color for estimating the spectral reflectance or color of the subject from the color image signal obtained by photographing the subject by the photographing unit based on the spectral sensitivity characteristic information of the color image photographing device corrected by the correcting unit. An estimation means;
The color estimation system according to claim 7, further comprising: The estimation means is
Obtained by photographing the subject by the photographing means based on the signal value of the color chart image photographed by the photographing means and the spectral reflectance information of the color chart selected or designated by the selection / designating means. Correction means for correcting the color image signal;
Color estimation means for estimating the spectral reflectance or color of the subject from the color image signal corrected by the correction means based on the spectral sensitivity characteristic information of the color photographing device;
The color estimation system according to claim 7, further comprising:   10. The color chart according to claim 1, wherein the color chart has a size equal to or larger than an angle of view photographed by the color image photographing device, and has a uniform spectral reflectance characteristic within a photographed range. The color estimation system described in any one.   The apparatus further includes color unevenness correcting means for correcting color unevenness in the spatial direction of the color image signal obtained by photographing the subject by the photographing means using the signal value of the color chart image photographed by the photographing means. The color estimation system according to claim 10.   11. The color estimation according to claim 1, further comprising attention area designation means for designating a part of the attention area in the color image signal obtained by photographing the subject by the photographing means. system.   13. The color estimation system according to claim 1, wherein the estimation unit uses a basis function of a spectral reflectance of a subject imaged by the imaging unit.   14. The color estimation system according to claim 1, wherein the color chart has substantially the same spectral reflectance as that of the subject. Using a color image photographing device, a color chart with a known spectral reflectance and a subject with a known spectral reflectance are photographed,
Enter the spectral reflectance information of the above color chart,
Using the signal value of the color chart image captured by the color image capturing apparatus, the spectral reflectance information of the input color chart, and the spectral sensitivity characteristics of the color image capturing apparatus, the color image capturing apparatus Estimate the spectral reflectance or color of the subject from the color image signal obtained by photographing the subject
A color estimation method characterized by the above. Using a color image photographing device, a color chart with a known spectral reflectance and a subject with a known spectral reflectance are photographed,
Store spectral reflectance information of multiple color charts prepared in advance,
Select or specify spectral reflectance information of the color chart photographed by the color image photographing device from the plurality of stored color charts,
Using the signal value of the color chart image captured by the color image capturing device, the selected or designated spectral reflectance information, and the spectral sensitivity characteristic information of the color image capturing device, the color image capturing is performed. Estimate the spectral reflectance or color of the subject from the color image signal obtained by photographing the subject with the device.
A color estimation method characterized by the above. Using a color image photographing device, a color chart with a known spectral reflectance and a subject with a known spectral reflectance are photographed,
Store spectral reflectance information of multiple color charts prepared in advance,
Based on the signal value of the subject image photographed by the color image photographing device, the color chart to be photographed by the color image photographing device is selected or designated from the plurality of stored color charts,
The color chart image signal value obtained by photographing the selected or designated color chart by the color image photographing device, the stored spectral reflectance information of the selected or designated color chart, The spectral reflectance or color of the subject is estimated from the color image signal obtained by photographing the subject with the color image photographing device using the spectral sensitivity characteristic information of the color image photographing device.
A color estimation method characterized by the above.   The color estimation method according to claim 15, wherein the color chart has substantially the same spectral reflectance as that of the subject.

Images