JP4247362B2 - Feature extraction device - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an apparatus for optically analyzing and extracting and classifying characteristics of a sample to be inspected, in particular, a paper surface of a paper sheet on which characters, design patterns, and the like are printed.
[0002]
[Prior art]
For banknotes, securities, stamps, and other items that use paper as a medium to demonstrate the value of exchange, authenticity identification is always a major problem. In distinguishing between authenticity and the difference, it is necessary to properly understand what the characteristics of the paper sheet are.
[0003]
In recent years, color copiers, color printers, scanners, etc. have been improved in performance and reduced in price, and it has become easy to induce counterfeiting of paper sheets. For this reason, markings that are not included in commercially available color copying toners and printer inks, such as fluorescent materials and other functional materials, are performed under normal lighting conditions and invisible to the naked eye. . These markings naturally constitute the characteristics of paper sheets.
[0004]
In the paper sheet authenticity identification device described in Japanese Patent Laid-Open No. 2001-52232, a specific part of the paper to be identified is irradiated with ultraviolet light, and transmitted light or reflected light from the ultraviolet light irradiation part is applied to a wavelength selection filter to specify a specific part. Only the light in the wavelength band is received by the photo sensor array, the authenticity of the paper quality of the paper to be identified is identified based on the light output signal of the photo sensor array, and the authenticity of the phosphor that generates fluorescence by ultraviolet irradiation is identified. When both the identification results are true, the identified paper is true.
[0005]
The paper sheet authenticity discrimination device discriminates whether or not the paper used for the paper sheet is genuine by the light output signal of the photosensor. However, since this apparatus is for confirming the already recognized optical characteristics and not for checking the printed figure on the paper, it is difficult to say that it is a perfect method for authenticating paper sheets.
[0006]
In the document inspection apparatus described in Japanese Patent Laid-Open No. 4-288977, at least two light portions having different wavelengths are directed on a common area of a document, and reflected light or transmitted light is detected by a linear detector.
[0007]
The document inspection apparatus can use a printed figure as an inspection standard. Further, reflected light or transmitted light is detected by a plurality of detectors, and signals coming from the detectors are evaluated alone or in an appropriate combination, and multifaceted inspection is possible. However, it is difficult to say that sufficient technical pursuit has been made as to how data obtained by a plurality of detectors are related and processed to improve the function of the inspection apparatus.
[0008]
An apparatus for measuring the optical characteristics of a sample in a wide wavelength band, such as a spectrophotometer, is well known, but only a spot-like measurement area is set, and it is difficult to measure characteristics over the entire paper surface. Further, it merely shows the measured value under the measurement conditions at the time of measurement. Although there is an apparatus that picks up an optical image of a sample paper with a plurality of wavelengths using a CCD camera or the like, the search wavelength band is narrow (400 to 1000 nm). Moreover, only a sample surface image is obtained, and feature extraction from an image corresponding to each search wavelength is not performed.
[0009]
Therefore, for the search and the feature extraction of the optical characteristics of the sample paper over a wide wavelength band, an imaging device composed of a plurality of types of cameras with different imaging wavelength bands, and a large number of captured images obtained for each wavelength from these types of cameras. It is necessary to have a computing device that extracts features for the target and classifies these features. For example, if 7200 images of about 1 million pixels are picked up from a wide wavelength band and features of all the images are extracted, enormous labor and time are required. Furthermore, when these captured images have different camera characteristics such as brightness and density gradient, feature extraction is difficult. In addition, even if the optical properties of a reference sample obtained from a large number of samples and a new sample are judged to be different, it is difficult to carry out the comparative judgment work under such circumstances, and sufficiently satisfactory results are obtained. I couldn't.
[0010]
[Problems to be solved by the invention]
The present invention can extract optical characteristics of a sample in a wide wavelength band not limited to the visible light region, and can perform centralized processing of the obtained data, and can efficiently use system resources of a computing device. An object is to provide an apparatus and a feature extraction system.
[0011]
[Means for Solving the Problems]
The feature extraction apparatus of the present inventionA sample holder for holding a sample, irradiation means for sequentially irradiating the sample with a plurality of types of light having wavelengths in a range from the ultraviolet region to the near-infrared region and having different wavelengths in stages, and the sample An imaging device that captures an image of the sample surface in a plurality of imageable wavelength bands having different center wavelengths and acquires an image corresponding to each wavelength band, and extracts the optical characteristics of the sample from the captured image; A feature extraction device that obtains representative images of a plurality of images, wherein the imaging devices have different wavelength bands that can be imaged from each other, each has a unique photosensitivity characteristic, and a part of the imageable wavelength band of a single camera Consists of a plurality of cameras that overlap with a part of the imageable wavelength band of the other one camera to form a wavelength overlap region, and a white reference point and a darkness indicating a brightness level in the captured image of the sample holder Black indicating level Provide a quasi-point, and within the wavelength overlap region, brightness of the white level value of the white reference point and the black level value of the black reference point captured by both cameras having the wavelength overlap region, and between the two reference points Computational imaging means for sequentially obtaining camera image data while performing scaling correction calculation of the grayscale value of the captured image at the same rate so that the density gradient has a degree of approximation equal to or higher than a predetermined level is provided..
[0012]
According to this configuration, it is possible to search for optical characteristics in a wide wavelength band by a plurality of cameras. Further, even when switching from one camera to the other, the difference in images is small, and the optical characteristics of these images can be extracted by the same image processing method. Therefore, even if there are a plurality of cameras, the image processing software and the data storage unit can be unified, and system resources can be saved and the processing speed can be increased.
[0013]
A sample is held in a sample holder, the sample is irradiated with inspection light generated by the inspection light generator, and the sample surface is imaged in a predetermined different wavelength band by the imaging device, and images corresponding to these wavelength bands To getTherefore, it is possible to reliably acquire images for each wavelength band necessary for extracting the optical characteristics of the sample..
[0014]
  The white reference point indicating the brightness level and the black reference point indicating the darkness level are provided in the captured image of the sample holder, and the white image captured by both cameras having the wavelength overlap region in the wavelength overlap region The gray level value of the captured image is subjected to an enlargement / reduction correction operation at the same rate so that the brightness of the white level value of the reference point and the black level value of the black reference point and the density gradient between the two reference points are close to a predetermined level or higher. However, since the camera image data is obtained sequentially, the dynamic range of each captured image is uniform, and variations in sensitivity of the imaging means (in terms of brightness and density gradient) can be eliminated..
[0015]
In the present invention,In the feature extraction apparatus having the above-described configuration, a pre-captured image display unit that performs pre-photographing under a predetermined photographing condition and displays the captured image at the time of characteristic photographing of the sample surface, and an image displayed on the pre-captured image display unit Reference point coordinate setting input means for setting the coordinates of the white reference point and the black reference point is provided, and the brightness of the black and white reference point and the density gradient between the two reference points, the coordinates of which are set by the setting input means. It is characterized by being used as an index for the same-scale enlargement / reduction correction operation of the captured image gray value obtained thereafter..
[0016]
  According to this configuration,Since the white reference point and the black reference point can be set at appropriate coordinate positions in the image while confirming the preceding photographed image, appropriate brightness and Can set both black and white reference points with density gradient between both reference points.
[0017]
In the present invention,In the feature extraction apparatus having the above-described configuration, regarding the arithmetic imaging unit, a difference between both black and white level values of both black and white reference points is obtained for a captured image obtained from the sample surface, and when the difference is equal to or less than a preset value, An enlargement / reduction correction operation is performed on a second predetermined level value obtained by adding / subtracting a predetermined set value to / from both level values so as to prevent excessive gradation calculation correction of a captured image with a small gray level value. It is characterized by.
[0018]
  According to this configuration,When an image with a very small gray level difference, that is, an image such as almost all white or black, is obtained, the minute density characteristics are not abnormally enlarged and corrected, and the meaningless feature due to the mixing of the abnormally corrected image Preventing extraction and / or similar classification operations of these features.
[0019]
  The feature extraction system of the present invention sequentially irradiates the sample with a sample holder for holding the sample and a plurality of types of light having wavelength ranges from the ultraviolet region to the near-infrared region and having different wavelengths step by step. An irradiation unit; and an imaging device that captures an image of the sample surface of the sample in a plurality of imageable wavelength bands having different center wavelengths and acquires an image corresponding to each wavelength band; A feature extraction system for extracting representative features and obtaining representative images of a plurality of images,
The sample holder, the irradiating means, and the imaging device are provided, and the imaging devices have different wavelength bands that can be imaged from each other, each has a unique light-sensitive characteristic, and a part of the imageable wavelength band of one camera is A white reference point indicating a brightness level and a darkness level in a picked-up image of the sample holder, comprising a plurality of cameras that overlap with a part of the imageable wavelength band of another camera and become a wavelength overlap region A brightness level of the white level value of the white reference point and the black level value of the black reference point captured by both cameras having the wavelength overlap region within the wavelength overlap region. An arithmetic device that sequentially obtains camera image data while performing a scaling correction calculation on the grayscale value of the captured image at the same rate so that the density gradient between the reference points and the density gradient between the two reference points is equal to or higher than a predetermined level, and the main body unit and the calculation Equipment It continued, and is characterized by having a communication line to collect the captured image obtained by the main unit to the arithmetic unit.
[0020]
  According to this configuration,It becomes possible to install the arithmetic unit at an arbitrary place away from the main unit..
[0027]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, an embodiment of the present invention will be described with reference to the drawings. First, the structure will be described with reference to FIGS.
[0028]
The feature extraction device 1 includes a main body 2 and an arithmetic device 3. The main body 2 has a housing 10, and its internal components are divided into an optical section 11 and a control section 12 as shown in FIG. 1 to 6 show the components included exclusively in the optical section 11.
[0029]
The housing 10 of the main body 2 is supported by a plurality of casters 21 so as to be movable on the floor surface. A part of the housing 10 has a door 22, and the sample holder 23 is taken in and out by opening the door 22. If the door 22 is closed, the inside of the housing 10 is in a dark room state. The sample holder 23 holds a sample of paper sheets such as banknotes, securities, and stamps so that the surface to be inspected is vertical. The surface to be inspected is irradiated with inspection light, and reflected light (including excitation light of the fluorescent material, if any) is imaged. Subsequently, the inspection light generator 30 functions as an irradiation unit that irradiates the sample with a plurality of types of light having different wavelengths step by step, and the sample surface is imaged in different wavelength bands to correspond to the respective wavelength bands. A configuration of the imaging device 50 that functions as an imaging unit that acquires an image will be described.
[0030]
The light source unit 31 of the inspection light generator 30 emits light in the wavelength range of 250 to 2,000 nm. Since such a wide band of light cannot be generated by a single lamp, two types of lamps, a xenon lamp 32 and a halogen lamp 33 are used (see FIGS. 4 and 5). The xenon lamp 32 is used for obtaining inspection light of 250 to 380 nm, and the halogen lamp 33 is used for obtaining inspection light of 381 to 2,000 nm.
[0031]
The xenon lamp 32 and the halogen lamp 33 are disposed facing each other in the lamp house 34. Since it takes time for the lamp to emit light of the specified spectrum after the power is turned on, both lamps are lit continuously while the device is in operation, instead of turning the lamp on and off each time the lamp is switched. In addition, the necessary side light is reflected by the movable mirror each time and is taken out. Reference numeral 35 denotes the movable mirror, and the reflection surface is directed to the xenon lamp 32 at a 45 ° angle, and the reflection surface is directed to the halogen lamp 33 at a 45 ° angle in FIG. It is switched by a motor or solenoid (not shown). The light hitting the movable mirror 35 turns at a right angle and enters the lens house 37 through the exit window 36 on one side of the lamp house 34.
[0032]
The light whose divergence angle is adjusted by the lens group in the lens house 37 enters the spectroscope 38. The spectroscope 38 is an aberration-correction type zero dispersion double monochromator. The drive wavelength range is set to 250 to 2,000 nm, the half-value width is set to 30 nm or 60 nm, and the resolution is set to 15 to 30 nm. It can also be set to “white light including the entire wavelength range”. Light of a predetermined wavelength band emitted from the spectroscope 38 becomes inspection light.
[0033]
A lens house 39 is connected to the exit window of the spectroscope 38. A lens (not shown) disposed in the lens house 39 is for adjusting the divergence angle of the inspection light. Similarly, when irradiating an area of 180 mm × 180 mm on the sample holder 23, 50 mm × 50 mm. The divergence angle is switched between when irradiating the area.
[0034]
The inspection light exiting the lens house 39 travels through the space while being repeatedly reflected by the fixed mirrors 40 and 41, and irradiates the sample held by the sample holder 23. Light reflected and absorbed by the surface to be inspected of the sample enters the imaging device 50.
[0035]
As described above, the inspection light band ranges from 250 to 2,000 nm. However, there is no imaging means having a wide sensing area, so the imaging apparatus 50 is configured with two types of cameras. One of them is a CCD camera 51, which is arranged in front of the sample holder 23 as seen in FIGS. The other is an infrared camera 52, which is arranged in parallel with the CCD camera 51 at a position shifted in the horizontal direction from the front in front of the sample holder 23.
[0036]
A CCD camera 51 composed of a silicon imaging device or the like is used for imaging reflected light having a wavelength of 250 to 1,000 nm. An infrared camera 52 having an imaging element made of platinum indium, indium antimony, or the like is used for imaging reflected light of 1,000 to 2,000 nm. These cameras have about 65,000 (≈256 × 256) pixels to about 1 million (≈1024 × 1024) pixels or more, and the sample surface to be analyzed has a density of more than 256 gradations. Capture and input as an image.
[0037]
The camera is switched by the movable mirror 53 shown in FIGS. The movable mirror 53 is moved in the horizontal direction by a driving mechanism (not shown). When the movable mirror 53 reaches the position shown in FIG. 2, the reflected light from the sample holder 23 reaches the CCD camera 51 straight. When the movable mirror 53 reaches the position shown in FIG. 3, the reflected light is bent at a right angle by the movable mirror 53 and is bent again at a right angle by the fixed mirror 54 and reaches the infrared camera 52.
[0038]
Reference numerals 55a, 55b, 56a and 56b denote converging lenses which appear and disappear in respective optical paths between the sample holder 23 and the CCD camera 51, and the fixed mirror 54 and the infrared camera 52 by a driving mechanism (not shown). The converging lenses 55a and 56a are used when converging the reflected light when the irradiation area is 180 mm × 180 mm, and the converging lenses 55b and 56b are used when converging the reflected light when the irradiation area is 50 mm × 50 mm. Use.
[0039]
In FIGS. 1, 2, and 3, the converging lenses 55a and 55b are shown in the optical path so that the combination of the converging lenses 55a and 55b or the converging lenses 56a and 56b exist at the same time. Actually, only one of the combinations advances into the optical path, and the other one is retracted out of the optical path.
[0040]
Since the light receiving area and / or image forming position on the CCD or infrared imaging device slightly changes between when the converging lenses 55a and 56a are used and when the converging lenses 55b and 56b are used, these cameras 51 are driven by a driving mechanism (not shown). , 52 are moved along the optical axis to compensate for the change. A zoom lens mechanism may be used instead of moving the camera.
[0041]
In order to narrow down the imaged light to a predetermined wavelength range, the CCD camera 51 is combined with a spectroscope (ultraviolet visible image spectroscope) 57, and the infrared camera 52 is combined with a bandpass filter (near infrared bandpass filter) 58. It is done. The spectroscope 57 has a half width of 30 nm and a resolution of 15 nm. The band pass filter 58 has a half width of 30 to 60 nm and a resolution of 30 nm. The spectroscope 57 and the band-pass filter 58 also have a function of selecting light in the entire wavelength range in the same manner as the spectroscope 38 on the light source side.
[0042]
FIG. 6 shows a structural example of the band pass filter 58. This is a combination of three turrets 59a, 59b and 59c, each of which is index rotated by a motor (not shown). Each turret 59a, 59b, 59c holds 13 filter holders 60 radially, and each one filter holder 60 is made to coincide with the optical path. In FIG. 6, a hatched circle (circle represents a filter) indicates the position of the optical path.
[0043]
Of the 13 filter holders 60 held by the turrets 59a, 59b, and 59c, the filter 61 having a slightly different pass wavelength band is attached to 12 of the filter holders 60, and the remaining one is transparent for the entire wavelength range. . If the filter holder 60 through which the turrets 59b and 59c pass is selected, the 12 types of filters of the turret 59a can be used alone. If the transparent filter holder 60 is selected by the turrets 59a and 59c, the 12 types of filters of the turret 59b can be used alone in the same manner. If the transparent filter holder 60 is selected with the turrets 59a and 59b, the 12 types of filters of the turret 59c can be used alone as well. Further, if the filter holder 60 that is transparent in all of the turrets 59a, 59b, and 59c is selected, light in the entire wavelength band is selected.
[0044]
Next, a block configuration of the feature extraction apparatus 1 will be described with reference to FIG.
[0045]
The control section 12 of the main body 10 includes a lamp controller 70, a mirror controller 71, a lens controller 72, an irradiation side spectrometer controller 73, an imaging side spectrometer controller 74, and a camera controller 75. The lamp controller 70 controls turning on / off of the xenon lamp 32 and the halogen lamp 33. The mirror control unit 71 controls the operation of the movable mirrors 35 and 53. The lens control unit 72 controls the operation of the lenses in the lens house 39 and the converging lenses 55a, 55b, 56a, and 56b. The irradiation side spectroscope control unit 73 controls the operation of the spectroscope 38. The imaging side spectroscope control unit 74 controls the operation of the spectroscope 57 and the band pass filter 58. The camera control unit 75 controls the exposure time and other shooting condition settings and imaging of the CCD camera 51 and the infrared camera 52.
[0046]
The block configuration of the arithmetic device 3 is as follows. A central processing unit 80 includes a microprocessor and a storage device, controls the arithmetic device 3, and processes and outputs data by a program. Reference numeral 81 denotes an operation unit, which includes input means such as a keyboard and various switches provided on the main body of the arithmetic device 3. A display unit 82 includes a monitor such as a CRT or a liquid crystal and a display unit such as a light emitting diode provided in the main body of the arithmetic unit 3. The arithmetic device 3 and the main body 2 are connected by a cable (not shown) and exchange signals with each other.
[0047]
Reference numerals 83 to 88 denote data processing block groups configured by using system resources of the central processing unit 80 as they are or adding hardware elements to the system resources of the central processing unit 80. Reference numeral 83 denotes an A / D converter, which converts analog data of an image transmitted from the imaging device 50 into digital data. 84 appropriately controls the CCD camera 51 and the infrared camera 52, unifies the image data obtained by the CCD camera 51 and the image data obtained by the infrared camera 52, and cuts out as an input image of a necessary size separately selected and set. At the same time, the correction calculation unit compresses the image size. Reference numeral 85 denotes a correction data storage unit that stores parameters such as a correction coefficient for calculation by the correction calculation unit 84. Reference numeral 86 denotes an image buffer unit that holds an image cut out to the required size, a compressed image, and a temporary intermediate image that is processed for image feature extraction. Reference numeral 87 denotes an image analysis unit for extracting features, and reference numeral 88 denotes a representative data storage unit that stores a representative image and related data selected as having clear features.
[0048]
Next, the operation of the feature extraction apparatus 1 will be described. First, the door 22 of the main body 2 is opened, the sample holder 23 is taken out, and a paper sheet sample is attached. The sample holder 23 to which the sample is attached is set at a predetermined position in the housing 10 and the door 22 is closed. Then, the sample is irradiated with the inspection light, and the light reflected from the sample is imaged by the imaging device 50.
[0049]
The inspection light generator 30 selects light in a wide wavelength range from 250 to 2,000 nm ranging from the ultraviolet region to the near infrared region as all wavelength light (white light) or inspection light in a predetermined band while shifting the center wavelength at a predetermined pitch. Occur sequentially. The sample surface looks according to the inspection light. For example, when a fluorescent material is contained in the material constituting the paper or in the printing ink, the fluorescent material emits excitation light in response to the inspection light having a specific wavelength. The light image obtained by combining the excitation light and the reflected light is passed through the spectroscope 57 or the band pass filter 58, and is picked up by the CCD camera 51 or the infrared camera 52 as a grayscale image of specific wavelength light.
[0050]
FIG. 8 illustrates the transition of the image of the paper sheet when the spectroscope and the bandpass filter on the camera side are passed through while shifting the wavelength range of the inspection light to be irradiated, that is, the image is captured as light in the entire wavelength range. In (a), the original pattern of the paper sheet is not seen at all, and the pattern due to the excitation of the fluorescent material is only visible in the lower right. As the wavelength of the inspection light becomes longer as (b) → (c) → (d) → (e) → (f), the original visual visible light pattern of the paper sheet gradually appears and the excitation light pattern disappears. In (g), the original pattern of the paper sheet has the clearest and brightest image. Thereafter, as the wavelength of the inspection light increases from (h) → (i) → (j) → (k), the image loses clarity and brightness. In (l), only a pattern sensitive only to the near infrared is visible.
[0051]
FIG. 8 is a drawing example simplified for illustrative purposes only, and an actual image does not follow the same transition. An image group obtained by imaging an actual paper sheet over a wavelength range of 250 nm to 2,000 nm shows much more complicated changes.
[0052]
As described above, the inspection light generation device 30 irradiates a plurality of types of light having different wavelengths in stages and having a narrow band sequentially as inspection light, or full wavelength light (white light) as inspection light. The central wavelength of the inspection light is set to 86 steps (of which one is the full wavelength light) between 250 and 2,000 nm, and the spectroscope 57 and the band pass filter 58 are combined on the imaging side to obtain 86 steps (of which Assuming that the pass wavelength band of all wavelengths is set in one stage, 86 × 86 = 7,396 images are picked up.
[0053]
Now, it is possible to acquire a large number of images with different imaging wavelength bands by the feature extraction device 1, but for example, all of fine image data having about 1 million pixels and each pixel having tens of thousands of grayscales. When dealing with the characteristics of the paper sheet, a huge amount of memory is required, and a lot of data is stored as a characteristic in this way when differentiating the authenticity or difference of the same kind of paper sheet later. Then, it takes too much time for verification. First, even if the samples are of the same type, the printing is accompanied by misalignment and fading, and the printing state is not necessarily the same. Also, if used, it may be possible that color fading or dirt components such as sebum will adhere, resulting in differences in the captured images, and even the same type of sample may be excluded as a counterfeit ticket. Absent. Therefore, the feature to be extracted is compressed to a certain extent by compressing the image size and / or the gray scale while maintaining the fine and fine optical characteristics, except when performing a very fine high-precision feature search. It is better to make it tolerable.
[0054]
Therefore, the arithmetic device 3 performs a similar classification operation based on feature extraction according to a predetermined rule for a large number of cut-out images and / or compressed images cut out to a required size and input to the image buffer unit 86, and the paper sheet The representative data storage unit 88 stores an image that clearly shows the above-described features as a representative image. Hereinafter, a calculation method for automatically extracting features will be described.
[0055]
Differences between images are used for feature extraction and similar classification. FIG. 9 shows the concept of analysis by difference. That is, pay attention to the difference in data between one image and another image. If the first image has “ABCDEF” data and the second image has only “ABDEF” data, the difference between them is “C”. The analysis is performed using the difference “C” as a material.
[0056]
As difference data, a difference value (difference value) for “brightness” for each pixel corresponding to each image and the frequency of each difference value (data number for each difference value) are obtained. In the present embodiment, “brightness” is “bright” means that the amount of reflected light is large in the captured wavelength band. That is, the characteristic pattern that should appear in the imaging wavelength band appears more clearly. “Bright” can be rephrased as “dark”. “Frequency” indicates the number of pixels having the difference value.
[0057]
Subsequent calculations are performed using AD values of 256 tones obtained by converting the brightness of actual pixels into digital values. Therefore, the difference value of “brightness” between the two images is replaced with one of −255 to +255 AD values as shown in FIG.
[0058]
In the calculation, the sum of the products is obtained after obtaining the product of the AD value of each difference value and the frequency. At this time, when the product is obtained with the sign of the AD value being added (as shown in FIG. 10). There is a case where the product relating to the “index” which is the absolute value of the AD value is obtained (the example shown in FIG. 11 corresponds to this). Here, the sum of products in the former case is referred to as “product sum”, and the sum of products in the latter case is referred to as “index sum”.
[0059]
When the calculation is performed using an exponent, the difference difference is further emphasized in order to facilitate the calculation. FIG. 11 shows an example of the method. In FIG. 11, the absolute value of the difference value is obtained, and a value obtained by further subtracting a certain value therefrom is used as a “difference value index”. If a negative value appears, the difference value index is set to “zero”.
[0060]
In FIG. 11, “constant value” is separately set to “30”, and as a result, the interval from −30 to +30 in the difference value is all “zero” in the difference value index as a noise component. That is, all the pixels (pixels) having a difference value of −30 to +30 are treated as the same pixel within the allowable error range. Therefore, the influence of noise between approximate images is eliminated, and only pixels having a true difference with a difference value of ± 30 or more are recognized as separate images. FIG. 12 shows a graph of the relationship between the difference value and the difference value index.
[0061]
A method of extracting features based on the difference values and selecting an image that clearly represents the features as a representative image will be described with reference to FIG. FIG. 13 shows eight images. These images are picked up for each wavelength band with different center wavelengths at a predetermined pitch, and are arranged in order of wavelength. Image P_i-4Is dark and P from there_i-3, P_i-2And P_iAs it approaches, the brightness gradually increases and image P_iAt peak. Image P_iAfter passing, image P_{i + 3}The brightness gradually decreases until. That is, an example is shown in which a person image gradually emerges from a black image, and then gradually fades out to return to a black image. Even when the brightness of the image is reversed, the operation is the same as described above, and the detailed description is omitted.
[0062]
The sum (product sum) of the product of the difference value between adjacent images and the frequency for each difference value index is D_iRepresented by A change from “low brightness” to “high brightness” is expressed as “plus” and vice versa. Product sum D_iIndicates the difference between the two images compared.
[0063]
Described below is a representative image selection processing procedure in the case where the above-mentioned sum of products is smaller than a separately determined threshold value and is determined to be a similar image.
[0064]
Image P_i-4And P_i-3, P_i-3And P_i-2, P_i-2And P_i-1, P_i-1And P_iProduct sum D between_iAre both in the “plus”, that is, in the area where the image of the person appears. However, P_iAnd P_{i + 1}D between_iTurns to “minus”, that is, the fade-out area of the figure, and P_{i + 1}And P_{i + 2}, P_{i + 2}And P_{i + 3}Maintain “minus” during the period. The trend from “low brightness” to “high brightness” is the image P_iAnd image P_{i + 1}Is the result of the reversal between_iAnd P_{i + 1}The point between is the point of change.
[0065]
Image P located immediately before the change point_iIs the brightest and clearest of the previous and next images. That is, since the features common to the image group appear in a form that is most easily identified, this is adopted as a representative image and P_i-4, P_i-3... P_{i + 4}Are registered in the representative data storage unit 88 as similar images.
[0066]
Next, a representative image selection processing procedure in the case where it is determined that the above-mentioned exponent sum is larger than a separately determined threshold value and is not a similar image will be described.
[0067]
FIG. 14 shows that the image group of group 1 that has changed from “low brightness” to “high brightness” is an image P._iAnd image P_{i + 1}Shows a case where the image group suddenly changes to a group 2 image group that is not similar. In this case, the exponent sum is larger than a separately defined threshold value. At this time, as described above, the last image P in the group 1 image group._iAre registered as representative images. Image P_i-4, P_i-3, P_i-2, P_i-1Only information other than image data is stored and registered in the representative data storage unit 88 as similar images.
[0068]
The above operation is repeated, and several types of representative images and their similar images are extracted from a large number of image groups acquired over a wavelength range of 250 to 2,000 nm, and stored and registered in the representative data storage unit 88 together with similar image information.
[0069]
In this way, optical features are automatically extracted from one sample, and an image that well represents the features is sorted and stored as a representative image. This series of image analysis is performed according to the flowchart of FIG. Carried out.
[0070]
In step S101 in FIG. 15, a difference value for each pixel between the “current image” and the “previous image” is obtained for all the pixels. “Current image” is an analysis image (image to be analyzed) that is currently being analyzed, and “previous image” is an analysis image that was analyzed immediately before. It is assumed that the “previous image” has already been found to be similar to any one of the representative images. In step S102, the sum of absolute values of the products of the difference value index emphasizing the value of the difference value obtained by the method of FIG. 11 and the frequency for each difference value index is obtained as the exponent sum. In step S103, it is determined whether the exponent sum is equal to or greater than a predetermined threshold value. If the predetermined threshold value is not reached, it is determined that the current image is similar to the previous image and the process proceeds to step S104.
[0071]
In step S104, it is determined whether or not the current image is “brighter” than the previous image. As described above, “bright” means that the amount of reflected light is large in the imaged wavelength band. That is, the characteristic pattern that should appear in the imaging wavelength band appears more clearly. In the “brightness” comparison, not the exponential sum but the sum of products of the difference value and the frequency with the sign of positive and negative (product sum in FIG. 10) is used (the same applies to step S105 and step S106).
[0072]
If the current image is brighter than the previous image, the process proceeds to step S105. Here, a difference value for each pixel between the representative image having the same relationship as the previous image and the current image is obtained over all the pixels, and the brightness is compared. As a result, if it is determined in step S106 that the current image is brighter than the representative image by a predetermined value or more, the process proceeds to step S107.
[0073]
In step S107, the representative image of the similar group is updated, the current image is used as a new representative image, and the image that has been the representative image so far is registered as a new representative image.
[0074]
If the current image is not brighter than the previous image in step S104, the process proceeds to step S108. Then, it is registered as being similar to the previous representative image.
[0075]
If it is determined in step S106 that the current image is not brighter than the representative image by a predetermined value, the process also proceeds to step S108. Then, it is registered as being similar to the previous representative image.
[0076]
If it is determined in step S103 that the exponent sum is equal to or greater than the predetermined threshold value, the current image and the previous image are dissimilar, and the process proceeds to step S109.
[0077]
In step S109, all the representative images registered so far and / or in the current feature extraction process for the sample are compared with the current image, and the minimum difference representative image is searched for as the most approximate image, and the process proceeds to step S110. .
[0078]
In step S110, the most approximate representative image is provisionally set as the representative image of the current image, and a difference value for each pixel between the temporary set representative image and the current image is obtained over all pixels. In step S111, the sum of the products of the difference value index emphasizing the difference value obtained by the method of FIG. 11 and the frequency for each index is obtained as the exponent sum. In step S112, it is determined whether the exponent sum is equal to or greater than a predetermined determination value. If the predetermined determination value has not been reached, the process proceeds to step S108 on the assumption that the minimum difference representative image as the temporary representative is similar to the current image, and the same processing is executed.
[0079]
If the index sum is greater than or equal to the predetermined determination value in step S112, the process proceeds to step S113 on the assumption that the minimum difference representative image as the temporary representative and the current image are dissimilar. The current image is registered as a new representative image.
[0080]
As for the image data, all the detailed image data used for the feature extraction calculation is left for the representative image. For images other than the representative image, only the collation data (classification information such as optical characteristic feature extraction conditions) with the representative image is left, and the image data itself can be erased. This makes it possible to save the amount of memory spent on one sample.
[0081]
The feature extraction apparatus 1 stores and holds a read image from the camera corresponding to the representative image as a representative real image. Even if high-speed processing is simply performed on the basis of the compressed image data separately set for the difference value calculation of the preceding and following images and other classification and sorting operations, detailed camera image data is retained as a representative real image as it is. When the re-sorting calculation is performed based on the calculation specifications, it is possible to perform feature extraction with high accuracy from each representative real image data and collation data.
[0082]
In the feature extraction device 1, when it is necessary to identify different samples using extracted features, the wavelength of light emitted at a predetermined pitch and / or the center wave of the received light is made different. Imaging is performed with emphasis on the optical characteristics of the new sample paper for each band. Then, a representative image of the reference specimen sample is acquired, and it is checked whether or not an image that approximates this representative image in a predetermined wavelength range has been acquired. Furthermore, the expression band of the similar image of each representative image is checked for difference at a necessary location, and collation determination is performed. In addition, when many similar samples with unique characteristics are sporadically divided and inspected, as described above, the characteristics of the samples over a wide wavelength band are extracted and classified in a unified manner. Therefore, it is possible to accurately perform the matching determination in a short time.
[0083]
Here, the collation data will be described using the imaging log example of FIG. The imaging log is composed of a search specification part and an imaging data index part. “Search number” is a number assigned to each sample or each search unit. “Sample size” indicates the size of the sample to be imaged, and is 50 mm × 50 mm in this example. The description of “irradiation wavelength” indicates that the wavelength range is 250 to 970 nm and changes at a pitch of 30 nm. “Camera” indicates that a CCD camera is used out of a CCD camera and an infrared camera. “CCD image size = 4 × 4 (256 × 256)” indicates that 4 pixels × 4 pixels are handled as one unit and is treated as an area of 256 pixels vertically and horizontally. The “CCD exposure time” is 30 mS. “CCD exposure time automatic adjustment = 0” indicates that automatic adjustment is off. “CCD detection wavelength” includes the values of the sweep start wavelength, the sweep end wavelength, and the imaging wavelength pitch, which are all 0 in this example. This indicates that all wavelengths (white light) are set.
[0084]
Arbitrary characters can be entered in the “MEMO” column for later reference. The “number of images” is the number of captured images calculated based on the setting contents such as the irradiation wavelength. In this example, the number of captured images is 25, and there are 25 types with different measurement conditions. “CCD cooler temperature = −39” indicates that the cooling temperature of the CCD image sensor is −39 ° C., and “IR cooler temperature = 0” indicates that the IR image sensor is not used. . “Cutout position” indicates coordinates within the effective area of the captured image, and is set to “10, 33, 229, 233” in this example. “White reference position” indicates a white reference image area, which is a rectangular area represented by “119, 39, 127, 47” in this example. Similarly, the “black reference position” is a rectangular area represented by “123, 238, 131, 246”. “Folder” is a folder name in the hard disk for storing captured image data, and uses a search number.
[0085]
In the index part of the latter half of the imaging data, consecutive numbers are assigned to the images from “search start” to “search end”, and the imaging conditions are recorded. For example, the line 00000003 is described as follows. “17:28:50” is the imaging time and indicates that the image was captured at 17:28:50. “F = MCX0250-0000.BMP” means an image file in the BMP format, and actual image data is stored with this file name. The image pickup means is a CCD image pickup device, the lamp is a xenon lamp, and the irradiation light. Indicates that the light-receiving side transmits light in the entire wavelength region.
[0086]
Reading the data following the file name is as follows. “T = 30.0” indicates that the exposure time is 30.0 mS. “W = 636” indicates that the white reference read value at the white reference position under the above-described conditions is a 636AD value. “B = 621” also indicates that the black reference read value at the black reference position is the 621AD value. “S = 735 >> 601” is a value of an effective dynamic range to be corrected. When corrected, a dynamic range of 256 gradations is taken between 735AD and 601AD values. “OK” at the end indicates a normal captured image.
[0087]
In the data shown in FIG. 25, the memory can be saved by leaving only the file name data and the imaging condition data attached thereto as collation data of the optical characteristic feature extraction condition and deleting the actual image data. .
[0088]
Now, each of the CCD camera 51 and the infrared camera 52 has a unique light sensing characteristic. FIG. 16 conceptually shows this. FIG. 16 shows the wavelength band on the horizontal axis and the sensitivity on the vertical axis. The imageable wavelength bands of both cameras overlap in the vicinity of 1,000 nm, and the two cameras are switched and used with the switching point being 1,000 nm. However, at the switching point as it is, the image brightness and the density gradient are completely different between the image captured by the CCD camera 51 and the image captured by the infrared camera 52. In this state, there are two types of image data under the same conditions, and the image processing software and the data storage unit are dualized, and the system resources of the arithmetic device 3 are consumed extra.
[0089]
Therefore, in the present invention, the degree of approximation of the captured images of the CCD camera 51 and the infrared camera 52 at the switching point is equal to or higher than a predetermined level, that is, as if the imaging across the switching point is performed with a single camera. Correct both cameras. Specifically, the brightness of the image is corrected by adjusting the exposure time of the camera, and the density gradient is corrected by adjusting the sensitivity and gain. Only one or both of the brightness and the density gradient may be corrected. Since it is desirable that the imaging characteristics be flat even when the switching point is separated, correction is performed in each wavelength band. FIG. 17 conceptually shows the sensitivity characteristics after correction.
[0090]
To correct the brightness and density gradient of the image, a brighter (white) reference point and a darker (black) reference point are set in advance on a part of the imaging surface. On the other hand, the darker one needs to be corrected so that it is the same level in the darker one. In determining the reference point, one of the following two methods is used.
[0091]
In the method shown in FIG. 18, prior photographing is performed under the photographing conditions set in advance for the feature photographing of the sample to display the picked-up image, and the coordinates of the reference point are set in the image plane of the sample. The display unit 82 of the arithmetic device 3 serves as a preceding captured image display unit for displaying a captured image of the preceding shooting. Input means such as a keyboard included in the operation unit 81 serves as setting input means for setting the coordinates of the reference point.
[0092]
Reference numeral 90 denotes an image of a sample, in which a white reference point 91 is set in a bright part and a black reference point 92 is set in a dark part. When the monochrome reference points 91 and 92 of the image captured by the CCD camera 51 are compared with the monochrome reference points 91 and 92 of the image captured by the infrared camera 52, the brightness of the both reference points and the reference points 91 and 92 by both cameras are compared. The camera control values are corrected so that the density gradient (light grayscale difference) between them becomes an approximation degree equal to or higher than a predetermined level.
[0093]
In the method shown in FIG. 19, a black and white reference marker separately disposed on the sample holder 23 so as to be positioned at a predetermined position outside the sample surface is used. That is, the white reference marker 93 and the black reference marker 94 are attached to a place where the white reference marker 93 and the black reference marker 94 are not covered by the sample of the paper sheet, and the images of the black and white reference markers 93 and 94 are captured together with the sample image. Based on the images of the black and white reference markers 93 and 94, the camera control value is corrected in the same manner as described above.
[0094]
Regarding the correction of the camera control value, a predetermined degree of approximation can be obtained by the same correction between the captured images of the respective cameras. An approximate image can be obtained in the same manner when the sensitivity difference for each wavelength band of the camera and / or the light source lamp is switched.
[0095]
The light accumulation time adjustment can be realized by using a CCD having a so-called variable time shutter function. The procedure for automatically setting the exposure time of both the CCD and infrared cameras in this embodiment will be described with reference to FIGS. In the present embodiment, as shown in FIG. 21, the white level value (W) of the reference image separately set as a target for the white level value (wn) of the obtained image within the separately set allowable maximum exposure time (Tm). The exposure time is automatically selected and set so as to be within a predetermined allowable value (reference white level value ± 30%).
[0096]
First, a white level value (w1) for photographing at a predetermined exposure time (T1) is obtained (step S201) and compared with a target reference white level value (W) (step S202). In this comparison, if the difference between the two levels is within the predetermined allowable value, this image is regarded as an appropriate image, and the automatic exposure time setting ends (step S203). On the other hand, if it is outside the allowable value, the appropriate exposure time (Tf) corresponding to the value of the difference between the two levels in the comparison is predicted and set from the preset appropriate exposure time table (step S204). A level value (wf) is obtained (step S205). The appropriate exposure time (Tf) can also be determined using a mathematical formula described later.
[0097]
Next, the new white level value (wf) is compared with the reference white level value (W) again (step S206). If it is within the allowable value, an appropriate image is obtained and the automatic exposure time setting ends. (Step S207). In the present embodiment, when the white level value (wf) of the captured image at the exposure time (Tf) is not appropriate, the appropriate exposure time (Te) is re-predicted and photographed (step S208). This is the final exposure time, and no further optimization of the exposure time is performed. That is, the exposure time optimization re-predictive shooting is prevented from continuing indefinitely due to abnormal fluctuations in the captured image grayscale value caused by the illumination light source light, the optical optical path, and / or the camera failure. In this embodiment, whether or not the difference value between the white level value (we) of the captured image at the final exposure time (Te) and the target reference white level value (W) is equal to or less than a predetermined approximate value set separately. Confirmation is performed (steps S209 to S212).
[0098]
In addition, even if the exposure time is changed, even if the white level value of each captured image cannot be changed accordingly, this is detected and an alarm is activated. Further, even when the predicted appropriate exposure time exceeds the allowable maximum exposure time (Tm), this is detected and an alarm is activated, but the detailed description is omitted.
[0099]
The appropriate exposure time (Tf) can be calculated from the following equation.
Tf = T1 × (W−w0) / (w1−w0)
Tf <Tm
Here, it is assumed that the exposure time and the imaging white level are directly proportional. FIG. 21 shows an example in which the captured image white level value at the camera exposure time “0” is approximately calculated as (w0).
[0100]
Furthermore, the feature extraction apparatus 1 can acquire a captured image in a state in which the photographing exposure time is fixed to an arbitrary set value by a selection operation, and can perform feature extraction and classification calculation from these images. From these captured image values, it is also possible to confirm and contrast the absorption and reflectance of the irradiated light under each imaging condition of the sample.
[0101]
In addition, regarding the change adjustment of the white level value of the captured image, a transition metal oxide or the like (IrOx, Ta₂O_Five・ WO_ThreeEtc.) may be placed, and a light transmittance or light transmission amount of the film may be adjusted by applying a voltage to the physical element. For this adjustment, an applied voltage used for each imaging wavelength or a shutter opening time setting table is prepared in advance as described above.
[0102]
An example of a density gradient correction method using the above-described black and white reference correction will be described with reference to FIG. Here, an “arithmetic imaging unit” in which the arithmetic device 3 and the imaging device 50 are combined is assumed. In the example of FIG. 23, a monochrome reference portion is set for a camera-captured image of about 1 million pixels with 16,384 gradations. The white level value W of the white reference portion is set so that the white portion w has the gradation value 200 in the 256 gradation camera image data. Similarly, the black level value B of the black reference portion is set so that the black portion b has the gradation value 30 in the 256-gradation camera image data.
[0103]
Enlargement / reduction correction operations are performed at the same ratio (here, defined as ratio α = (WB) / (w−b)) so as to obtain a gradation image with the above settings, and camera image data is obtained sequentially. . As a result, the camera image data in which the characteristics of the sample to be measured are expressed with a predetermined grayscale level from a camera-captured image in which the offset (brightness level) value and / or the black and white reference value fluctuate due to changes in the measurement environment. Is obtained.
[0104]
In the present embodiment, an enlargement calculation prevention measure is taken so that an excessive enlargement calculation is not performed in the process of the enlargement / reduction correction calculation. This will be described with reference to the flowchart of FIG. In the conversion from the 16,384 gradation shown in FIG. 23 to the 256 gradation scale, the conversion areas indicated by MAX to MIN in FIG. 23 are determined. If the difference between the white reference light level W and the black reference light level B at 16,384 gradations is extremely small, it will be extremely enlarged if converted to w and b of 256 gradations. In order to prevent this, the allowable ranges DW and DB are set. The final determined range of 16,384 gradations MAX to MIN is converted to a 256 gradation 255 to 0 scale.
[0105]
In the flow of FIG. 22, first, imaging is performed (step S301), and a lower limit value X (154) in which a difference value (W−B) between the white level value W and the black level value B of the black and white reference portion of the camera captured image is set separately. ) Or not (step S302). If it is equal to or greater than the lower limit value X, the MAX value is set to W + α * (255−w) = W + 55α (step s303). On the other hand, the MAX value is set to W + DW = W + 50 so that an excessive enlargement calculation is not performed when the value is less than the lower limit value X (step S304).
[0106]
Next, the lower limit MIN is determined. It is checked whether or not the difference value (WB) between the white level value W and the black level value B is not less than a separately set lower limit value Y (113) (step S305). If the difference value is not less than the lower limit value Y, the MIN value is set to W + α ( 0−w) = W−200α (step S306). On the other hand, if it is less than the lower limit Y, the MIN value is set to B−DB = B−20 so that an excessive enlargement calculation is not performed (step S307).
[0107]
Each pixel value of the sample imaged by the CCD camera is converted so that the range of MAX to MIN values in 16,384 gradations is 0 to 255 in 256 gradations.
[0108]
As described above, when the difference value between the white level value W and the black level value B is equal to or smaller than a preset value, the white level value W and the black level value B are not subjected to the enlargement / reduction operation as they are. By adding / subtracting the set value to obtain the second predetermined level value, and performing the enlargement / reduction correction operation on the second predetermined level value, in other words, the imaging screen having a slight difference in light and shade levels, in other words, the black and white reference portion Images having substantially the same level value are prevented from being converted into a grayscale image in which the difference is excessively subjected to gradation correction calculation.
[0109]
The lower limit values (X, Y) and the operation constants (DW, DB) used in the above description are merely examples, and are not limited to these. In addition, although the calculation is performed using both the black and white level values (W, B) of the black and white reference portion of the camera photographed image, the same scaling correction calculation is performed even if the maximum and minimum gray values in the sample image are used. It can be performed.
[0110]
In the present embodiment, the following functions are provided for the purpose of capturing a minute grayscale image using an infrared camera or the like as shown in FIG. In FIG. 24, the number of the camera imaging element (consent with the pixel) is taken on the horizontal axis 65,535 (= 256 × 256), the received light amount level is taken at 16,384 gradations on the vertical axis, and both monochrome reference for each element. It is also possible to perform gradation gradation correction calculation for each image pickup device by setting a value. In this case, as shown in the lower diagram of FIG. 24, each image value of the camera image pickup element in a state where the light source irradiation light is shielded as shown in the upper diagram of FIG. Each image value of the camera image pickup element in a state where the light source light is irradiated on the reference paper serving as the white reference is stored as each white reference value. The black and white reference values for the respective image pickup devices stored in this way are predetermined black and white level values, for example, in a 256 gradation image, the white portion w has a gradation value of 200 and the black portion b has a gradation value. By performing the correction calculation for the image pickup value of the sample for each image pickup element so as to be 30, the enlargement / reduction correction calculation of the image pickup element can be executed as described above, and the features can be grasped in detail and precisely.
[0111]
In addition, by setting a plurality of black and white reference values corresponding to the light source irradiation light and / or each light emitting / receiving wavelength band, it is possible to perform appropriate correction under each measurement condition.
[0112]
By correcting the camera control value in this way, the image processing software and the data storage unit can be unified, and system resources can be saved and the processing speed can be increased. In addition, when the brightness and / or density gradient is additionally corrected with different accuracy as necessary, the original captured images are approximated so that accurate calculation processing is possible, and image features Can be clarified. Data for performing these correction calculations is stored in the correction data storage unit 85.
[0113]
When the number of pixels (pixel size) is different between cameras, it is possible to perform matching processing by image compression correction or the like (detailed explanation is omitted).
[0114]
FIG. 26 shows a feature extraction system according to the second embodiment of the present invention. In this system, the image analysis function is separated from the feature extraction device 1A, and the image analysis is performed by the arithmetic device 500 connected to the feature extraction device 1A via the communication line 400.
[0115]
The configuration of the main body 2 of the feature extraction device 1A is not different from the configuration of the main body 2 of the feature extraction device 1 of the first embodiment. That is, although not shown in FIG. 26, the optical section 11 includes the sample holder 23, the inspection light generator 30, and the imaging device 50. In this case as well, the sample holder 23 holds paper sheets such as banknotes, securities, and stamps. The inspection light generator 30 functions as an irradiation unit that selectively irradiates the sample with a plurality of types of light as inspection light. The imaging device 50 functions as an imaging unit that captures images of reflected light from the sample (including excitation light of the fluorescent substance, including excitation light) in different wavelength bands and acquires images corresponding to the respective wavelength bands. .
[0116]
The configuration of the control section 12 of the main body 2 is the same as that of the feature extraction device 1 of the first embodiment. However, the configuration of the arithmetic unit 3 is different from that in the first embodiment. That is, the image analysis unit 87 and the representative data storage unit 88 disappear from the arithmetic device 3 of the first embodiment, and a communication unit 89 is provided instead. The communication unit 89 is connected to the communication line 400. The communication line 400 may be in any form. It can be a dedicated line or the Internet.
[0117]
An arithmetic device 500 is connected to the communication line 400. The arithmetic device 500 is separate from the feature extraction device 1 and can be installed in a facility such as a data center.
[0118]
The block configuration of the arithmetic device 500 is as follows. A central processing unit 501 includes a microprocessor and a storage device. The central processing unit 501 controls the arithmetic device 500 and processes and outputs data according to a program. An operation unit 502 includes an input unit such as a keyboard and various switches provided on the main body of the arithmetic device 500. Reference numeral 503 denotes a display unit, which includes a monitor such as a CRT or liquid crystal, and a display unit such as a light emitting diode provided in the main body housing of the arithmetic device 500. A communication unit 504 exchanges data with the communication unit 89 of the feature extraction apparatus 1A. The communication units 89 and 504 can be configured by a communication interface such as a modem.
[0119]
Reference numeral 505 denotes an image analysis unit that extracts features by performing an operation according to a predetermined rule for the image data transmitted from the feature extraction apparatus 1A. A representative data storage unit 506 stores a representative image selected as having clear characteristics and related data. The image analysis unit 505 and the representative data storage unit 506 are configured by using the system resources of the central processing unit 501 as they are or by adding hardware elements to the system resources of the central processing unit 501. The image analysis unit 505 and the representative data storage unit 506 perform the same processing as the image analysis unit 87 and the representative data storage unit 88 of the first embodiment, but can handle a larger amount of data.
[0120]
As described above, a feature extraction system is configured by the feature extraction apparatus 1A installed at the site where the sample is handled and the arithmetic unit 500 installed in the remote data center and connected to the feature extraction apparatus 1A via the communication line 400. In this case, the image data collected by the plurality of feature extraction apparatuses 1A can be collected in the arithmetic unit 500 of the data center, and feature extraction and aggregation, and feature matching with the reference sample can be collectively processed.
[0121]
As a result, unified feature data over a wide wavelength band can be collected and managed by sample in a large-scale data processing environment such as a data center. By having the same feature extraction apparatus 1A re-inspect the same sample and verifying whether or not the inspection data obtained as a result matches the previous data, the inspection capability of the feature extraction apparatus 1A is affected by environmental changes and secular fluctuations. It is also easy to diagnose whether there is no change in the above and whether there is a problem in reproducibility.
[0122]
Also, when performing characteristic inspection for the first sample, read out the inspection light wavelength band, control setting values of various cameras, various calculation constants at the time of correction, etc. of the same inspection sample and the same as those values Can be set to a value. Therefore, feature extraction and collation determination processing can be easily performed under the same conditions as the same type of sample.
[0123]
Furthermore, regarding a plurality of separately selected samples, a plurality of feature data corresponding to each sample that have been feature-extracted and classified to form part of the above-described centralized feature data, and variations of these feature data, And average characteristics can be retrieved and displayed. Further, when the feature data of the first appearance sample is obtained, the similarity determination calculation based on the Euclidean distance between the feature data and the selected plurality of samples can be performed.
[0124]
Further, when a plurality of feature extraction devices 1A are connected to one arithmetic device 500, individual control or integrated control is performed using the above-described one control arithmetic method while taking into account variations of the individual feature extraction devices 1A. However, detailed description is omitted.
[0125]
Although various embodiments of the present invention have been described above, various modifications can be made without departing from the spirit of the present invention.
[0126]
【The invention's effect】
The present invention has the following effects.
[0127]
When capturing an image of a sample to extract features by performing an operation on the sample image according to a predetermined rule, an imaging apparatus that captures an image of the sample includes a plurality of cameras having different imageable wavelength bands in a predetermined wavelength overlapping region. It is used by switching at the wavelength, and the degree of approximation of the captured images of both cameras at the switching point is above a predetermined level.The brightness of the white level value of the white reference point and the black level value of the black reference point captured by both cameras having the wavelength overlap region and the density gradient between both reference points within the wavelength overlap region are predetermined levels. Enlargement / reduction correction calculation is performed on the grayscale values of the captured images at the same rate so as to achieve the above approximation.Therefore, it is possible to search in a wide wavelength band by a plurality of cameras. Further, even when switching from one camera to the other, the difference in images is small, and the optical characteristics of these images can be extracted by the same image processing method. Therefore, even if there are a plurality of cameras, the image processing software and the data storage unit can be unified, and system resources can be saved and the processing speed can be increased.
[0128]
By correcting the camera and / or the input image so that the brightness of all the captured images and / or the degree of approximation of the gray scale gain are equal to or higher than a predetermined level, together with the captured images of both cameras at the switching point, Image data can be unified and data processing can be easily performed while covering a wide wavelength band by a plurality of cameras having different sensitivity bands.
[0129]
A sample is held in a sample holder, the sample is irradiated with inspection light generated by the inspection light generator, and the sample surface is imaged in a predetermined different wavelength band by the imaging device, and images corresponding to these wavelength bands Therefore, it is possible to reliably acquire images for each wavelength band necessary for extracting the optical characteristics of the sample.
[0130]
Since the sample is irradiated with multiple types of light with different wavelengths in stages as inspection light, the optical characteristics of the sample surface corresponding to each of these multiple types of irradiated light are precisely searched and analyzed, and features Can be accurately grasped.
[0131]
A white reference point indicating a brightness level is provided in the captured image, and the brightness level value of the reference point of the captured image obtained sequentially is set to a brightness that matches a preset white level value within a predetermined allowable error value. Since the image pickup means to be controlled is provided, variations in camera sensitivity can be compensated for, and adverse effects caused by variations in camera sensitivity can be eliminated. In addition, it is possible to search for optical characteristics in a wide wavelength band by a plurality of cameras. In addition, even when switching from one camera to the other, the difference in images is small, and optical features can be extracted by the same image processing technique. Therefore, even if there are a plurality of cameras, the image processing software and the data storage unit can be unified, and system resources can be saved and the processing speed can be increased.
[0132]
Since the white and black reference point white level value and black level value each have a predetermined grayscale level value, the grayscale value of the captured image is subjected to enlargement / reduction correction calculation at the same rate so that camera images are obtained sequentially. The dynamic range of each captured image is aligned, and variations in sensitivity of the imaging means (in terms of brightness and density gradient) can be eliminated.
[0133]
Even when precise matching correction that cannot be corrected by correction of brightness level, such as correction of camera exposure time, is required, a white reference point indicating the brightness level and a black reference point indicating the darkness level in the captured image Is used as an index for uniform correction of the captured image, and the grayscale value of the captured image is scaled and corrected at the same rate so that each of the black and white level values of both the black and white reference points becomes a predetermined grayscale level value. By calculating, it is possible to perform approximate correction so that the respective images have substantially the same level. For this reason, the clarification correction of the feature in the arithmetic processing can be performed more easily and accurately.
[0134]
In order to make the image characteristics clearer, the gray value of the image is corrected so that the black and white level of the black and white reference point obtained by the imaging means becomes the preset black and white levels (light and shade gradation values). By obtaining the camera image data while calculating, the brightness and / or shading gain of the imaging signal level is corrected to a constant level even for fluctuations in the imaging means, light source quantity, and / or other measurement environments. It is unified and it is possible to accurately grasp the characteristics of these captured image data.
[0135]
In addition, each black and white reference point is composed of a plurality of pixels close to each other, and the average value of these pixels is used as a reference value to prevent erroneous calculation due to a reference value that is inappropriate due to cosmic ray noise or other point noise. is doing. In addition, the black and white reference point gray value of the corrected image is set to stay inside with a predetermined width so as not to fall below the minimum value or above the maximum value of the camera image data. As a result, it is possible to accurately correct and display image features that are somewhat over or under-current from both black and white reference points without being over-current or under-current.
[0136]
An accurate centralized target level can also be corrected by separately providing a white reference marker and a black reference marker in the imaging area and inputting these coordinates.
[0137]
The black and white reference point provided in the captured image of the camera can be set at an arbitrary position in the captured image of the sample surface, or can be set at a black and white reference marker provided at a predetermined position outside the sample surface. By doing so, it is possible to perform correction so as to clarify the characteristics of the light and shade of the sample surface, and it is also possible to clarify the characteristics between the captured images using the monochrome reference marker as a reference.
[0138]
Pre-photographed image display means for performing pre-photographing under predetermined photographing conditions and displaying this picked-up image at the time of characteristic photographing of the sample surface, and the coordinates of the white reference point and the black reference on the image displayed on the pre-photographed image display means Reference point coordinate setting input means for setting the coordinates of the point is provided, and both the black and white reference points whose coordinates are set by the setting input means are provided.Brightness and concentration gradient between both reference pointsIs used as an index of the same-scale enlargement / reduction correction calculation of the captured image gray value obtained thereafter, so that the white reference point and the black reference point can be set at appropriate coordinate positions in the image while confirming the preceding captured image, A black and white reference point having an appropriate level value can be set as an index for calculating the same ratio scaling correction of the gray value of the captured image obtained thereafter.
[0139]
With respect to the arithmetic imaging means, a difference between the black and white level values of the black and white reference points is obtained for a captured image obtained from the sample surface, and when the difference value is equal to or smaller than a preset value, the black and white both level values are set to a predetermined value. Since the enlargement / reduction correction calculation is performed on the second predetermined level value obtained by adding / subtracting the set value, an excessive gradation calculation correction of a captured image with a small gray level value is prevented. When an image with a minute difference, that is, an image such as almost all white or black is obtained, the minute gradation characteristic is not abnormally enlarged and corrected, and it is meaningless due to the mixing of an abnormally corrected and misleading corrected image. It is possible to prevent feature extraction and / or similar classification operations of these features from being performed.
[0140]
In the case of an image such as almost all white or black as described above, the maximum and / or minimum value of black and white is set to a value far from the signal level of the black and white reference point, and is corrected and displayed as a singular image different from other images. By doing so, it is possible to reliably prevent meaningless feature extraction and / or similar classification operation of these features.
[0141]
When capturing images with minute differences in light and shade levels, when setting the monochrome reference value for each image sensor on the light receiving surface of the camera so that tone gradation correction can be selected for each image sensor, and light source illumination light is eliminated The camera image value for each pixel is used as the black reference value for each pixel, and the camera image value when the light source light is irradiated on the separately provided white reference paper is used as the white reference value for each pixel. By performing the enlargement / reduction correction operation so that the value becomes a predetermined gradation level value, it is possible to accurately grasp the characteristics of a minute gradation level image. It is also possible to set a plurality of black and white reference values corresponding to the light source light to be measured and / or the wavelength band of light transmission / reception, and to select and correct these according to measurement conditions.
[0142]
For each image captured by the camera, the type of camera that captured each image, its exposure time, the coordinate values and reference values of both black and white reference points, the type of light source lamp, the irradiation wavelength and imaging wavelength, both corrected black and white By storing together with reference values and measurement conditions such as the pixel size (resolution) of the acquired image, feature extraction from captured images obtained under different imaging conditions, for example, images corrected with different black and white reference values, and these Comparison of images, confirmation of image reproduction before correction, and the like can be performed.
[0143]
A sample holder for holding the sample;Irradiation means for sequentially irradiating the sample with a plurality of types of light having different wavelengths stepwise as inspection light, and a plurality of light beams having different center wavelengths on the sample surface of the sample. An imaging device that captures an image corresponding to each wavelength band and obtains an image corresponding to each wavelength band, extracts optical characteristics of the sample from the captured image, and obtains representative images of a plurality of images A feature extraction system comprising the sample holder, the irradiating means, and the imaging device, wherein the imaging devices have different wavelength bands that can be imaged from each other, each has a unique photosensitivity characteristic, and imaging by a single camera It consists of a plurality of cameras in which a part of the possible wavelength band overlaps with a part of the imageable wavelength band of the other one camera to form a wavelength overlapping region, and the brightness level is shown in the captured image of the sample holder White A main body provided with a black reference point indicating a quasi-point and a darkness level; and a white level value of the white reference point and the black reference point captured by both cameras having the wavelength overlap region in the wavelength overlap region An arithmetic device that sequentially obtains camera image data while performing the scaling correction calculation of the grayscale value of the captured image at the same rate so that the brightness of the black level value and the density gradient between the two reference points are equal to or greater than a predetermined level.Connected with a communication line, the captured images obtained by the main unit are collected in the arithmetic unit and feature extraction is performed, so the arithmetic unit can be installed at an arbitrary location away from the main unit become.
[0144]
Arithmetic units can be installed without physical connection to the main unit, so the computing unit is positioned as a data center, giving it a large data processing capability, and even if the hardware becomes larger, it can provide an environment that can accept it Is possible. As a result, even if enormous feature data is obtained from the sample, the data can be collected and managed by sample.
[0145]
The inspection ability of the main body can be diagnosed by re-inspecting the same sample and checking the degree of coincidence with the previous inspection data. Further, by checking and checking the degree of coincidence of the inspection data of a large number of samples belonging to the same category, it is possible to grasp the variation in characteristics between these samples. In addition, since the average characteristics of a large number of samples can be obtained, it is possible to more accurately perform the feature collation determination when the feature extraction and the difference determination are necessary for a new sample.
[Brief description of the drawings]
FIG. 1 is a configuration diagram of a feature extraction device according to a first embodiment of the present invention, in which an optical mechanism portion is a cross-sectional view.
FIG. 2 is a horizontal sectional view of the main body of the feature extraction device.
FIG. 3 is a horizontal sectional view similar to FIG. 2, showing different states
FIG. 4 is a horizontal sectional view of the light source section.
FIG. 5 is a horizontal sectional view similar to FIG. 4, showing different states
FIG. 6 is a front view of a bandpass filter.
FIG. 7 is a block diagram of the feature extraction device.
FIG. 8 is a diagram showing the transition of an image when imaging is performed while shifting the wavelength.
FIG. 9 is a diagram for explaining a feature extraction method and showing an image of a difference between images.
FIG. 10 is a table of AD values expressing the difference value of an image as a digital value.
FIG. 11 is a table of exponents that are absolute values of difference values.
Fig. 12: Index graph
FIG. 13 is a first explanatory diagram of a method for selecting a representative image based on a difference value;
FIG. 14 is a second explanatory diagram of a method for selecting a representative image based on a difference value.
FIG. 15 is a flowchart of image classification work.
FIG. 16 is a diagram conceptually illustrating a situation in which captured images of a plurality of cameras are used without correction.
FIG. 17 is a diagram conceptually illustrating a situation in which images captured by a plurality of cameras are used after correction.
FIG. 18 is a diagram showing a situation in which a monochrome reference point for camera correction is set in a sample image
FIG. 19 is a black and white reference marker for camera correction.
FIG. 20 is a flowchart for setting an exposure time.
FIG. 21 is a graph illustrating automatic setting of exposure time
FIG. 22 is a flowchart of monochrome reference enlargement / reduction correction.
FIG. 23 is a diagram for explaining black and white reference enlargement / reduction correction;
FIG. 24 is a diagram showing an example in which gradation gradation correction is performed for each image sensor;
FIG. 25 is a diagram showing an example of an imaging log
FIG. 26 is a block diagram of a feature extraction system according to the second embodiment of the present invention.
[Explanation of symbols]
1 Feature extraction device
2 Body
3 arithmetic unit
10 Housing
11 Optical section
12 Control section
23 Sample holder
30 Inspection light generator
31 Light source
38 Spectrometer
50 Imaging device
51 CCD camera
52 Infrared camera
57 Spectrometer
58 Bandpass filter
87 Image analysis unit
91 White reference point
92 Black reference point
93 White reference marker
94 Black reference marker
1A feature extraction device
400 communication line
500 arithmetic unit

Claims (4)

A sample holder for holding the sample;
Irradiation means for sequentially irradiating the sample with a plurality of types of light having different wavelengths in stages as light in a wavelength range from the ultraviolet region to the near infrared region,
An imaging device that captures an image corresponding to each wavelength band by imaging the sample surface of the sample in a plurality of imageable wavelength bands having different center wavelengths;
A feature extraction device that extracts optical features of the sample from the captured image and obtains representative images of a plurality of images,
The imaging devices have different wavelength bands that can be imaged from each other, have unique photosensitivity characteristics, and a part of the imageable wavelength band of one camera is a part of the imageable wavelength band of the other camera. Consisting of multiple cameras that overlap the wavelength overlap region,
While providing a white reference point indicating the brightness level and a black reference point indicating the darkness level in the captured image of the sample holder,
Within the wavelength overlap region, the brightness of the white level value of the white reference point and the black level value of the black reference point captured by both cameras having the wavelength overlap region, and the density gradient between both reference points are equal to or higher than a predetermined level. A feature extraction apparatus comprising arithmetic and imaging means for sequentially obtaining camera image data while performing a scaling correction calculation on the grayscale values of the captured image at the same rate so as to obtain a degree of approximation . A pre-captured image display means for displaying the captured image by performing pre-photographing under a predetermined photographing condition during characteristic photographing of the sample surface;
Reference point coordinate setting input means for setting the coordinates of the white reference point and the black reference point in the image displayed on the preceding captured image display means,
The brightness of both black and white reference points whose coordinates are set by the setting input means and the density gradient between the two reference points are used as indexes of the same-scale enlargement / reduction correction calculation of the captured image gray value obtained thereafter. 2. The feature extraction apparatus according to 1. With respect to the arithmetic imaging means, a difference between black and white level values of both black and white reference points is obtained for a captured image obtained from the sample surface, and when this difference value is equal to or smaller than a preset value, a predetermined set value is set for both black and white level values. 2. An excessive gradation calculation correction of a captured image having a small gray level value is prevented by performing an enlargement / reduction correction operation on a second predetermined level value obtained by adding / subtracting The feature extraction device described. A sample holder for holding the sample;
Irradiation means for sequentially irradiating the sample with a plurality of types of light having different wavelengths in stages as light in a wavelength range from the ultraviolet region to the near infrared region,
An imaging device that captures an image corresponding to each wavelength band by imaging the sample surface of the sample in a plurality of imageable wavelength bands having different center wavelengths;
A feature extraction system that extracts optical features of the sample from the captured image and obtains representative images of a plurality of images,
The sample holder, the irradiating means, and the imaging device are provided, and the imaging devices have different wavelength bands that can be imaged from each other, each has a unique light-sensitive characteristic, and a part of the imageable wavelength band of one camera is A white reference point indicating a brightness level and a darkness level in a picked-up image of the sample holder, comprising a plurality of cameras that overlap with a part of the imageable wavelength band of another camera and become a wavelength overlap region A main body provided with a black reference point indicating
Within the wavelength overlap region, the brightness of the white level value of the white reference point and the black level value of the black reference point captured by both cameras having the wavelength overlap region, and the density gradient between both reference points are equal to or higher than a predetermined level. An arithmetic device that sequentially obtains camera image data while performing a scaling correction calculation on the grayscale value of the captured image at the same rate so as to be an approximation of
A communication line for connecting the main body and the arithmetic device, and collecting captured images obtained by the main body on the arithmetic device;
A feature extraction system characterized by comprising:

Images