JP2008191817A - Image processing apparatus, image processing method, and computer program - Google Patents

Abstract

In a configuration in which an image stored in a first memory is acquired, normalized, and written to a second memory, the memory can be reduced in size and written accurately.
In an image processing apparatus for performing face identification processing, processing for extracting an image of a first memory (SRAM1) that stores a face image before normalization, performing normalization, and storing the image in a second memory (SRAM2). In the configuration to be performed, only a part of the face is stored in the first memory (SRAM 1) for storing the face image before normalization without storing the entire face. Further, the scan sequence as the data writing order to the second memory is changed according to the inclination of the face stored in the first memory (SRAM 1). With this configuration, a small memory can be miniaturized and an accurate normalized image can be written.
[Selection] Figure 16

Description

  The present invention relates to an image processing apparatus, an image processing method, and a computer program. More specifically, the present invention relates to an image processing apparatus, an image processing method, and a computer program that execute normalization processing of a face image included in captured image data, for example.

  Face recognition technology can be widely applied to man-machine interfaces, such as gender identification, as well as personal authentication systems that do not burden the user. Initially, a recognition technique using a profile was also studied, but at present, the recognition technique of the front image is the center.

  In the face recognition system, a face extraction process for extracting a face pattern from a captured image by a CCD camera or the like based on face feature information, and comparing the extracted face pattern with a registered face image, the face of a specific person is obtained. A face identification process for identifying this is executed. For face extraction processing and face identification processing, for example, “Gabor Filtering” that extracts facial image feature quantities by using a plurality of filters having directional selectivity and different frequency components is applied. (For example, see Patent Document 1).

  It has already been found that there are cells having selectivity for a specific orientation in human visual cells. It consists of cells that respond to vertical lines and cells that respond to horizontal lines. Similarly, the Gabor filter is a spatial filter composed of a plurality of filters having direction selectivity.

  The Gabor filter is spatially expressed by a Gabor function based on a Gaussian function for a window and a sine function or a cosine function for a frequency response. For example, the size of the filter window is fixed to 24 × 24. Further, assuming that the frequency f is 5 types and the angle θ is 8 directions, 40 types of Gabor filters are configured.

  The calculation of the Gabor filter is a convolution of the pixel to which the Gabor filter is applied and the coefficient of the Gabor filter. The coefficient of the Gabor filter can be separated into a real part whose frequency response is a cosine function and an imaginary part whose frequency response is a sine function. A Gabor filter result consisting of a scalar value of can be obtained. In such a calculation, a maximum of 40 types of Gabor filters are applied while switching between the frequency f and the angle θ, and a set of the maximum 40 scalar values obtained is referred to as a “Gabor jet”. A Gabor jet is obtained as a local feature amount for each feature amount extraction position detected at regular intervals in the horizontal and vertical directions on the face image data. The Gabor jet has a feature that it is invariant to a certain amount of positional shift or deformation of the feature amount extraction position.

  In the face identification process for identifying the face of a specific person, the extracted face pattern is compared with the registered face image. For the registered face image, a Gabor jet for each feature amount extraction position is calculated in advance. Then, the similarity between the Gabor jet of the input face and the Gabor jet of the registered face at the same feature amount extraction position is calculated, and a similarity vector that is a set of similarities at a plurality of feature amount extraction positions is obtained. Subsequently, class determination is performed by a support vector machine (SVM) to recognize an input face image and a registered face image. The support vector machine calculates the distance from the boundary surface (surface at a position where the value is 0) for determining the value of the boundary surface of the similarity vector as, for example, +1 or −1, and sets the intra-personal class or It is determined which of the extra-personal class it belongs to. If there is no similarity vector determined to belong to the intra-personal class, it is determined that an unregistered person's face has been input (for example, see Non-Patent Document 1). In addition, if one support vector machine learns (that is, registers) many face images, does the newly input face image match the registered (learned) face image (intra-)? whether it belongs to personal class) or does not match (whether it belongs to extra-personal class) (see, for example, Patent Document 2 and Patent Document 3). Support vector machines are recognized in the industry as having the highest level of learning generalization in the field of pattern recognition.

  As described above, in the face recognition system, a face extraction process for extracting a face pattern from a captured image by a CCD camera or the like based on face feature information is compared with the extracted face pattern and a registered face image, and specified. Face identification processing that identifies a person's face is executed. In the face identification processing, the image is extracted from the input image as preprocessing of comparison processing so that accurate comparison processing with a registered image is possible. A fitting process is performed on the face image.

  The fitting process is executed, for example, as a process of taking out the face image recorded in the first memory, executing the normalization process, and recording it in the second memory. Specifically, for example, the position of both eyes of the face image in the first memory is detected, and the face size, face position, and face angle are obtained from the detected eye position information, and the second memory is fixed. A process for creating a face image necessary for face recognition on the second memory by reducing, shifting, and rotating the face image in the first memory so that the right eye and the left eye are aligned with the coordinates is executed as a fitting process. To do.

  The above-described face identification process using the Gabor filter is realized by applying an image after the fitting process. That is, the similarity between the Gabor jet of the input face and the Gabor jet of the registered face at the same feature amount extraction position is calculated, and a similarity vector that is a set of similarities at a plurality of feature amount extraction positions is obtained. Class determination is performed by a machine (SVM), and face identification is performed based on collation between an input face image and a registered face image.

  In the normalization process as the fitting process described above, a normalization process is performed in which image conversion is performed with a reduction, shift, and rotation process on the face image in the first memory, and an execution result is created in the second memory. However, in this normalization (image conversion) process, the entire face image is stored in the first memory, and the pixel values of the pixels to be written in the second memory are the values before normalization stored in the first memory. It is common to perform a process of making a decision with reference to data.

  However, actually, the reference data necessary for determining the pixel value of the pixel to be written in the second memory is not the entire face image data before normalization, but only a part thereof. Therefore, there is no necessity to store the face image data before normalization in the first memory.

JP 2006-4041 A Japanese Patent Publication No. WO03 / 019475 JP 20064003 A B. Sholkopf et al., “Advanced in Kernel Support Vector Learning” (The MIT Press, 1999.)

  The present invention has been made in view of such a situation. In normalization (image conversion) processing, the first memory does not store the entire face image data but stores a part of the area data. An object of the present invention is to provide an image processing apparatus, an image processing method, and a computer program that can achieve downsizing of a memory and that can accurately write a normalized image into a memory using the downsized memory. .

The first aspect of the present invention is:
An image processing apparatus for performing identification processing of an input face image,
An image conversion unit that inputs a face image to be subjected to identification processing, performs image conversion on the input face image, and performs processing to normalize to an image having a predetermined setting;
A similarity calculation unit that calculates the similarity between the feature amount obtained from the converted input image converted by the image conversion unit and the feature amount of the registered image;
A match determination unit that determines whether the input face image and the registered face image match based on the similarity calculated by the similarity calculation unit;
The image conversion unit
A configuration for acquiring a face image from the first memory storing the face image to be subjected to normalization processing, executing normalization by image conversion, and storing the normalized face image in the second memory And
The first memory stores only a part of the face image to be normalized, and stores the data stored in the first memory in accordance with the progress of the writing of the image after the normalization process to the second memory. An image processing apparatus includes a memory control unit that executes update processing.

  Furthermore, in an embodiment of the image processing apparatus of the present invention, the memory control unit includes a write address of the second memory and a vertical direction (Y component) of a corresponding pixel position of the first memory corresponding to the write address. And the corresponding pixel position of the first memory is substantially centered in the longitudinal (Y) direction in the stored data of the first memory based on the comparison information of the address comparison unit. It is the structure which performs the update process of the stored data of the said 1st memory so that it may become.

  Furthermore, in one embodiment of the image processing apparatus of the present invention, the image conversion unit inputs angle information of image rotation executed during normalization processing, and corresponds to the second memory write address according to the angle information. The counter includes a counter that outputs a counter value, and the counter includes a plurality of sub-counters that output different counter values according to different angles.

Furthermore, in one embodiment of the image processing apparatus of the present invention, each of the sub-counters is
Image rotation angle: θ = arctan (± 1 / n)
Where n is an integer,
The counter value corresponding to the second memory write address corresponding to the angle is output.

Furthermore, in an embodiment of the image processing apparatus of the present invention, the image conversion unit calculates a conversion parameter for calculating a corresponding point of the first memory corresponding to each pixel position of the second memory. And a counter that inputs image rotation angle information included in the conversion parameter calculated by the calculation unit, and outputs a counter value different from the counter value corresponding to the second memory write address according to the input angle information; Based on the conversion parameter calculated by the calculation unit, the coordinate conversion unit that calculates the corresponding point of the first memory corresponding to each pixel position of the second memory, and the corresponding point information calculated by the coordinate conversion unit, An interpolation processing unit for determining a pixel value corresponding to each pixel position in the second memory;
It is the structure which has these.

  Furthermore, in an embodiment of the image processing apparatus of the present invention, the calculation unit is applied when converting the face image stored in the first memory as the conversion parameter into a face image stored in the second memory. A conversion parameter that defines at least one of image reduction processing, image rotation processing, or image translation processing to be performed, and the coordinate conversion unit is configured to perform image reduction processing, image rotation processing, or image translation processing. The conversion parameter that defines at least one of the processes is applied to calculate the corresponding point of the first memory corresponding to each pixel position of the second memory.

Furthermore, the second aspect of the present invention provides
An image processing method for identifying an input face image in an image processing apparatus,
An image conversion step in which an image conversion unit inputs a face image to be subjected to identification processing, executes image conversion on the input face image, and executes processing for normalizing to an image having a predetermined setting;
A similarity calculation step in which the similarity calculation unit calculates the similarity between the feature amount obtained from the converted input image converted by the image conversion unit and the feature amount of the registered image;
A match determination step for determining whether or not the input face image and the registered face image match based on the similarity calculated by the similarity calculation unit;
The image conversion step includes
A step of acquiring a face image from the first memory storing the face image to be subjected to normalization processing, executing normalization by image conversion, and storing the normalized face image in the second memory When,
The memory control unit stores only a part of the face image to be normalized in the first memory, and the memory control unit stores the first memory in accordance with the progress of the writing of the image after the normalization process to the second memory. An image processing method includes a memory control step of executing a storage data update process.

  Furthermore, in an embodiment of the image processing method of the present invention, the memory control step includes: a writing address of the second memory; and a vertical direction (Y component) of a corresponding pixel position of the first memory corresponding to the writing address. Based on the comparison information, the corresponding pixel position of the first memory is substantially centered in the longitudinal (Y) direction in the stored data of the first memory. Update processing is executed.

  Furthermore, in an embodiment of the image processing method of the present invention, the image conversion unit inputs angle information of image rotation executed during normalization processing, and corresponds to the second memory write address according to the angle information. A counter that outputs a counter value, and the counter includes a plurality of sub-counters that output different counter values according to different angles, and is selected by selecting a sub-counter to be applied according to input angle information. The counter value of the sub-counter is output.

Furthermore, in one embodiment of the image processing method of the present invention, each of the sub-counters includes:
Image rotation angle: θ = arctan (± 1 / n)
Where n is an integer,
A counter value corresponding to the second memory write address corresponding to the angle is output.

  Furthermore, in an embodiment of the image processing method of the present invention, the image conversion unit includes a conversion parameter for calculating a corresponding point of the first memory corresponding to each pixel position of the second memory. The calculation step and the counter input the angle information of the image rotation included in the conversion parameter calculated in the calculation step, and the counter value corresponding to the second memory write address is set differently according to the input angle information. And a coordinate conversion step in which the coordinate conversion unit calculates a corresponding point in the first memory corresponding to each pixel position in the second memory based on the conversion parameter calculated in the calculation step. And the interpolation processing unit determines each pixel position of the second memory according to the corresponding point information calculated in the coordinate conversion step. Characterized in that it has an interpolation processing step of determining a pixel value corresponding to.

  Furthermore, in an embodiment of the image processing method of the present invention, the calculation step is applied when converting the face image stored in the first memory as the conversion parameter into a face image stored in the second memory. A conversion parameter that defines at least one of image reduction processing, image rotation processing, or image translation processing to be calculated, and the coordinate conversion step includes image reduction processing, image rotation processing, or image translation processing A process for calculating a corresponding point of the first memory corresponding to each pixel position of the second memory is performed by applying a conversion parameter defining at least one of the processes.

Furthermore, the third aspect of the present invention provides
In the image processing apparatus, a computer program for performing input face image identification processing,
An image conversion step for causing the image conversion unit to input a face image to be subjected to identification processing, executing image conversion on the input face image, and performing processing to normalize to an image having a predetermined setting;
A similarity calculation step for causing the similarity calculation unit to calculate the similarity between the feature amount obtained from the converted input image converted by the image conversion unit and the feature amount of the registered image;
A match determination step for causing the match determination means to determine whether or not the input face image and the registered face image match based on the similarity calculated by the similarity calculation unit;
The image conversion step includes
A step of acquiring a face image from the first memory storing the face image to be normalized and executing normalization by image conversion to store the normalized face image in the second memory When,
Under the control of the memory control unit, only a part of the face image to be normalized is stored in the first memory, and in accordance with the progress of the writing of the image after the normalization process to the second memory, the first memory A computer program comprising a memory control step for executing a process of updating data stored in one memory.

  The computer program of the present invention is, for example, a storage medium or a communication medium provided in a computer-readable format to a computer system that can execute various program codes, such as a DVD, a CD, and an MO. It is a computer program that can be provided by a recording medium or a communication medium such as a network. By providing such a program in a computer-readable format, processing corresponding to the program is realized on the computer system.

  Other objects, features, and advantages of the present invention will become apparent from a more detailed description based on embodiments of the present invention described later and the accompanying drawings. In this specification, the system is a logical set configuration of a plurality of devices, and is not limited to one in which the devices of each configuration are in the same casing.

  According to the configuration of an embodiment of the present invention, in an image processing apparatus that performs face identification processing, an image of a first memory (SRAM 1) that stores a face image before normalization is extracted and normalized to execute the second memory. In the configuration in which the process of storing in (SRAM2) is performed, the memory is reduced in size as a configuration in which only a part is stored without storing the entire face in the first memory (SRAM1) that stores the face image before normalization. In addition, since the scan sequence as the data writing order to the second memory is changed according to the inclination of the face stored in the first memory (SRAM 1), the memory can be downsized and an accurate normalized image can be obtained. Write processing is realized.

  Hereinafter, an image processing apparatus, an image processing method, and a computer program according to the present invention will be described in detail with reference to the drawings.

  In the face recognition system, a face extraction process for extracting a face image from an input image and a comparison between the extracted face image and a registered image are performed to identify which registered image the input face image corresponds to As described above, the identification process is executed. Here, the face detection process and the face identification process are defined again.

  Face detection is processing for detecting a person's face and obtaining its position and size from a certain image (corresponding to one picture or one picture (field or frame) of a moving image). There may be a plurality of faces in one image. On the other hand, face identification refers to identifying whether one detected face is the same person as a previously registered face.

  FIG. 1 schematically shows the overall configuration of a face recognition system according to an embodiment of the present invention. The illustrated face recognition system 1 includes an image reducer 11, a face detector 12, a binocular position detector 13, a face discriminator 14, and a memory (SDRAM) 15 that stores a plurality of images. Further, a local memory (SRAM) is provided in the face detector 12, the binocular position detector 13, and the face discriminator 14. The input to the system 1 is an image, and the output (Yes / No) from the system 1 is a flag indicating whether or not the input face image and the registered face image are the same person.

  The image reducer 11 creates a reduced image that is halved in each of the horizontal and vertical directions with respect to the input image, and stores it in an SDRAM (Synchronous DRAM) 15 together with the original input image. In the example shown in FIG. 1, the input image is image 1, the reduced image obtained by halving the input image in the horizontal and vertical directions is image 1/2, and the input image is ¼ in the horizontal and vertical directions, respectively. These reduced images are set as the image 1/4 and these three images are created and stored in the SDRAM 15.

  When creating reduced images, considering the calculation accuracy, it is desirable to create each reduced image directly from the input image. However, in consideration of the hardware scale, a method of performing 1/2 reduction in order, that is, first creating an image 1/2 from an input image (image 1) and then creating the image 1/2 You may make it implement | achieve by the method of producing the image 1/4.

  The face detector 12 detects a face from the input image and all the reduced images stored in the SDRAM 15 and obtains the detected face size and face position. The detection of the face position can be easily realized by detecting the entire image without considering the processing speed. On the other hand, regarding the detection of faces of various sizes, a method of fixing the resolution of the image (that is, various sizes of the input image) due to the relative relationship between the resolution of the image and the size of the detected face. A method to prepare a face detector) and a method to fix the size of the detected face (one face detector with a fixed face size that can be detected is used to reduce the input image to various resolutions and detect it. Can be considered, but the latter method is practical. Therefore, as shown in FIG. 1, the face detector 12 has a face within 24 × 24 pixels for each reduced image of the image 1, the image 1/2, and the image 1/4 generated by the image reducer 11. Whether or not exists.

  If the reduction rate increments are 1/2 or 1/4 and are too coarse and the accuracy is insufficient, for example, the image reducer 11 reduces the images 7/8, 6/8, and 5/8. It is only necessary to further create a rate image and perform face detection for each reduction rate image by the face detector 12.

  A two-point pixel difference method can be applied as feature extraction for face detection. This is a method of performing a difference between two pixels that easily extract facial features within 24 × 24 pixels at various two points. Further, an adaboost algorithm can be used as the discriminator operation.

  In the binocular position detector 13, binocular position detection is performed in order to normalize the face of the image whose resolution is increased with respect to the face detected by the face detector 12 as preparation for face recognition of the face image. Specify the position of the eyes. That is, the positions of both eyes are detected, and the face size, face position, and face angle obtained by increasing the resolution of the face image are obtained from the face size and face position obtained by face detection.

  The face detector 12 performs face detection with a resolution of 24 × 24 pixels, and the face discriminator 14 performs face recognition with a resolution of 60 × 66 pixels. In other words, the resolution required for the face discriminator 14 is required to be more accurate than that of the face detector 12. Therefore, if the face detected by the face detector is an image 1/2, the left and right eyes are detected at a location where an equivalent face of the image 1 with higher resolution exists and above the face where the eye exists. .

  A two-point pixel difference method can be applied as feature extraction for detecting both-eye positions. The application range of the two-point pixel difference method is 24 × 24 pixels, which is the same as that of the face detector 12, so that the processing time can be realized serially with the same hardware as the face detector 12.

  The face discriminator 14 obtains the face size, position, and angle from the face image in which the right and left eye positions are specified by the binocular position detector 13, and normalizes the face according to these to normalize the inside of 60 × 66 pixels. Once stored in the SRAM, it is checked whether or not it matches the registered image.

  The face discriminator 14 applies Gabor filtering as, for example, feature extraction for face recognition. In addition, “gentleboost” is used as a classifier operation. The degree of similarity between the result obtained by applying the Gabor filter to the normalized face image and the result obtained by applying the Gabor filter to the previously registered image is obtained. Then, by performing a gentle boost on the obtained similarity, it is identified whether or not it matches the registered image.

  FIG. 2 shows a configuration example of the face discriminator 14. The illustrated face discriminator 14 includes an image conversion unit 141, a Gabor filter application unit 142, a similarity calculation unit 143, and a coincidence determination unit 144.

  First, the image conversion unit 141 transfers a face image corresponding to the resolution required for face identification based on the binocular position information detected by the binocular position detection from the SDRAM 15 to the local SRAM 1 to the face classifier 14. . Next, the face size, face position, and face angle are obtained from the binocular position information, and normalization processing is performed by image conversion of the face image in the SRAM 1 to create a face image necessary for face identification processing. And stored in the SRAM 2 which is a local memory provided in the face discriminator 14.

  That is, the image conversion unit 141 performs image conversion (normalization) such as reduction, shift, and rotation processing on the face image of the SRAM 1 so that the positions of the right eye and the left eye are aligned with the fixed coordinates of the SRAM 2 from the binocular position information. Then, a fitting process for creating a face image necessary for face recognition in the SRAM 2 is executed. That is, as shown in FIG. 3, normalization processing by image conversion of the face image in the SRAM 1 is executed and stored in the SRAM 2.

  In the apparatus of the present invention, image conversion (normalization) processing is performed in which occurrence of errors in the position and shape of face parts such as eyes, nose and mouth, which are important in face identification processing, is suppressed. This process will be described in detail later.

  The Gabor filter application unit 142 applies a Gabor filter to the face image normalized by the image conversion unit 141.

  The Gabor filter is spatially expressed by a Gabor function based on a Gaussian function for a window and a sine function or a cosine function for a frequency response. In the present embodiment, the filter window is fixed to 24 × 24 pixels as shown in FIG. 4A. When a filter window is applied to a response function including a sine function or cosine function of a specific frequency component as shown in FIG. 4B, a Gabor filter as shown in FIG. 4C can be created. In addition, for example, a sine function or cosine of 8 directions shifted by 22.5 degrees from 0 degree, 22.5 degrees, 45 degrees, 67.5 degrees, 90 degrees, 112.5 degrees, 135 degrees, and 157.5 degrees. When a filter window is applied to each response function composed of functions, eight Gabor filters can be created as shown in FIG. 4D. Further, by applying the filter window at the angle θ in eight directions similarly for the response functions of five types of frequencies f from the low frequency range to the high frequency range, a total of 40 types of Gabor filters can be obtained as shown in FIG. 4E. Composed. The Gabor filter (Gabor kernel K (x, y)) is expressed by the following arithmetic expression.

  The calculation of the Gabor filter is a convolution of the pixel I (x, y) to which the Gabor filter Gi (x, y) is applied and the coefficient of the Gabor filter. The coefficients of the Gabor filter can be separated into a real part Re (x, y) whose frequency response is a cosine function and an imaginary part whose frequency response is a sine function Im (x, y). The Gabor filter result Ji (x, y) consisting of one scalar value can be obtained by performing the above and combining the respective components. However, (x, y) indicates the pixel position of the feature amount extraction position, and i indicates the i-th of the 40 Gabor filters described above.

  A set {J1, J2,..., J40} of maximum 40 scalar values obtained by applying a maximum of 40 types of Gabor filters at the image feature extraction position (x, y) is called a Gabor jet.

  The similarity calculation unit 143 performs a normalized correlation operation on the Gabor Jet GS calculated for the input image and the Gabor Jet GR of the registered image to obtain the similarity d for each feature amount extraction position.

  In the present embodiment, normalized correlation calculation is performed at the 82 feature amount extraction positions (see FIG. 5) where feature amounts are easily extracted from the normalized face image of 60 × 66 pixels. Find the degree. As a result, a similarity vector D whose elements are normalized correlation calculation results obtained at the 82 feature amount extraction positions d0, d1,..., D81 is obtained.

  In the above description, “maximum 40 types” of Gabor filters are described. This means that it is not necessary to apply all 40 types of Gabor filters at all 82 feature extraction positions. To do. The number of types of Gabor filters to be applied depends on the location of the feature quantity extraction position where the similarity is extracted, and the types and the number of Gabor filters (the number of Gabor jet elements) differ depending on the location.

  The coincidence determination unit 144 determines the coincidence of both images based on the similarity vector between the input image and the registered image. In the present embodiment, “gentleboost” is used as the discriminator operation. Here, the gentle boost is calculated using the following arithmetic expression. However, it is assumed that a, q, and b in the formula are registered in advance as a dictionary.

  Then, according to the following discriminant, it is determined whether or not the input image matches the registered image.

  In the present embodiment, gentleboost is used for matching determination. However, for example, by using a support vector machine to classify the values of boundary surfaces of similar vectors, the matching of face images is determined. Also good.

  Next, details of the image conversion processing executed in the image conversion unit 141 (see FIG. 2) in the face discriminator 14 shown in FIG. 1 will be described. As described above with reference to FIG. 3, the image conversion unit 141 extracts the face image stored in the SRAM 1 and aligns the positions of the right eye and the left eye to the coordinates fixed in the SRAM 2 from the binocular position information of the extracted face image. As described above, the fitting process for executing the image conversion (normalization) such as the reduction, shift, and rotation processing on the face image in the SRAM 1 to create the face image necessary for the face recognition in the SRAM 2 is executed. In the apparatus of the present invention, image conversion (normalization) processing is performed in which occurrence of errors in the position and shape of face parts such as eyes, nose and mouth, which are important in face identification processing, is suppressed. Details of this image conversion processing will be described below.

  The image conversion unit 141 in the face discriminator 14 first executes a process (write) for writing a face area portion of an image on the SDRAM 15 (see FIG. 1) into the SRAM 1 (see FIG. 2). The image conversion unit 141 extracts a face image area from the image written in the SRAM 1 (see FIG. 2), executes image conversion as a fitting process, and writes the processing result in the SRAM 2. A fitting process is a process which normalizes the image of SRAM1 and produces it in SRAM2. Normalization in this apparatus is image conversion processing including at least one of face image alignment, rotation, and enlargement / reduction processing.

  Details of the image conversion processing executed in the image conversion unit 141 will be described with reference to FIG. FIG. 6 is a diagram illustrating a detailed configuration of the image conversion unit 141. The calculation unit 203 of the image conversion unit 141 first reads the face image written in the SRAM 1 and 201 from the SDRAM 200, and further inputs the left and right eye position coordinates of the face image written in the SRAM 1 and 201 from the eye position detection unit 202. To do.

  Based on the coordinate data, the calculation unit 203 obtains the rotation angle, magnification, and origin position of the face image applied to the image conversion executed as the fitting process. Further, the coordinate conversion unit 205 obtains the position of each of the SRAM 2 and 251 corresponding to each pixel in the SRAM 1 and 251 with decimal precision from the obtained angle, magnification, and origin. For example, the pixel (p, q) of the SRAM 2 251 illustrated in FIG. 7 calculates the pixel position (m, n) on the memory of the SRAM 1 201. For example, when the SRAM 2 251 shown in FIG. 7 is a memory that stores 60 × 66 pixel information, the pixel position (in the memory of the SRAM 1 201 corresponding to each pixel position included in the face area in the 60 × 66 pixel ( m, n) is calculated. In this case, m and n of the pixel position (mn) on the memory of the SRAM 1 and 201 may not be an integer value, but the arithmetic unit is not limited to the integer part for the corresponding pixel position (m, n). The corresponding pixel position (m, n) is calculated with a precision of a decimal part, for example, two digits after the decimal point.

  Based on the corresponding pixel position information calculated by the coordinate conversion unit 205, the interpolation processing unit 207 shown in FIG. 6 determines the pixel value of each of the SRAMs 2 and 251. That is, in the corresponding pixel position information calculated by the coordinate conversion unit 205, the pixel position (mn) on the memory of the SRAM 1, 201 corresponding to the pixel (p, q) of the SRAM 2, 251 has integer values m, n. In this case, the pixel value of the pixel (p, q) of the SRAM 2, 251 is set as the pixel value of the pixel position (mn) on the memory of the corresponding SRAM 1, 201, but the corresponding pixel position (m, n) When the value of m or n is not an integer value but has a fractional part, the pixel (p, q) of the SRAM 2, 251 is interpolated based on the pixel value of the SRAM 1 pixel around the corresponding pixel position (m, n). The pixel value of is determined.

  The pixel interpolation processing executed by the interpolation processing unit 207 of the image conversion unit 141 illustrated in FIG. 6 will be described with reference to FIG. When the pixel interpolation processing executed by the interpolation processing unit 207 is a bi-linear method, as shown in FIG. 8A, four surrounding pixels (SRAM1) of the corresponding pixel position (m, n) on the SRAM 1, 201 are used. , 201 integer part coordinates) and the distance to the surrounding four pixels (SRAM1 decimal part coordinates). When the pixel interpolation processing executed by the interpolation processing unit 207 is a bi-cubic method, as shown in FIG. 8B, the pixels 16 around the corresponding pixel position (m, n) on the SRAM 1, 201 are displayed. The point (SRAM 1, 201 integer part coordinates) and the distance to the surrounding 16 pixels (SRAM 1 decimal part coordinates) are obtained.

  Next, the weight as the contribution of each pixel value is calculated by applying the pixel of the integer part coordinate in the SRAM 1, 201 and the decimal part coordinate data in the SRAM 1, 201, and the correspondence of the SRAM 2, 251 based on the calculated weight. The pixel value of the pixel (p, q) is calculated, and the pixel value to be recorded (written) in the SRAM 2 is calculated.

  In this manner, the pixel value of the image stored in the SRAM 2 251 is determined based on each constituent pixel value of the image stored in the SRAM 1 201. As described above, the calculation unit 203 of the image conversion unit 141 first reads the face image written in the SRAM 1, 201 from the SDRAM 200, and further writes it in the SRAM 1, 201 from the eye position detection unit 202. The left and right eye position coordinates of the face image are input, and the rotation angle, magnification, and origin position of the face image applied to the image conversion executed as the fitting process are obtained based on the coordinate data. Further, the calculation unit 203 obtains, with decimal precision, the position in the SRAM 1 or 201 where each pixel of the SRAM 2 251 corresponds from the obtained angle, magnification, or origin.

  This fitting process is an image conversion process executed as a normalization process for more accurately executing a matching process with a registered image. This normalization process (image conversion process) will be described below. The calculation unit 203 defines the coordinates of the SRAMs 2 and 251 that store the normalized face images first.

  In the normalization process according to the present embodiment, first, the coordinates of the positions of the left and right eyes on the SRAM 2 251 for accumulating normalized face images are determined. That is, the fixed coordinates of the SRAMs 2 and 251 at the positions of the left and right eyes are determined. Specifically, for example, the fixed coordinates of the SRAM 2 251 are assigned by the following processing.

As shown in FIG. 9, the pixel coordinates (x, y) of the face image storage area of the SRAM 2, 251 for storing the face image after normalization (image conversion processing),
Horizontal axis (X axis): size 60 pixels, x = −29 to +30,
Vertical axis (Y axis): size 66 pixels, y = −39 to +26,
A processing example in the case of the above will be described.

  In such face image storage area coordinates of the SRAM 2 251, the (x, y) coordinate of the right eye position is assigned to (−11, 0), and the (x, y) coordinate of the left eye position is assigned to (11, 0). The origin (x, y) coordinates are the center (0, 0) of the right eye position and the left eye position.

  Next, the coordinates of the SRAMs 1 and 201 storing the face images before normalization are assigned. The face image data stored in the SRAM 1, 201 corresponds to the face image stored in the SDRAM 200 shown in FIG. 6, and first consider the coordinates on the SDRAM 200. In the eye position detection unit 202 shown in FIG. 6, left and right eye position information on the SDRAM 200 is acquired, and this information is input to the calculation unit 203. The detected left and right eye positions do not necessarily correspond to specific pixels of the SDRAM. Furthermore, the center of the left and right eye positions does not necessarily correspond to a specific pixel defined on the SDRAM.

That is, the origin (left and right eye center) coordinates on the SDRAM corresponding to the center of the left and right eyes, which is the origin set in the SRAM 2.251 storing the normalized face image, corresponds to a specific pixel position on the SDRAM. Not always.
The origin (center of left and right eyes) coordinates on the SDRAM
(Org_drx, org_dry)
And

Origin (center of left and right eye) coordinates on SDRAM 200 (org_drx, org_dry)
Exists in the following range for a certain pixel (x, y) on the SDRAM 200.
x <= org_drx <x + 1
y <= org_dry <y + 1
When a pixel on the SDRAM 200 is cut out and stored (written) in the SRAM 1, 201, the pixel (x, y) on the SDRAM 200 is aligned with the origin (0, 0) of the SRAM 1, 201.

As shown in FIG. 10, the pixel coordinates (x, y) of the face image storage area of the SRAM 1, 201 that stores the face image before normalization (image conversion processing),
Horizontal axis (X axis): size 192 pixels, x = −95 + 96,
Vertical axis (Y axis) size 180 pixels, y = −107 to +72,
And

With this setting, the origin positions (org_s1x, org_s1y) on the SRAM 1, 201 corresponding to the origin (center of left and right eyes) coordinates (org_drx, org_dry) on the SDRAM 200 exist in the following range as shown in FIG. .
0 <= org_s1x <1
0 <= org_s1y <1

FIG. 11 shows an enlarged view of the vicinity of the origin area of the SRAM 1, 201 shown in FIG. [▲] shown in the figure corresponding to the origin of SRAM 2 is the coordinate system in SRAM 1.
0 <= org_s1x <1
0 <= org_s1y <1
In the position. here,
org_s1x_int: the integer part of the origin coordinate x of the SRAM 1,
org_s1x_frc: Decimal part of the origin coordinate x of the SRAM 1,
org_s1y_int: the integer part of the origin coordinate y of the SRAM 1,
org_s1y_frc: the decimal part of the origin coordinate y of the SRAM 1,
Then,
org_s1x_int = 0,
org_s1y_int = 0,
It becomes.

As described above, the origin in the SRAM 2 251 for storing the image after normalization (image conversion processing) is the origin (0, 0) coordinate in the SRAM 2 coordinate system, but as described with reference to FIGS. In addition, the coordinate position of the SRAM 2 coordinate system corresponding to this origin in the SRAM 1, 201 storing the image before normalization (image conversion processing) is not (0, 0),
0 <= org_s1x <1
0 <= org_s1y <1
(Org_s1x, org_s1x).

  Therefore, when normalizing (image converting) the face image stored in the SRAM 1, 201 and storing it in the SRAM 2 251, processing for determining where in the coordinate system of the SRAM 1, 201 the coordinates of the SRAM 2 251 correspond. Is required.

  That is, as a normalization (image conversion) process for the face image recorded in the SRAM 1, 201, at least one of an image reduction process, a rotation process, and a parallel movement process is executed, and the normalized image is stored in the SRAM 2 251. Is stored, it is necessary to determine where each coordinate of the SRAM 2, 251 as the image storage memory after normalization corresponds to the coordinate system of the SRAM 1, 201 as the image storage memory before normalization. The calculation unit 203 of the image conversion unit 141 shown in FIG. 6 calculates the rotation angle, conversion magnification, and origin information as image conversion parameters necessary for determining the correspondence, and outputs these conversion parameters to the coordinate conversion unit 205. .

  The coordinate conversion unit 205 applies the conversion parameters input from the calculation unit 203 to calculate the coordinates of the SRAM 1 and 201 corresponding to the coordinates of the SRAM 2 and 251, respectively. In this process, the coordinate conversion unit 205 receives an address corresponding to the pixel position of the face image storage area in the SDRAM 200 or the SRAM 1, 201 from the address generation unit 204 of the image conversion unit 141 shown in FIG. Pixel position calculation is executed. The calculation result is output to the interpolation processing unit 207, and the interpolation processing unit 207 executes pixel value interpolation processing using the bi-linear method or the bi-cubic method described above with reference to FIG. The pixel value is determined, and the determined pixel value is written in each coordinate position of the SRAM 2 251.

  Referring to FIG. 12, the corresponding pixel calculation process required in the normalization process, that is, the coordinates of SRAM2, 251 which are image storage memories after normalization are SRAM1, 201 which are image storage memories before normalization. The details of the processing for determining where in the coordinate system corresponds will be described. In FIG. 12, the SDRAM 200 storing the input image, the SRAM 1, 201 which cuts out and stores the face image area stored in the SDRAM 200, and the normalization (image conversion) processing of the face image stored in the SRAM 1, 201 are executed. 3 shows the memory area of the SRAM 2 252 for storing the image after normalization (conversion processing).

The memory areas of the SDRAM 200, SRAM 1, 201, SRAM 2, 251 are as shown in FIG.
SDRAM: 320 × 240 pixels SRAM1: 192 × 180 pixels SRAM2: 60 × 66 pixels

  Further, the origin in the face image stored in the SRAM 2 251 after normalization is set as the center position of the left and right eyes of the face image stored in the SRAM 2 251, and the origin (0, 0) in this coordinate SRAM 2 coordinate system To do. The distance between the left and right eyes of the face image stored in the SRAM 2 251 is [P], for example, 22 pixels.

As shown in FIG. 12, the coordinates of the left eye and right eye coordinates of the face image and the origin of the center of the left and right eyes in the SRAM 1, 201 storing the image before normalization are as follows.
Left eye coordinates (lex, ley),
Right eye coordinates (rex, rey),
Origin coordinates (cex, cey),
And Also,
The distance between the left eye coordinates (lex, ley) and the right eye coordinates (rex, rey) is [c],
An x coordinate difference between the left eye coordinates (lex, ley) and the right eye coordinates (rex, rey) is represented by [a],
The difference in y coordinates between the left eye coordinates (lex, ley) and the right eye coordinates (rex, rey) is represented by [b],
And
a, b, c correspond to the lengths of the sides of the right triangle in the SRAM 1, 201 shown in FIG.
c ² = a ² + b ² ,
Have a relationship. Let θ be the angle between sides a and c.

Here, the (x, y) coordinates in the coordinate system of the SRAMs 1, 201 as image storage memories before normalization and the coordinate system of the SRAMs 2, 251 as image storage memories after normalization are defined as follows. .
s1x: x coordinate in the SRAM1 coordinate system,
s1y: y coordinate in the SRAM1 coordinate system,
s2x: x coordinate in the SRAM2 coordinate system,
s2y: y coordinate in the SRAM2 coordinate system,
Define it like this.

In the coordinate conversion unit 205, each of the coordinates (s2x, s2y) of the SRAM 2 that is the image storage memory after normalization is stored in the SRAM 1, 201 that is the image storage memory before normalization based on the following equation (Equation 1). The coordinates (s1x, s1y) are calculated.
s1x = (c / P) × ((s2x) × cos θ + s2y × sin θ) + (org_s1x_frc)
= (A * s2x + b * s2y) / P + (org_s1x_frc)
s1y = (c / P) × ((− s2x) × sin θ + s2y × cos θ) + (org_s1y_frc)
= (-B * s2x + a * s2y) / P + (org_s1y_frc)
... (Formula 1)

In the above formula (Formula 1),
a = (lex) − (rex),
b = ley-rey,
c = √ (a × a + b × b),
cos θ = a / c,
sin θ = b / c,
It is.

  The coordinate conversion unit 205 applies the above equation (Equation 1), and for each of the coordinates (s2x, s2y) of the SRAM 2 that is the image storage memory after normalization, the SRAM 1, 201 that is the image storage memory before normalization. The coordinates (s1x, s1y) are calculated.

The calculation unit 203 obtains the rotation angle, the conversion magnification, and the origin information as the conversion parameters applied in the above equation (Equation 1), and outputs these conversion parameters to the coordinate conversion unit 205. In the above formula (Formula 1),
Conversion magnification (image reduction processing parameter): (c / P)
Rotation angle (image rotation processing parameter): θ,
Origin information (image parallel displacement processing parameters): (org_s1x_frc, org_s1y_frc)
It is.
The origin information (image parallel movement processing parameter) is information (see FIG. 11) indicating a position corresponding to the origin in the SRAM 2 in the SRAM 1 coordinate system. Based on the origin information of the SRAMs 2 and 251 and the face image information stored in the SRAMs 1 and 201, the calculation unit 203 obtains each of the above conversion parameters: rotation angle, conversion magnification, and origin information, and calculates these conversion parameters. The data is output to the coordinate conversion unit 205.

  The coordinate conversion unit 205 inputs the count value of each pixel (s2x, s2y) of the SRAM2, 251 from the counter 206, applies the above equation (Equation 1) for each pixel, and each of the SRAM2, 251 The pixel position (s1x, s1y) of the SRAM 1 corresponding to the pixel (s2x, s2y) is calculated.

  The calculation result is output to the interpolation processing unit 207. The interpolation processing unit 207 executes pixel value interpolation processing using the bi-linear method or the bi-cubic method described above with reference to FIG. And the determined pixel value is written to each coordinate position of the SRAM 2 251. It should be noted that the count value of each pixel (s2x, s2y) of the SRAM 2 251 is input from the counter 206 as the write address of the SRAM 2, and the determined pixel value is written according to the address position.

  As an example of pixel value determination processing in the interpolation processing unit 207, a processing example to which the bi-linear method is applied will be described with reference to FIG. FIG. 13 shows a corresponding point [★] 301 in the SRAM 1 coordinate system of one coordinate point (s2x, s2y) of the SRAM 2 in the SRAM 1 coordinate system.

The corresponding point [★] 301 does not correspond to one specific coordinate point in the coordinate system of the SRAM 2, and is located at a position surrounded by a plurality of coordinate points (e, f, g, h). Each of the coordinate points E, F, G, and H is a coordinate point of the SRAM1 coordinate and has the following coordinates.
E = (s1x_int, s1y_int)
F = (s1x_int + 1, s1y_int)
G = (s1x_int, s1y_int + 1)
H = (s1x_int + 1, s1y_int + 1)

Pixel values are set for these pixel points E, F, G, and H, respectively. Here, the pixel values of the pixel points E, F, G, and H are e, f, g, and h, respectively. At this time, the interpolation processing unit 207 calculates the pixel value [Q] set to the coordinate point (s2x, s2y) of the SRAM 2 by the following calculation formula (Formula 2) by applying the bi-linear method.
Q = [(1-s1x_frc) * (1-s1y_frc) * e + (s1x_frc) * (1-s1y_frc) * f + (1-s1x_frc) * (s1y_frc) * g + (s1x_frc) * (s1y_frc) * h] 4
... (Formula 2)

The above equation (Equation 2) is a pixel value interpolation process by the bi-linear method, and is a pixel value determination process according to the process described above with reference to FIG. That is, the distance (SRAM1 decimal part coordinates) between the corresponding pixel position [★] on the SRAM 1 corresponding to the pixel (s2x, s2y) of the SRAM 2 and the four surrounding pixels (E, F, G, H) is obtained and obtained. This is a process of calculating a weight as a contribution degree of each pixel value based on the measured distance and calculating a pixel value of the pixel (s2x, s2y) of the SRAM 2 based on the calculated weight. By the above formula
The determined pixel value is written in each coordinate position of the SRAM 2 251.

  As described above, the pixel value determination processing method in the interpolation processing unit 207 is not limited to the bi-linear method described above, and other methods such as a bi-cubic method may be applied.

[Size reduction configuration of image storage memory (SRAM 1) before normalization]
Next, a description will be given of a configuration example that realizes a reduction in the size of the SRAM 1, which is a memory for storing an image before normalization. In the embodiment described above, for example, as described with reference to FIG. 12, the SRAMs 1 and 201 which are memories for storing the face images before normalization temporarily store the entire face images stored in the SDRAM 200. This is a configuration that requires a data storage capacity for recording the entire face image.

In the example described with reference to FIG.
The memory areas of the SDRAM 200, SRAM 1, 201, SRAM 2, 251 are
SDRAM: 320 × 240 pixels SRAM1: 192 × 180 pixels SRAM2: 60 × 66 pixels Each pixel value region has been described as a configuration.

  However, when the face image data in the SRAM 1 is normalized (image conversion) and stored in the SRAM 2, the normalized image is written to the SRAM 2 while determining the pixel values one by one for the constituent pixels of the SRAM 2. For example, the data necessary for the pixel value writing process of one constituent pixel of the SRAM 2 is not the entire face image data stored in the SRAM 1 but only a part thereof. Specifically, for example, when performing the pixel value interpolation processing by the above-described bi-linear method, as is apparent from the above description, the corresponding position on the SRAM 1 corresponding to the pixel position (s2x, s2y) of the SRAM 2 is determined. Only the pixel value information of the surrounding four pixels.

  Thus, the data necessary for writing the normalized face image data to the SRAM 2 is very limited data. In the following, in view of this fact, a configuration example in which the size of the SRAM 1 that is a memory for storing the face image data before normalization is reduced will be described.

A specific example will be described with reference to FIG. As shown in FIG. 14, the SRAM 1 for storing the face image data before normalization is set to store only a part of the face image, not the whole face image. In the example shown,
SRAM 1: A memory for recording data of 192 × 48 pixels. For example, if the configuration stores 8 bits of data per pixel, the SRAMs 1 and 401 shown in FIG.
SRAM 1: 192 x 48 pixels x 8 bits
Data storage capacity.

  The face image data stored in the SRAM 1 401 is updated as the data writing process to the SRAM 2 progresses. That is, as shown in FIG. 15, the data stored in the SRAM 1, 401 is updated. Specifically, the SRAM 1, 401 is set so that the pixel to be written to the SRAM 2 and the pixel position on the SRAM 1 corresponding to the write pixel 452 shown in FIG. Update the stored data.

  The memory control unit that executes data transfer control from the SDRAM that stores the entire face image to the SRAM 1, the corresponding pixel position of the SRAM 1, 401 is based on the address of the write pixel in the SRAM 2, 451, in the memory area of the SRAM 1, 401. The data stored in the SRAMs 1 and 401 are updated so as to be set at approximately the center of the upper and lower 48 pixels.

  That is, the memory control unit has an address comparison unit that compares the address of the write pixel of the SRAM 2 451 and the vertical direction (Y component) of the corresponding pixel position of the SRAM 1 401, and based on this address comparison information, the SDRAM To the SRAM 1 is executed. As shown in FIG. 15, the memory control unit controls the SRAM 1 so that the pixel position on the SRAM 1 corresponding to the writing pixel 452 is set to approximately the center of 48 pixels above and below the memory area of the SRAM 1 401 by such control. The stored data 401 is updated. As described above, the SRAMs 1 and 401 have a ring buffer configuration in which the stored data is sequentially updated.

  The configuration and processing of the image conversion unit 141 that executes this memory control processing will be described with reference to FIG. FIG. 16 includes SRAMs 1 and 401 which are small memories with a small data storage capacity described with reference to FIGS. 14 and 15. Only a part of the face image data stored in the SDRAM 200 is stored in the SRAMs 1 and 401. The SRAMs 1 and 401 correspond to the progress of the normalization (image conversion) process and the process of writing the normalized images into the SRAMs 2 and 451. The stored data is updated as described with reference to FIG.

  The memory controller 420 shown in FIG. 16 executes the data write to the SRAMs 1 and 401, that is, the data read from the SDRAM 200 and the data write control to the SRAMs 1 and 401. The address for data read of the SDRAM 404 is set. The SDARM address generation unit 404 to be generated, the SRAM 1 address generation unit 405 to generate an access address for the SRAM 1, 401, the counter 406 for sequentially setting the pixel positions to be processed in the SRAM 1, 401, and the write pixels of the SRAM 2, 451 described above. An address comparison unit 407 that compares the address and the vertical direction (Y component) of the corresponding pixel position of the SRAM 1 401 is provided.

  The address comparison unit 407 of the memory control unit 420 compares the address of the writing pixel of the SRAMs 2 and 451 with the vertical direction (Y component) of the corresponding pixel position of the SRAMs 1 and 401, and from the SDRAM to the SRAM 1 based on the address comparison information. As shown in FIG. 15, the data is stored in the SRAM 1 401 so that the pixel position on the SRAM 1 corresponding to the write pixel 452 is set to approximately the center of 48 pixels above and below the memory area of the SRAM 1 401. Update the data.

The data writing process to the SRAMs 2 and 451 is executed by applying a part of face image data stored in the SRAMs 1 and 401. This process is basically the same as the process described above, and as an image conversion parameter applied to the normalization process in the calculation unit 403,
Conversion magnification (image reduction processing parameter),
Rotation angle (image rotation processing parameters),
Origin information (image translation processing parameters),
These conversion parameters are calculated, and these conversion parameters are output to the coordinate conversion unit 409. The coordinate conversion unit 409 applies these conversion parameters and applies the equation (Equation 1) described above. , The correspondence between the SRAM1, 401 and the SRAM2, 451 is obtained. That is, in the coordinate conversion unit 409, each of the coordinates (s2x, s2y) of the SRAM 2, which is the image storage memory after normalization, is stored in the SRAM 1, which is the image storage memory before normalization, based on the following formula (Formula 1). The coordinates (s1x, s1y) of 201 are calculated.
s1x = (c / P) × ((s2x) × cos θ + s2y × sin θ) + (org_s1x_frc)
= (A * s2x + b * s2y) / P + (org_s1x_frc)
s1y = (c / P) × ((− s2x) × sin θ + s2y × cos θ) + (org_s1y_frc)
= (-B * s2x + a * s2y) / P + (org_s1y_frc)

  The calculation result is output to the interpolation processing unit 410, and the interpolation processing unit 410 executes pixel value interpolation processing using the bi-linear method or the bi-cubic method described above with reference to FIG. And the determined pixel value is written to each coordinate position of the SRAM 2 251.

  However, when the normalization processing result based on a part of the face image recorded in the SRAM 1 401 is written in the SRAM 2 451, the image stored in the SRAM 1 401 and the SRAM 2, as shown in FIG. If the images to be recorded in 451 coincide with each other with no tilt angle deviation, the pixels to be written in SRAM 2 and 451 are sequentially set in the raster order (60 × 66 cnt) to be written in SRAM 2 and 451, and are necessary for determining the pixel value. The pixel information of the corresponding SRAM 1, 401 can be acquired.

  However, in practice, the image before normalization stored in the SRAM 1, 401 is recorded based on the photographed image, and the image before normalization stored in the SRAM 1, 401 as shown in FIG. In many cases, the image has a slope different from that of the normalized image recorded in the SRAM 2 451. In this case, if processing is performed in the raster order (60 × 66 cnt) written in the SRAM 2 451 as shown in FIG. 17A, in order to maintain the state where the corresponding pixel position is stored from the SRAM 1 401, it is constantly It is necessary to change the position of the data stored in the SRAMs 1 and 401, which is inefficient.

  In order to solve this, the sequence of writing to the SRAMs 2 and 451 is changed. That is, the data writing sequence to the SRAMs 2 and 451 is changed according to a part of the image data stored in the SRAMs 1 and 401. Hereinafter, this processing example will be described.

  The following processing example is a processing example in which the scan order is changed in accordance with the difference in the inclination of the face image between the SRAM 1, 401 and the SRAM 2, 451, that is, the rotation angle of the face image. As one configuration for making it possible to change a plurality of different scan orders, that is, a pixel order for writing data to the SRAMs 2 and 451, there is a configuration using a ROM (60 × 66 addresses) that generates a plurality of scan orders. However, in order to obtain a configuration corresponding to an arbitrary angle, registration information in the scan order corresponding to each angle is required, and there is a problem that the memory size for recording this data becomes enormous.

  Therefore, in this embodiment, as shown in FIG. 18, a limited number of counters (5 in the embodiment) that output different scan order information according to the tilt angle are prepared, and these counters are selectively selected. use. A counter 408 illustrated in FIG. 18 is an exemplary configuration of the counter 408 of the image conversion unit 141 illustrated in FIG.

  As shown in FIG. 16, the counter 408 reads the address of the SRAM 1, 401, inputs angle information from the arithmetic unit 403, that is, rotation angle information to be executed in normalization processing, and writes to the SRAM 2 451. This is a counter that sequentially outputs addresses. The angle information input from the calculation unit 403 is a rotation process parameter as a conversion parameter applied to the normalization process. The angle information is a difference between the inclination angle of the face image stored in the SRAM 1 401 and the face image written in the SRAM 2 451. Equivalent to.

As illustrated in FIG. 18, the counter 408 includes five sub-counters 501 to 505 and a selection unit 511.
The first sub-counter has an inclination angle θ≈−27 degrees (arctan (−1/2)),
The second sub-counter has an inclination angle θ≈−14 degrees (arctan (−1/4)),
The third sub-counter has an inclination angle θ≈0 degrees (arctan (0)),
The fourth sub-counter has an inclination angle θ≈ + 14 degrees (arctan (+1/4)),
The fifth sub-counter has an inclination angle θ≈ + 27 degrees (arctan (+1/2)),
Scan order information corresponding to each is output.
The inclination angle θ is an inclination angle between the normalized image and the image stored in the SRAM 1 401. The selection unit 511 selects and outputs one of these five counter outputs according to the angle information input from the calculation unit 403. The output information is output to the SRAMs 2 and 451 and the coordinate conversion unit 409. The angle type is a setting in which θ is selected such that tan θ = 1/4, 1/2, 0, −1/4, −1/2. The scan order of this angle has regularity suitable for a counter and can be realized with a small-scale logic.

Actually, the inclination angles of the images stored in the SRAMs 1 and 401 are various, and the angle information input from the calculation unit 403 to the counter 408 has various values. In the present embodiment, for example, as shown in FIG. 19, these five counters are selected and used for each region having a certain inclination angle. As shown in FIG.
Inclination angle θ <−21 degrees: first sub-counter (corresponding to −27 degrees),
−21 degrees ≦ tilt angle θ <−7 degrees: second sub-counter (corresponding to −14 degrees),
−7 degrees ≦ tilt angle θ <+7 degrees: third sub-counter (corresponding to 0 degrees),
+7 degrees ≦ tilt angle θ <+21 degrees: fourth sub-counter (corresponding to +14 degrees),
+21 degrees ≦ tilt angle θ: fifth sub-counter (corresponding to +27 degrees),
In this way, a small number of counters are used according to various inclination angles.

Examples of setting the scan order based on actual counter outputs will be described with reference to FIGS. 20 to 24 show the scan order when the counters 501 to 505 are selected and used, that is, the data write order to the SRAMs 2 and 451.
FIG. 20 shows the scan order when the first sub-counter (corresponding to −27 degrees) is used.
FIG. 21 shows the scan order when the second sub-counter (corresponding to −14 degrees) is used,
FIG. 22 shows the scan order when the third sub-counter (corresponding to 0 degree) is used,
FIG. 23 shows the scan order when the fourth sub-counter (corresponding to +14 degrees) is used.
FIG. 24 shows the scan order when the fifth sub-counter (corresponding to +27 degrees) is used,
Each of these scan orders is shown.

FIG. 20 shows the scan order when the first sub-counter (corresponding to −27 degrees) is used. The first sub-counter 501 outputs a counter value indicating the scanning order according to the (counter output algorithm) shown in FIG. That is, (x, y) is used as a data write address for the SRAM 2,451.
cx: current x,
cy: current y,
nx: next x,
ny: next y,
In this case, according to the following counter value output algorithm, the counter value, that is, the designated data of the scan order is output.
cx = 0; cy = 0;
if (cx [0]! = 1) {-
ny = cy; nx = cx + 1;
} Else if (cy == 0) {(1)
ny = cx [5: 1] +1; nx = 0;
} Else if (cx == 59) {
if (cy <36) {(2)
ny = cy + 30; nx = 0;
} Else {(3)
ny = 65; nx = (cy-35) x2;
}
} Else {/
ny = cy−1; nx = cx + 1;
}

  By the counter value output according to the above algorithm, the scan is executed in accordance with the connection order of the arrows in FIG. Note that the processing of the counter value output algorithm is switched as shown in (1) to (3) in (B) (counter output algorithm) according to the areas (1), (2), and (3).

FIG. 21 shows the scan order when the second sub-counter (corresponding to −14 degrees) is used. The second sub-counter 502 outputs a counter value indicating the scanning order according to the (counter output algorithm) shown in FIG. That is, (x, y) is used as a data write address for the SRAM 2,451.
cx: current x,
cy: current y,
nx: next x,
ny: next y,
In this case, according to the following counter value output algorithm, the counter value, that is, the designated data of the scan order is output.
cx = 0; cy = 0;
if (cx [1: 0]! = 3) {-
ny = cy; nx = cx + 1;
} Else if (cy == 0) {(1)
ny = cx [5: 2] +1; nx = 0;
} Else if (cx == 59) {
if (cy <51) {(2)
ny = cy + 15; nx = 0;
} Else {(3)
ny = 65; nx = (cy-50) x4;
}
} Else {/
ny = cy−1; nx = cx + 1;
}

  By the counter value output according to the above algorithm, scanning is executed in accordance with the connection order of the arrows in FIG. Note that the processing of the counter value output algorithm is switched as shown in (1) to (3) in (B) (counter output algorithm) according to the areas (1), (2), and (3).

FIG. 22 shows the scanning order when the third sub-counter (corresponding to 0 degree) is used. The third sub-counter 503 outputs a counter value indicating the scanning order according to the (counter output algorithm) shown in FIG. That is, (x, y) is used as a data write address for the SRAM 2,451.
cx: current x,
cy: current y,
nx: next x,
ny: next y,
In this case, according to the following counter value output algorithm, the counter value, that is, the designated data of the scan order is output.
cx = 0; cy = 0;
if (cx == 59) {
ny = cy + 1; nx = 0;
} Else {
ny = cy; nx = cx + 1;
}

  By the counter value output according to the above algorithm, the scan is executed in accordance with the connection order of the arrows in FIG.

FIG. 23 shows the scan order when the fourth sub-counter (corresponding to +14 degrees) is used. The fourth sub-counter 504 outputs a counter value indicating the scanning order in accordance with (counter output algorithm) shown in FIG. That is, (x, y) is used as a data write address for the SRAM 2,451.
cx: current x,
cy: current y,
nx: next x,
ny: next y,
In this case, according to the following counter value output algorithm, the counter value, that is, the designated data of the scan order is output.
cx = 59; cy = 0;
if (cx [0]! = 0) {-
ny = cy; nx = cx−1;
} Else if (cy == 0) {(1)
ny = 30−cx [5: 1]; nx = 59;
} Else if (cx == 0) {
if (cy <36) {(2)
ny = cy + 30; nx = 59;
} Else {(3)
ny = 65; nx = 59− (cy−35) × 2;
}
} Else {¥
ny = cy-1; nx = cx-1;
}

  By the counter value output according to the above algorithm, scanning is executed in accordance with the connection order of the arrows in FIG. Note that the processing of the counter value output algorithm is switched as shown in (1) to (3) in (B) (counter output algorithm) according to the areas (1), (2), and (3).

FIG. 24 shows the scan order when the fifth sub-counter (corresponding to +27 degrees) is used. The fifth sub-counter 505 outputs a counter value indicating the scanning order according to the (counter output algorithm) shown in FIG. That is, (x, y) is used as a data write address for the SRAM 2,451.
cx: current x,
cy: current y,
nx: next x,
ny: next y,
In this case, according to the following counter value output algorithm, the counter value, that is, the designated data of the scan order is output.
cx = 59; cy = 0;
if (cx [1: 0]! = 0) {−
ny = cy; nx = cx−1;
} Else if (cy == 0) {(1)
ny = 15−cx [5: 2]; nx = 59;
} Else if (cx == 0) {
if (cy <51) {(2)
ny = cy + 15; nx = 59;
} Else {(3)
ny = 65; nx = 59− (cy-50) × 4;
}
} Else {¥
ny = cy-1; nx = cx-1;
}

  By the counter value output according to the above algorithm, scanning is executed in accordance with the connection order of the arrows in FIG. Note that the processing of the counter value output algorithm is switched as shown in (1) to (3) in (B) (counter output algorithm) according to the areas (1), (2), and (3).

As described above, the counter 408 of the image conversion unit 141 illustrated in FIG. 16 includes the five counters 501 to 505 and the selection unit 511 as described with reference to FIG.
The first sub-counter has an inclination angle θ≈−27 degrees (arctan (−1/2)),
The second sub-counter has an inclination angle θ≈−14 degrees (arctan (−1/4)),
The third sub-counter has an inclination angle θ≈0 degrees (arctan (0)),
The fourth sub-counter has an inclination angle θ≈ + 14 degrees (arctan (+1/4)),
The fifth sub-counter has an inclination angle θ≈ + 27 degrees (arctan (+1/2)),
Are output in accordance with the algorithm described with reference to FIGS.

  As shown in FIG. 16, the counter 408 reads the address of the SRAM 1, 401, inputs angle information from the arithmetic unit 403, that is, rotation angle information to be executed in the normalization process, and sets a write address for the SRAM 2 451. , Output sequentially. Since one of the five counters is selectively output in accordance with the angle information from the calculation unit 403, the configuration can be reduced, and data can be written to the SRAMs 2 and 451 after normalization. At this time, processing using a small memory area of the SRAM 1, 401 is realized.

  The present invention has been described in detail above with reference to specific embodiments. However, it is obvious that those skilled in the art can make modifications and substitutions of the embodiments without departing from the gist of the present invention. In other words, the present invention has been disclosed in the form of exemplification, and should not be interpreted in a limited manner. It is obvious that those skilled in the art can make modifications and substitutions of the embodiments without departing from the gist of the present invention. In order to determine the gist of the present invention, the claims should be taken into consideration.

  In the present specification, the embodiment applied to the face recognition apparatus has been mainly described, but the gist of the present invention is not limited to this. The present invention can be applied to various image processing apparatuses that perform conversion of face image data and store it in a memory or output it to a display.

  The series of processing described in the specification can be executed by hardware, software, or a combined configuration of both. When executing processing by software, the program recording the processing sequence is installed in a memory in a computer incorporated in dedicated hardware and executed, or the program is executed on a general-purpose computer capable of executing various processing. It can be installed and run.

  According to the configuration of an embodiment of the present invention, in an image processing apparatus that performs face identification processing, an image of a first memory (SRAM 1) that stores a face image before normalization is extracted and normalized to execute the second memory. In the configuration in which the process of storing in (SRAM2) is performed, the memory is reduced in size by storing only a part of the first memory (SRAM1) that stores the face image before normalization without storing the entire face. In addition, since the scan sequence as the data writing order to the second memory is changed according to the inclination of the face stored in the first memory (SRAM 1), the memory can be downsized and an accurate normalized image can be obtained. Write processing is realized.

FIG. 1 is a diagram schematically showing the overall configuration of a face recognition system according to an embodiment of the present invention. FIG. 2 is a diagram illustrating a configuration example of the face discriminator 14. FIG. 3 is a diagram illustrating a state in which a detected face image is reduced, shifted, and rotated to be normalized to a face image necessary for face recognition. FIG. 4A is a diagram showing a filter window composed of a Gaussian function configured with 24 × 24 pixels fixed. FIG. 4B is a diagram illustrating a response function including a sine function or a cosine function. FIG. 4C is a diagram showing a Gabor filter obtained by applying the filter window shown in FIG. 4A to the response function shown in FIG. 4B. FIG. 4D is a diagram showing eight Gabor filters obtained by applying filter windows to response functions in eight directions shifted by 22.5 degrees. FIG. 4E is a diagram showing 40 types of Gabor filters obtained by applying filter windows at eight angles θ in the same manner with respect to response functions of five types of frequencies f. FIG. 5 is a diagram showing 82 feature amount extraction positions provided in a normalized 60 × 66 pixel face image. FIG. 6 is a diagram for explaining the details of the image conversion process executed in the image conversion unit 141. FIG. 7 is a diagram for explaining processing for obtaining the pixel position of the SRAM 1 corresponding to the pixel position 2 of the SRAM 2. FIG. 8 is a diagram for explaining pixel interpolation processing executed by the interpolation processing unit 207 of the image conversion unit 141. FIG. 9 is a diagram illustrating a configuration of pixels in the face image storage area of the SRAM 2 251 that stores the face image after normalization (image conversion processing). FIG. 10 is a diagram illustrating a configuration of pixels in the face image storage area of the SRAM 1 201 storing the face image before normalization (image conversion processing). FIG. 11 is an enlarged view of the vicinity of the origin area of the SRAM 1, 201 shown in FIG. FIG. 12 is a diagram for explaining the corresponding pixel calculation process required in the normalization process. FIG. 13 is a diagram illustrating a processing example to which the bi-linear method is applied as an example of a pixel value determination processing in the interpolation processing unit 207. FIG. 14 is a diagram illustrating a configuration example of the SRAM 1 that stores only a part of face image data. FIG. 15 is a diagram for explaining a data update example of the SRAM 1 that stores only a part of face image data. FIG. 16 is a diagram illustrating a configuration example of an image conversion unit including the SRAM 1 that stores only a part of face image data. FIG. 17 is a diagram for explaining the scan order as the order of writing the images after normalization. FIG. 18 is a diagram illustrating a counter configuration example of the image conversion unit having the SRAM 1 that stores only a part of face image data. FIG. 19 is a diagram for explaining a usage example of the counter of the image conversion unit having the SRAM 1 that stores only part of the face image data. FIG. 20 is a diagram for explaining a scan example and a counter value output algorithm to which the counter of the image conversion unit having the SRAM 1 that stores only part of the face image data is applied. FIG. 21 is a diagram illustrating a scan example and a counter value output algorithm to which the counter of the image conversion unit having the SRAM 1 that stores only part of the face image data is applied. FIG. 22 is a diagram for explaining a scan example and a counter value output algorithm to which the counter of the image conversion unit having the SRAM 1 that stores only part of the face image data is applied. FIG. 23 is a diagram illustrating a scan example and a counter value output algorithm to which the counter of the image conversion unit having the SRAM 1 that stores only part of the face image data is applied. FIG. 24 is a diagram for explaining a scan example and a counter value output algorithm to which the counter of the image conversion unit having the SRAM 1 that stores only part of the face image data is applied.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Face recognition system 11 Image reducer 12 Face detector 13 Both-eye position detector 14 Face identifier 15 Memory 141 Image conversion part 142 Gabor filter application part 143 Similarity calculation part 144 Match determination part 200 SDRAM
201 SRAM1
202 Eye position detection unit 203 Calculation unit 204 Address creation unit 205 Coordinate conversion unit 206 Counter 207 Interpolation processing unit 251 SRAM 2
401 SRAM1
402 Eye position detection unit 403 Calculation unit 404 SDRAM address generation unit 405 SRAM1 address generation unit 406 Counter 407 Address Y generation unit 408 Counter 409 Coordinate conversion unit 410 Interpolation processing unit 451 SRAM2
501 to 505 counter 511 selection unit

Claims (13)

An image processing apparatus for performing identification processing of an input face image,
An image conversion unit that inputs a face image to be subjected to identification processing, performs image conversion on the input face image, and performs processing to normalize to an image having a predetermined setting;
A similarity calculation unit that calculates the similarity between the feature amount obtained from the converted input image converted by the image conversion unit and the feature amount of the registered image;
A match determination unit that determines whether the input face image and the registered face image match based on the similarity calculated by the similarity calculation unit;
The image conversion unit
A configuration for acquiring a face image from the first memory storing the face image to be subjected to normalization processing, executing normalization by image conversion, and storing the normalized face image in the second memory And
The first memory stores only a part of the face image to be normalized, and stores the data stored in the first memory in accordance with the progress of the writing of the image after the normalization process to the second memory. An image processing apparatus having a memory control unit that executes update processing. The memory control unit
An address comparison unit that compares the write address of the second memory with the vertical direction (Y component) of the corresponding pixel position of the first memory corresponding to the write address, and based on the comparison information of the address comparison unit The update processing of the data stored in the first memory is executed so that the corresponding pixel position of the first memory is substantially in the center in the longitudinal (Y) direction in the data stored in the first memory. The image processing apparatus according to claim 1. The image conversion unit
A counter that inputs angle information of image rotation to be executed in the normalization process and outputs a counter value corresponding to the second memory write address according to the angle information;
The counter is
The image processing apparatus according to claim 1, further comprising a plurality of sub-counters that output different counter values corresponding to different angles. Each of the sub-counters
Image rotation angle: θ = arctan (± 1 / n)
Where n is an integer,
4. The image processing apparatus according to claim 3, wherein a counter value corresponding to a second memory write address corresponding to the angle is output. The image conversion unit
An arithmetic unit for calculating a conversion parameter for calculating a corresponding point of the first memory corresponding to each pixel position of the second memory;
A counter that inputs image rotation angle information included in the conversion parameter calculated by the calculation unit, and outputs a counter value different from the counter value corresponding to the second memory write address according to the input angle information;
A coordinate conversion unit that calculates a corresponding point of the first memory corresponding to each pixel position of the second memory based on the conversion parameter calculated by the calculation unit;
An interpolation processing unit for determining a pixel value corresponding to each pixel position of the second memory according to the corresponding point information calculated by the coordinate conversion unit;
The image processing apparatus according to claim 1, wherein the image processing apparatus includes: The computing unit is
As the conversion parameter, at least one of an image reduction process, an image rotation process, and an image translation process applied when converting the face image stored in the first memory into the face image stored in the second memory Calculate the conversion parameter that defines the processing of
The coordinate converter is
Applying a conversion parameter that defines at least one of image reduction processing, image rotation processing, and image parallel movement processing, to calculate corresponding points in the first memory corresponding to each pixel position in the second memory The image processing apparatus according to claim 5, wherein the image processing apparatus is configured to execute the process. An image processing method for identifying an input face image in an image processing apparatus,
An image conversion step in which an image conversion unit inputs a face image to be subjected to identification processing, executes image conversion on the input face image, and executes processing for normalizing to an image having a predetermined setting;
A similarity calculation step in which the similarity calculation unit calculates the similarity between the feature amount obtained from the converted input image converted by the image conversion unit and the feature amount of the registered image;
A match determination step for determining whether or not the input face image and the registered face image match based on the similarity calculated by the similarity calculation unit;
The image conversion step includes
A step of acquiring a face image from the first memory storing the face image to be subjected to normalization processing, executing normalization by image conversion, and storing the normalized face image in the second memory When,
The memory control unit stores only a part of the face image to be normalized in the first memory, and the memory control unit stores the first memory in accordance with the progress of the writing of the image after the normalization process to the second memory. An image processing method comprising: a memory control step for executing a stored data update process. The memory control step includes
The write address of the second memory is compared with the vertical direction (Y component) of the corresponding pixel position of the first memory corresponding to the write address, and the corresponding pixel position of the first memory is determined based on the comparison information. 8. The image processing method according to claim 7, wherein the update processing of the data stored in the first memory is executed so as to be substantially centered in the longitudinal (Y) direction in the data stored in the first memory. 9. The image conversion unit includes a counter that inputs angle information of image rotation to be executed in normalization processing and outputs a counter value corresponding to the second memory write address in accordance with the angle information,
The counter includes a plurality of sub-counters that output different counter values according to different angles, selects a sub-counter to be applied according to input angle information, and outputs a counter value of the selected sub-counter The image processing method according to claim 7. Each of the sub-counters
Image rotation angle: θ = arctan (± 1 / n)
Where n is an integer,
10. The image processing method according to claim 9, wherein a counter value corresponding to a second memory write address corresponding to the angle is output. The image conversion unit
A calculation step for calculating a conversion parameter for calculating a corresponding point of the first memory corresponding to each pixel position of the second memory;
A counter value that outputs angle information of image rotation included in the conversion parameter calculated in the calculation step and outputs a counter value different from the counter value corresponding to the second memory write address according to the input angle information Steps,
A coordinate conversion step in which a coordinate conversion unit calculates a corresponding point in the first memory corresponding to each pixel position in the second memory based on the conversion parameter calculated in the calculation step;
An interpolation processing step in which an interpolation processing unit determines a pixel value corresponding to each pixel position of the second memory according to the corresponding point information calculated in the coordinate conversion step;
The image processing method according to claim 7, further comprising: The calculation step includes:
As the conversion parameter, at least one of an image reduction process, an image rotation process, and an image translation process applied when converting the face image stored in the first memory into the face image stored in the second memory Calculate the conversion parameter that defines the processing of
The coordinate transformation step includes
Applying a conversion parameter that defines at least one of image reduction processing, image rotation processing, and image parallel movement processing, to calculate corresponding points in the first memory corresponding to each pixel position in the second memory The image processing method according to claim 11, wherein the processing is performed. In the image processing apparatus, a computer program for performing input face image identification processing,
An image conversion step for causing the image conversion unit to input a face image to be subjected to identification processing, executing image conversion on the input face image, and performing processing to normalize to an image having a predetermined setting;
A similarity calculation step for causing the similarity calculation unit to calculate the similarity between the feature amount obtained from the converted input image converted by the image conversion unit and the feature amount of the registered image;
A match determination step for causing the match determination means to determine whether or not the input face image and the registered face image match based on the similarity calculated by the similarity calculation unit;
The image conversion step includes
A step of acquiring a face image from the first memory storing the face image to be normalized and executing normalization by image conversion to store the normalized face image in the second memory When,
Under the control of the memory control unit, only a part of the face image to be normalized is stored in the first memory, and in accordance with the progress of the writing of the image after the normalization process to the second memory, the first memory A computer program comprising a memory control step for executing a process of updating stored data in one memory.