JP2008027273A - Rotation angle detection device and rotation angle detection method - Google Patents

Abstract

A rotation angle detection device and a rotation angle detection method capable of easily and quickly detecting an inclination of an object such as a human face from a reference posture, that is, a rotation angle of an object image in an image plane. provide.
Frequency conversion means 201 for obtaining a periodic function (sine function and cosine function) related to a change in luminance level detected by scanning an object along a predetermined path R, and power of the periodic function obtained by the frequency conversion means 201 And a rotation angle conversion unit 204 that detects a rotation angle with respect to a reference posture of the object based on a phase obtained from a spectrum ratio.
[Selection] Figure 2

Description

  The present invention relates to a rotation angle detection device and a rotation angle detection method for detecting a rotation angle that is an angle of inclination of an object such as a human face with respect to a reference posture.

  Face detection is performed assuming various applications. For example, entrance / exit management and the like are performed by ID photos and face recognition. In the case of an ID photo, the position, size, face direction, etc. of the face in the photo are strictly defined. Various techniques have already been proposed for face detection and face recognition, but all assume that the face direction is upright. For this reason, establishing a technique for detecting the direction of the face and correcting it upright has a great impact on improving the accuracy of face detection and face recognition and improving the spread of applications.

  Conventionally, in order to detect the orientation of a face using a camera, markers are pasted on the corners and mouths of both eyes, which are feature points of the face, and the measured values of the distance between the eyes and the distance between the eyes and the mouth are used as a detection device. In contrast to technical proposals for registering and detecting the tilt of a face using the registered values, it is complicated to attach a marker to the face and to measure the distance between eyes and the distance between eyes and mouth Proposals have been made with the technical challenge of eliminating this problem.

In this proposal, reference parameters are generated in advance from the barycentric points and facial feature points of the skin color area of the face, and based on the rotation width and pitch in the three-axis directions determined in advance centered on the rotation position estimated from the reference parameters. Coordinate position information is generated, information serving as a reference for the relative positional relationship between the barycentric point of the skin color area and the facial feature point is stored, and the relative point between the barycentric point of the skin color area and the facial feature point is recorded from the captured face image The positional relationship information is calculated, and the inclination of the face is detected based on the calculated information and reference information (see Patent Document 1).
On the other hand, proposals have already been made for a face detection apparatus and a face detection method that can detect even a non-front face whose face is tilted or has a different orientation.

  In this proposal, a correlation between the input image and a template image having a predetermined size representing an average face image is obtained by the correlation calculation means, and it is determined whether the face image is included in the input image based on the obtained correlation. When it is determined that the face image is not included in the input image, a template image obtained by rotating the template image by a predetermined angle around an axis perpendicular to the screen is applied to obtain a correlation with the input image. When it is determined that a face image is included in the input image, an initial template is applied to obtain a correlation with the next input image.

Even when the face image included in the input image is not a front face, detection processing can be performed using a front face template image, and matching processing speed is improved (see Patent Document 2).
Furthermore, in the conventional face image segmentation method, the average face is used to normalize the position and size of the face in the general image. The position is often shifted and it is difficult to detect accurately because it is not oriented in a certain direction. Furthermore, in the method using the template, the template of the eye, nose, etc. becomes a small image, so that the background There is also a proposal to deal with the problem of being confused with miscellaneous objects and to make it possible to perform recognition with higher accuracy than before.

  In this proposal, first, a grayscale face image obtained by photographing a face is input. On the other hand, a large number of deformation parameters based on an affine transformation taking into account all the expected deformations are prepared in advance. Using each of them, a face pattern is cut out from the input image, and the cut out face pattern is inversely transformed into a square normalized pattern. Next, feature vectors are extracted from the normalized face patterns obtained by the respective inverse transformations, compared with reference feature vectors of various individuals prepared in advance, similarity is calculated, and an identification result is output.

In addition, the similarity output from each discriminator is multiplied by a weighting factor based on the frequency of occurrence of the deformation removed by each corresponding inverse transform to increase the final recognition accuracy.
Inverse transformation, identification, and multiplication for each deformation are executed in parallel, and finally, these identification results are collected to determine whether or not a specific individual is a comprehensive result (see Patent Document 3). ).
Japanese Patent No. 3454726 (paragraphs 0002 to 0007, FIG. 1) Japanese Patent Laying-Open No. 2004-133637 (paragraphs 0008 to 0010, FIG. 1) JP 2000-90191 A (paragraphs 0004 to 0006, paragraph 0019, FIG. 1)

However, in the technique proposed in Patent Document 1 described above, the apparent color range, that is, the skin color range, varies greatly depending on differences in conditions such as race differences and lighting conditions. In practice, it is extremely difficult to detect the tilt of the face, that is, the rotation angle of the object with sufficient accuracy without being affected by the difference.
Further, in the technique proposed in Patent Document 2, it is necessary to prepare an extremely large number of templates according to the shape and orientation of the face, and the template matching requires a large amount of calculation. Not very suitable for doing.

Furthermore, in the technique proposed in Patent Document 3, preparing in advance a large number of deformation parameters based on affine transformations that take into account all possible deformations is already a burden on itself, and it is easy and quick for objects. It is difficult to deal with the difference in rotation angle.
In view of the above situation, the present invention provides a rotation angle detection device that can easily and quickly detect an inclination of an object such as a human face from a reference posture, that is, a rotation angle of an object image in an image plane. It is another object of the present invention to provide a rotation angle detection method.

In order to solve the above problems, the present application proposes the following technologies.
(1) Frequency conversion means for obtaining a periodic function related to a change in luminance level detected by scanning an object along a predetermined path, and rotation of the object relative to a reference posture based on the phase of the periodic function obtained by the frequency conversion means And a rotation angle conversion means for detecting the angle.

In the rotation angle detection device of (1) above, a periodic function related to a change in luminance level detected by scanning the object along a predetermined path is obtained by the frequency conversion means. Based on the phase of the obtained periodic function, the rotation angle conversion means detects the rotation angle with respect to the reference posture of the object.
With a relatively simple calculation, for example, the rotation angle of an object such as a human face with respect to a reference posture can be detected.

(2) The frequency converting unit is configured to generate a plurality of periodic functions having different initial phases, and the rotation angle converting unit is configured to generate a plurality of periodic functions having different initial phases generated by the frequency converting unit. The rotation angle detection device according to (1), wherein the rotation angle is detected based on a power spectrum ratio.
In the rotation angle detection device of (2) above, in particular with respect to the rotation angle detection device of (1), the frequency conversion means generates a plurality of periodic functions having different initial phases such as a sine function and a cosine function, for example, The rotation angle conversion means detects the rotation angle based on the ratio of power spectra of a plurality of periodic functions having different initial phases.
By a relatively simple calculation, for example, the rotation angle of an object such as a human face with respect to a reference posture can be accurately detected.

(3) The frequency conversion unit is configured to generate the periodic function based on an average value of luminances of adjacent pixels with respect to pixels arranged in a predetermined path for performing the scanning. (1) The rotation angle detection apparatus.
In the rotation angle detection device of (3) above, the frequency conversion means is based on the average value of the luminance of each adjacent pixel with respect to the pixels arranged in a predetermined path for performing the scanning, particularly with respect to the rotation angle detection device of (1). Since the periodic function is generated, the rotation angle of the object can be detected robustly against various noises.

(4) When a person's face is selected as the object, the frequency conversion means is configured to select a scanning path that is inside the outline of the person's face and passes through the eyes of the face. (1) The rotation angle detection apparatus characterized by these.
In the rotation angle detection device of (4) above, particularly when a person's face is selected as the object, the frequency conversion means is located inside the contour of the person's face, with respect to the rotation angle detection device of (1). In order to select the scanning path that passes through the eyes of the face, at least the data of the eyes of the face (both one eye and both eyes are acceptable), which is a prominent or dominant feature part of the face detection, are considered In addition, the rotation angle of the object can be detected robustly against illumination light and other various variations.

(5) The rotation angle conversion means stores, in advance as reference data, data representing a correspondence relationship between a power spectrum ratio of a plurality of periodic functions having different initial phases generated by the frequency conversion means and the rotation angle. The rotation angle is detected by comparing the ratio of the power spectrum of the plurality of periodic functions having different initial phases generated by the frequency conversion means and the reference data held in the storage means. (2) The rotation angle detection apparatus characterized by the above-mentioned.

  In the rotation angle detection device of (5) above, in particular with respect to the rotation angle detection device of (2), the rotation angle conversion means is the ratio of the power spectrum of the various periodic functions generated by the frequency conversion means and the rotation angle. Is stored in advance as reference data in the storage means, and the ratio of the power spectra of the plurality of periodic functions having different initial phases generated by the frequency conversion means and the reference data held in the storage means In contrast, since the rotation angle is detected, the rotation angle of the object can be detected easily and accurately.

(6) The rotation angle detection device according to (2), wherein the frequency conversion means is configured to generate a sine function and a cosine function as a plurality of types of periodic functions.
In the rotation angle detection device of the above (6), the frequency conversion means generates a sine function and a cosine function as a plurality of periodic functions, particularly with respect to the rotation angle detection device of (2). Becomes simple, and the rotation angle of the object can be detected quickly.

(7) The rotation angle detection according to any one of (1) to (6), further comprising resizing means for resizing an image including an object to be scanned by the frequency converting means to a certain size. apparatus.
In the rotation angle detection device according to (7) above, in particular with respect to the rotation angle detection device according to any one of (1) to (6), the image including the object to be scanned by the frequency conversion means is resized to a certain size. In addition, since the resizing means is further provided, it is possible to detect an accurate rotation angle in which uniformity is maintained with respect to application of a reference for detection to each object to be detected.

(8) Obtaining a periodic function related to a change in luminance level detected by scanning the object along a predetermined path, and detecting a rotation angle with respect to the reference posture of the object based on the phase of the obtained periodic function A feature of a rotation angle detection method.
In the above rotation angle detection method (8), the rotation angle of an object such as a human face with respect to a reference posture can be detected by a relatively simple calculation.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the drawings to be referred to below, for the sake of convenience, the main part that is the subject of the description is exaggerated as appropriate, and other than the main part is appropriately simplified or omitted.

  FIG. 1 is a conceptual diagram showing a technique according to an embodiment of the present invention. FIG. 1A shows a state in which the midline of the face is in a normal (reference) posture along a line that bisects the image symmetrically, and FIG. 1B shows that the midline of the face is an image. Represents a state in which the upper side is inclined to the left with respect to a line that bisects the image bilaterally (that is, rotated counterclockwise in the image plane), and FIG. This represents a state in which the upper side is inclined to the right with respect to the bisecting line (that is, rotated right in the image plane).

In this example, in any case, the image 100 is extracted (extracted) through resizing means for resizing the image including the object (person's face) to be scanned to a certain size. )
In the present embodiment, an image of a person's face (face partial image) 100 as an object is scanned along a predetermined path R shown by a series of short arrows, and the pixels P1 and P1 to be sampled are scanned. P2, P3, P4, P5,... Pn-2, Pn-1, and a periodic function related to a change in luminance level detected for Pn are obtained, and a reference of the object is obtained based on the phase of the obtained periodic function. The rotation angle with respect to the posture is detected.

In the present embodiment, a sine function and a cosine function, that is, a plurality of (two in this case) periodic functions having different initial phases are generated as the above-described periodic functions, and these initial phases differ by the rotation angle conversion means. The rotation angle is detected based on the ratio of power spectra of a plurality of periodic functions.
A sine function as a periodic function related to a change in luminance level detected for each of the pixels P1, P2, P3, P4, P5,... Pn-2, Pn-1, and Pn is expressed by the following formula 1 by Fourier transform. In addition, the cosine function has a power spectrum represented by the following Equation 2 by Fourier transform. And based on these, the calculation of Formula 3 is executed to calculate the rotation angle.

It should be noted that the pixels used to generate Equations 1 and 2 are sampled one by one for each pixel along P1, P2, P3, P4, P5,... Pn-2, Pn-1, Pn and path R as described above. However, the present invention is not limited to this, and the periodic function may be generated based on the average value of the brightness of each adjacent pixel with respect to the pixels arranged in a predetermined path for performing the scanning. Good.

  There are various ways of taking this average. For example, for each of the above-described pixels P1, P2, P3, P4, P5,... Pn-2, Pn-1, and Pn in FIG. 2, (P3 + P4) / 2,..., (Pn-1 + Pn) / 2 may be selected so that pixels to be averaged do not overlap, or (P1 + P2) / 2, (P2 + P3) / 2, (P3 + P4) / 2,..., (Pn-2 + Pn-1) / 2, (Pn-1 + Pn) / 2, even if the pixels to be averaged are selected so as to overlap one by one Well, moreover, (P1 + P2 + P3) / 3, (P2 + P3 + P4) / 3, (P3 + P4 + P5) / 3,. Average) There. In either case, it is possible to perform more robust detection (stable generation) against various noises.

FIG. 2 is a block diagram showing the configuration of the rotation angle detection device 200 of the present invention. In the figure, the frequency conversion unit 201 converts a sine function and a cosine function as a plurality of periodic functions having different initial phases, that is, a plurality of periodic functions having different initial phases (in this case, two types). Generate.
These periodic functions are generated by sampling each pixel along the selected path R, such as P1, P2, P3, P4, P5,... Pn-2, Pn-1, and Pn. However, as described above, the periodic function may be generated by the average luminance calculation unit 202 based on the average value of the luminance of adjacent pixels.

It should be noted that the selection of the path R is desirably a pixel that better reflects the facial features and changes greatly according to the rotation. When the object image is a person's face, both eyes 101 and 102 (FIG. 1) well represent facial features. That is, in general, the luminance of a portion above a certain pixel is lower than that of surrounding portions. Moreover, since it exists across the axis of symmetry (midline), it is presumed that this is an effective feature for detecting the rotation angle.
On the other hand, locations with large fluctuations are undesirable depending on the individual or shooting conditions. For example, it is desirable to avoid the vicinity of the lip 103 because of the influence of the presence or absence of wrinkles, a mask, or the like. The arrangement of the pixels selected so as to satisfy the above conditions is a portion (that is, a path R) shown by a series of short arrows in FIG.

In addition, each image that is a target for detecting the rotation angle of the object (person's face) is cut out (extracted) as an image having a certain size through resizing means 203 that resizes the image including the object to a certain size. ).
For this resizing, techniques such as the bicubic method and the bilinear method are applied.
The rotation angle conversion unit 204 obtains power spectra of two periodic functions of Equation 1 and Equation 2, and detects the rotation angle based on the ratio of the power spectrum of the periodic function by Equation 3.

  FIG. 3 is a diagram showing data (reference data) representing the correspondence between the phase angle and the angle (rotation angle) of the object. In the embodiment of the present invention, reference data (phase angle-tilt angle characteristics) as shown in this diagram is preliminarily converted into the ratio of power spectra of two kinds of periodic functions having different initial phases. Data corresponding to the inclination angle (rotation angle) of the object and stored in an appropriate memory (storage means) in the rotation angle conversion means 204 or outside the means.

Then, the rotation angle is detected by comparing the ratio of the power spectra of a plurality of periodic functions having different initial phases generated by the frequency conversion means 201 with the reference data held in the memory.
The tilt angle (rotation angle) is obtained based on

FIG. 4 is a flowchart showing a process for detecting the rotation angle in the embodiment of the present invention.
When the operation starts, first, the resizing means 203 resizes the area including the object whose tilt angle (rotation angle) is to be detected to a certain size (step S401).
Next, a pixel to be applied to calculation for calculating an inclination angle (rotation angle) is selected from the area resized in step S401 (step S402). This is performed, for example, as described with reference to FIG.

Further, as described above, the average luminance calculation unit 202 obtains the average value of the luminance values of the pixel selected in step S402 and its adjacent pixels (step S403).
Next, the power spectra of the two periodic functions are obtained from the periodic function obtained by the frequency conversion means 201 according to Equation 1 and Equation 2 (step S404).
The phase of the periodic function is calculated from the ratio (formula 3) of the power spectrum of the periodic function obtained in step S404 (step S405).

From the phase obtained in step S405, the rotation angle is obtained by comparing the reference data represented by the diagram of FIG. 3 held in the above-described memory (step S406).
The technical idea described above obtains a periodic function (sine function and cosine function) related to a change in luminance level detected by scanning an object along a predetermined path R, and based on the phase of the obtained periodic function. It can also be seen as a rotation angle detection method characterized by detecting the rotation angle with respect to the reference posture of the object (Equation 3).

Although the above embodiment has been described on the assumption that the image is resized to a certain size, the arrangement of pixels to be used may be changed according to the size of the image.
As described above, according to the present invention, the rotation angle of the object can be detected easily and accurately.

It is a conceptual diagram showing the technique of embodiment of this invention. It is a block diagram showing the structure of the rotation angle detection apparatus of this invention. It is a diagram showing the data (reference data) showing the correspondence between the phase angle and the angle (rotation angle) of the object. It is a flowchart showing the process for the rotation angle detection in embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 200 ... Rotation angle detection apparatus 201 ... Frequency conversion means 202 ... Average brightness calculation means 203 ... Resizing means 204 ... Rotation angle conversion means

Claims (8)

  Frequency conversion means for obtaining a periodic function related to a change in luminance level detected by scanning an object along a predetermined path, and detecting a rotation angle with respect to the reference posture of the object based on the phase of the periodic function obtained by the frequency conversion means And a rotation angle conversion means. A rotation angle detection device comprising:   The frequency converting unit is configured to generate a plurality of periodic functions having different initial phases, and the rotation angle converting unit is configured to generate power spectra of a plurality of periodic functions having different initial phases generated by the frequency converting unit. The rotation angle detection device according to claim 1, wherein the rotation angle is detected based on a ratio.   2. The frequency conversion unit is configured to generate the periodic function based on an average value of luminance of each adjacent pixel with respect to pixels arranged in a predetermined path for performing the scanning. A rotation angle detection device according to claim 1.   When a person's face is selected as the object, the frequency converting means is configured to select a scanning path that is inside the outline of the person's face and passes through the eyes of the face. The rotation angle detection device according to claim 1.   The rotation angle conversion means holds data representing a correspondence relationship between the power spectrum ratios of a plurality of periodic functions with different initial phases generated by the frequency conversion means and the rotation angle in a storage means in advance, The rotation angle is detected by comparing a ratio of power spectra of a plurality of periodic functions having different initial phases generated by the frequency conversion means with reference data held in the storage means. The rotation angle detection device according to claim 2.   The rotation angle detection device according to claim 2, wherein the frequency conversion unit is configured to generate a sine function and a cosine function as a plurality of types of periodic functions.   The rotation angle detection apparatus according to claim 1, further comprising resizing means for resizing an image including an object to be scanned by the frequency converting means to a certain size.   A periodic function related to a change in luminance level detected by scanning an object along a predetermined path is obtained, and a rotation angle with respect to a reference posture of the object is detected based on a phase of the obtained periodic function. Rotation angle detection method.

Images