JP2018088183A - Image recognition apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image recognition apparatus excellent in the capability of fingertip detection.SOLUTION: An image recognition apparatus is configured with a hand-area detection part 110 that detects a hand area of a hand 80 from image information of the hand 80 through binarization processing, a longer-direction detection part 120 that detects a longer direction indicative of a fingertip direction in the hand area, a center detection part 130 that detects a center position of a palm in the hand area, a candidate extraction part 140 that extracts a fingertip candidate or finger-crotch candidate in a contour part of the hand area, a fingertip candidate integration part 150 that integrates the fingertip candidates extracted by the candidate extraction part 140 according to a predetermined condition, and a fingertip position specification part 160 that specifies a fingertip position of the hand from the fingertip candidates integrated by the fingertip candidate integration part 150.SELECTED DRAWING: Figure 2

Description

  The present invention relates to an image recognition device, and more particularly, to an image recognition device having an operator's fingertip detection function.

  As a conventional technique, an image recognition device that binarizes an image captured by an infrared camera and detects a fingertip position from the image is known (for example, see Patent Document 1).

  This image recognition device calculates the shortest distance from the point inside the hand region to the contour for the image after binarization processing and isolated point removal, detects the position where the shortest distance is maximum, Is recognized as the center position of the palm. An angle-distance curve diagram is obtained in which the angle from the center position to the horizontal axis (from 90 degrees to the right to 90 degrees) and the vertical axis to the distance to the contour (in each direction) at each angle are obtained. The maximum value appears in the direction (angle) of the fingertip of the hand region image. The fingertip position detection unit obtains the coordinates of the fingertip position using the position on the contour line in the direction of taking the maximum value as the fingertip position. In addition, a local minimum value between a plurality of local maximum values appears in the direction (angle) of the finger or the part between the fingers. It is said that the fingertip position can be obtained by such a method.

International Publication No. 2015/037273

  However, in the image recognition apparatus disclosed in Patent Document 1, in the case of a hand posture in which a finger is extended and the gap between the finger and the finger is closed, depending on how the hand is tilted with respect to the camera and the resolution of the camera, it should occur between the finger and the finger. There is a problem that the concave shape of the contour cannot be captured, and as a result, the convex shape of the fingertip cannot be detected.

  Therefore, an object of the present invention is to provide an image recognition device having excellent fingertip detection capability.

[1] In order to achieve the above object, a hand region detection unit that detects the hand region of the hand by binarization processing from hand image information, and a longitudinal direction detection that detects a longitudinal direction indicating the fingertip direction in the hand region A center detection unit that detects a center position of a palm in the hand region, a candidate extraction unit that extracts a fingertip candidate or a fingertip candidate in a contour portion of the hand region, and a fingertip candidate extracted by the candidate extraction unit An image recognition apparatus, comprising: a fingertip candidate integrating unit that integrates the fingertip candidates on a predetermined condition; and a fingertip position specifying unit that specifies the fingertip position of the hand from the fingertip candidates integrated by the fingertip candidate integrating unit. provide.

[2] The center detection unit creates a distance conversion image from the binarized image obtained by the hand region detection unit, and detects the center position of the palm based on the distance conversion image. The image recognition device according to [1] may be used.

[3] Further, the candidate extraction unit extracts the fingertip candidate or fingertip candidate based on a maximum value or a minimum value from one or more reference points to the contour portion. [1] Alternatively, the image recognition device according to [2] may be used.

[4] Further, in the above [1], the reference point is a point at a center position of the palm and a point on the contour portion passing through the center position and intersecting the longitudinal direction. The image recognition apparatus described may be used.

[5] Further, the fingertip candidate integration unit integrates the fingertip candidates under a predetermined condition, and integrates an average position of a plurality of fingertip candidates sandwiched between the fingertip candidates as a representative position. The image recognition device according to any one of [1] to [4] may be used.

[6] In addition, the fingertip position specifying unit specifies five fingertips having the largest distance from the center position of the palm to the fingertip candidate as fingertips among the fingertip candidates obtained by the fingertip candidate integration unit. The image recognition apparatus according to any one of [1] to [5], characterized in that:

  According to the image recognition device of the present invention, it is possible to provide an image recognition device with excellent fingertip detection capability.

FIG. 1 is a side view showing an example of an arrangement of an image recognition apparatus according to an embodiment of the present invention. FIG. 2 shows an example of a block configuration diagram of the image recognition apparatus according to the embodiment of the present invention. FIG. 3A is an example of a captured hand luminance image, and FIG. 3B is a binary image diagram obtained by binarization processing from the hand image information. FIG. 4 is a coordinate diagram showing a detection method using an inertia main axis for detecting the longitudinal direction. FIG. 5A is an example of a distance conversion image, and FIG. 5B is a binary image diagram showing the center position of the palm obtained from FIG. 5A on the binary image. FIG. 6A is a binary image diagram for explaining that a trace for extracting a fingertip candidate or a fingertip candidate is started from a trace start point, and FIG. 6B is a trace position on the horizontal axis. FIG. 6C is a binary image diagram in which the extracted fingertip candidates and fingertip candidates are indicated by × (fingertip candidates) and ○ (fingertip candidates), respectively, with the vertical axis as the distance between the contour and the palm center. It is. FIG. 7A is a binary image diagram in which the fingertip candidate and the fingertip candidate extracted using the second reference point are indicated by x (fingertip candidate) and ◯ (fingertip candidate), respectively. ) Is a binary image diagram illustrating the fingertip candidate and the fingertip candidate extracted using the third reference point as x (fingertip candidate) and ○ (fingertip candidate), respectively. FIG. 8A is a diagram illustrating integration of fingertip candidates, FIG. 8B is a diagram illustrating integration of fingertip candidates, and FIG. 8C is a diagram illustrating integration of fingertip candidates. FIG. 5 is a binary image diagram illustrating the result of integrating the fingertip candidate and the fingertip candidate under predetermined conditions with × (fingertip candidate) and ○ (fingertip candidate), respectively. FIG. 9 is a binary image diagram illustrating the result of extracting five fingertip candidates and determining the fingertip position as x (fingertip candidates) and ◯ (fingertip candidates). FIG. 10 is a binary image diagram illustrating the result of extracting the fingertip candidate by the conventional method and specifying the fingertip position with × (fingertip candidate) and ○ (fingertip candidate).

(Embodiment of the present invention)
The image recognition device 1 according to the embodiment of the present invention includes a hand region detection unit 110 that detects a hand region of the hand 80 from the image information of the hand 80 by binarization processing, and a longitudinal direction that indicates a fingertip direction in the hand region. A longitudinal direction detecting unit 120 for detecting, a center detecting unit 130 for detecting the center position of the palm in the hand region, a candidate extracting unit 140 for extracting a fingertip candidate or a finger ridge candidate in the contour portion of the hand region, and a candidate extracting unit 140 A fingertip candidate integrating unit 150 that integrates the fingertip candidates extracted by the predetermined conditions, and a fingertip position specifying unit 160 that specifies the fingertip position of the hand from the fingertip candidates integrated by the fingertip candidate integrating unit 150. It is configured.

  FIG. 1 is a side view showing an example of an arrangement of an image recognition apparatus according to an embodiment of the present invention. As shown in FIG. 1, the image recognition apparatus 1 has a camera 10 as an imaging unit attached to the mounting portion 50 and images in a substantially horizontal direction. The camera 10 includes a semiconductor image sensor such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal-Oxide Semiconductor). The illumination 20 illuminates a range that is substantially the same as the imaging range 60 of the camera 10. In addition, although the wavelength of the light to image can use the light of arbitrary frequencies, the system which illuminates and images using infrared light and near-infrared light is preferable. Thereby, an operator does not see illumination light at the time of imaging, and the uncomfortable feeling at the time of imaging can be prevented or reduced.

  The camera 10 images the operator's hand 80 from a predetermined direction, and outputs the captured frame image S (n) to the control unit 100 as image information. In the present embodiment, the predetermined direction is a direction in which the palm side of the operator's palm is imaged.

(Configuration of control unit 100)
FIG. 2 shows an example of a block configuration diagram of the image recognition apparatus according to the embodiment of the present invention. The control unit 100 includes, for example, a CPU (Central Processing Unit) that performs operations and processes on acquired data according to a stored program, a RAM (Random Access Memory) that is a semiconductor memory, a ROM (Read Only Memory), and the like. Microcomputer. In this ROM, for example, a program for operating the control unit 100 is stored. For example, the RAM is used as a storage area for storing a plurality of pieces of image information, temporary calculation results, and the like. Further, the control unit 100 has means for generating a clock signal therein, and executes processing based on this clock signal.

  The control unit 100 includes a hand region detection unit 110, a longitudinal direction detection unit 120, a center detection unit 130, a candidate extraction unit 140, a fingertip candidate integration unit 150, and a fingertip position specifying unit 160.

(Hand area detection)
FIG. 3A is an example of a captured hand luminance image, and FIG. 3B is a binary image diagram obtained by binarization processing from the hand image information. The hand region detection unit 110 detects the hand region of the hand 80 by binarization processing from hand image information based on the frame image S (n) input from the camera 10. The hand area detection unit 110 generates, for example, a luminance image 210 with the background removed as illustrated in FIG. 3A based on the image information, and extracts an area whose brightness is equal to or greater than a predetermined threshold as a hand area 2 A value image 220 is generated. FIG. 3B is a binary image obtained by binarization processing from this type of image information.

(Longitudinal direction detection)
The longitudinal direction detection unit 120 detects the longitudinal direction of the hand 80 shown in FIG. The upward direction is the fingertip direction. As a detection method in the longitudinal direction, for example, a detection method using the following inertial main axis can be considered.

FIG. 4 is a coordinate diagram showing a detection method using an inertia main axis for detecting the longitudinal direction. The longitudinal direction (angle θ) of the region 300 as shown in FIG. 4 is obtained as a solution of the following equation.
However, moment
Is a binary image that takes 1 inside the figure and 0 outside the figure.

(Center detection)
FIG. 5A is an example of a distance conversion image, and FIG. 5B is a binary image diagram showing the center position of the palm obtained from FIG. 5A on the binary image. The center position of the palm is estimated by center detection. The center position of the palm is estimated from the hand region by, for example, the following method. A distance conversion image is created from the binary image shown in FIG. 3B as shown in FIG. 5A, and the pixel having the largest distance conversion value is set as the center position of the palm as shown in FIG. 5B. And Here, the distance conversion image is an image in which the pixel value of the pixel in the region on the binary image is replaced with the shortest distance from the contour of the region.

(Candidate extraction)
FIG. 6A is a binary image diagram for explaining that a trace for extracting a fingertip candidate or a fingertip candidate is started from a trace start point, and FIG. 6B is a trace position on the horizontal axis. FIG. 6C is a binary image diagram in which the extracted fingertip candidates and fingertip candidates are indicated by × (fingertip candidates) and ○ (fingertip candidates), respectively, with the vertical axis as the distance between the contour and the palm center. It is.

  The first reference point is a point SP1 shown in FIG. 6A obtained by center detection. The distance between all the pixels that form the contour of the hand area and the first reference point (point SP1) that is the center of the palm is calculated, and the pixel that has the maximum value is the fingertip candidate and the pixel that has the minimum value is the fingertip candidate. Extract.

  As shown in FIG. 6A, the distance between the contour pixel and the first reference point (SP1) is calculated along the contour of the trace start / end point clockwise. The result is as shown in FIG. 6B in which the horizontal axis represents the trace position and the vertical axis represents the distance between the contour and the first reference point (point SP1). When the local maximum / minimum value is extracted from the calculated distance, many noises (small peaks and valleys) are included. Therefore, the local maximum / minimum value is extracted after rounding the distance data by moving average or the like. (The noise removal method is not particularly limited to moving average.)

  FIG. 6C is a binary image in which the fingertip candidates and fingertip candidates extracted from the maximum / minimum values in FIG. 6B are indicated by × (fingertip candidates) and ◯ (fingertip candidates) on the contour, respectively. FIG. From this binary image diagram, it can be seen that the index finger, ring finger, and little finger are not detected. Note that if the removal of distance noise in the preprocessing is eliminated or rounding is reduced, the undetected rate decreases, but the false detection rate increases due to an increase in noise.

(Candidate extraction by the second and third reference points)
In the previous step, the fingertip / finger candidate is extracted using the reference point as the center of the palm and the distance between the reference point and the contour pixel. In this step, the fingertip / finger is further extracted using the following two reference points. Candidates are also extracted. When the fingertip direction (longitudinal direction) is 0 ° as viewed from the center of the palm, the palm ends perpendicular to each other, that is, ± 90 °, are defined as the second reference point (SP2) and the third reference point (SP3). The second reference point (SP2) and the third reference point (SP3) are not limited to points on the contour portion orthogonal to the longitudinal direction, and may be points on the contour portion in the direction of ± 45 ° to 135 °. Good.

  FIG. 7A is a binary image diagram in which the fingertip candidate and the fingertip candidate extracted using the second reference point are indicated by x (fingertip candidate) and ◯ (fingertip candidate), respectively. ) Is a binary image diagram illustrating the fingertip candidate and the fingertip candidate extracted using the third reference point as x (fingertip candidate) and ○ (fingertip candidate), respectively. By extracting a fingertip candidate and a fingertip candidate using the second reference point (SP2) and the third reference point (SP3) at the end of the palm perpendicular to the fingertip direction (longitudinal direction) when viewed from the center of the palm, It can be seen that the index finger, ring finger, and little finger that could not be detected by extraction using the reference point (SP1) can be detected.

(Fingertip candidate integration)
FIG. 8A is a diagram illustrating integration of fingertip candidates, FIG. 8B is a diagram illustrating integration of fingertip candidates, and FIG. 8C is a diagram illustrating integration of fingertip candidates. FIG. 5 is a binary image diagram illustrating the result of integrating the fingertip candidate and the fingertip candidate under predetermined conditions with × (fingertip candidate) and ○ (fingertip candidate), respectively. For the finger-and-cane candidates extracted using the first reference point (SP1) to the third reference point (SP3), those having a short distance on the contour line are grouped together and the representative position is set as the average position.

  As shown in FIG. 8 (a), if there are two finger fold candidates (circles), if the distance between the candidates (thick line length) is less than or equal to the threshold, the two candidates are grouped together and the representative position is contoured. The upper average position (marked with ◇).

  Further, as shown in FIG. 8B, all fingertip candidates (Δ mark) sandwiched between fingertip candidates (◇ mark) are set to the same group, and the representative position is set to the average position (× mark) on the contour. .

  When the fingertip candidates and fingertip candidates obtained by the above-described processing are shown in an integrated manner, it is as shown in FIG. It can be seen that fingertip candidates (× marks) can also be extracted from the index finger, ring finger, and little finger.

(Fingertip location specification)
FIG. 9 is a binary image diagram illustrating the result of extracting five fingertip candidates and determining the fingertip position as x (fingertip candidates) and ◯ (fingertip candidates). That is, a candidate is extracted in which the distance between the first reference point SP1 (the palm center) and the fingertip candidate is greater than or equal to the threshold. Among the extracted candidates, a maximum of five from the far (large) distance in the fingertip direction viewed from the first reference point SP1 is determined as the fingertip. As shown in FIG. 9, it can be seen that five fingertips of the thumb, the index finger, the middle finger, the ring finger, and the little finger are specified. On the other hand, FIG. 10 is a binary image diagram illustrating the result of extracting a fingertip candidate by a conventional method and specifying the fingertip position as x (fingertip candidate) and ○ (finger or finger candidate). It can be seen that the little finger is not detected or specified.

(Effect of embodiment)
The embodiment of the present invention has the following effects.
(1) The image recognition apparatus 1 according to the present embodiment includes a hand region detection unit 110 that detects a hand region of the hand 80 from the image information of the hand 80 by binarization processing, and a longitudinal direction that indicates a fingertip direction in the hand region A longitudinal direction detection unit 120 that detects a finger, a center detection unit 130 that detects a center position of a palm in a hand region, a candidate extraction unit 140 that extracts a fingertip candidate or a fingertip candidate in a contour portion of a hand region, and a candidate extraction unit A fingertip candidate integration unit 150 that integrates the fingertip candidates extracted by 140 under predetermined conditions; and a fingertip position specification unit 160 that specifies the fingertip position of the hand from the fingertip candidates integrated by the fingertip candidate integration unit 150. Configured. In the candidate extraction for extracting the fingertip candidate or fingertip candidate in the contour part of the hand region, the fingertip candidate or fingertip candidate is extracted using the first reference point (SP1) to the third reference point (SP3). The fingertip that could not be detected by the method can be detected, and the fingertip detection rate can be improved. Accordingly, it is possible to provide an image recognition device having excellent fingertip detection capability.
(2) When extracting a fingertip candidate or a fingertip candidate, if a maximum / minimum value is extracted from the distance calculated from the distance between the contour pixel and the reference point, a lot of noises (small peaks and valleys) are included, so a moving average The maximum / minimum values are extracted after the distance data is rounded. However, if smoothing processing is performed by moving average or the like for noise removal, the detection accuracy of fingertip candidates and fingertip candidates decreases. The image recognition apparatus 1 according to the present embodiment extracts fingertip candidates or fingertip candidates using the first reference point (SP1) to the third reference point (SP3). Therefore, the influence of noise can be reduced by the smoothing process in the fingertip candidate and fingertip candidate extraction process, and the maximum / minimum values can be obtained using a plurality of reference points (first reference point to third reference point). Since extraction is performed, the detection rate of the fingertip candidate and the fingertip candidate does not decrease. Accordingly, it is possible to provide an image recognition device having excellent fingertip detection capability.

  As mentioned above, although some embodiment and modification of this invention were demonstrated, these embodiment and modification are only examples, and do not limit the invention based on a claim. These novel embodiments and modifications can be implemented in various other forms, and various omissions, replacements, changes, and the like can be made without departing from the scope of the present invention. it can. In addition, not all combinations of features described in these embodiments and modifications are necessarily essential to the means for solving the problems of the invention. Furthermore, these embodiments and modifications are included in the scope and gist of the invention, and are included in the invention described in the claims and the equivalents thereof.

DESCRIPTION OF SYMBOLS 1 ... Image recognition apparatus, 10 ... Camera, 20 ... Illumination, 50 ... Mounting part, 60 ... Imaging range, 80 ... Hand 100 ... Control part, 110 ... Hand area detection part, 120 ... Longitudinal direction detection part, 130 ... Center detection Unit 140 ... candidate extraction unit, 150 ... fingertip candidate integration unit, 160 ... fingertip position specifying unit 210 ... luminance image, 220 ... binary image 300 ... region θ ... angles SP1, SP2, SP3 ... first reference point, second reference Point, third reference point

Claims (6)

A hand region detection unit for detecting the hand region of the hand by binarization processing from image information of the hand;
A longitudinal direction detection unit for detecting a longitudinal direction indicating a fingertip direction in the hand region;
A center detector for detecting the center position of the palm in the hand region;
A candidate extraction unit for extracting a fingertip candidate or a fingertip candidate in the contour portion of the hand region;
A fingertip candidate integration unit that integrates the fingertip candidates extracted by the candidate extraction unit under a predetermined condition;
A fingertip position specifying unit for specifying the fingertip position of the hand from the fingertip candidates integrated by the fingertip candidate integration unit;
An image recognition apparatus comprising:   The center detection unit creates a distance conversion image from the binarized image obtained by the hand region detection unit, and detects the center position of the palm based on the distance conversion image. The image recognition apparatus according to 1.   3. The image according to claim 1, wherein the candidate extraction unit extracts the fingertip candidate or the fingertip candidate based on a maximum value or a minimum value from one or more reference points to the contour portion. Recognition device.   The image recognition apparatus according to claim 3, wherein the reference point is a point at a center position of the palm and a point on the contour portion that passes through the center position and intersects the longitudinal direction.   The fingertip candidate integration unit integrates the fingertip candidates under a predetermined condition, and integrates an average position of a plurality of fingertip candidates sandwiched between the fingertip candidates as a representative position. 5. The image recognition apparatus according to any one of 4 above.   The fingertip position specifying unit specifies, as fingertips, five of the fingertip candidates obtained by the fingertip candidate integration unit from those having a large distance from the center position of the palm to the fingertip candidate. Item 6. The image recognition device according to any one of Items 1 to 5.