JP2020071627A - Image processing device and image processing method - Google Patents

Abstract

To increase resolution of an object image by improving positioning accuracy of the object image.SOLUTION: An image processing device includes a high-resolution processing unit. The high-resolution processing unit performs high-resolution processing relative to an object image using a reference image to generate a high-resolution image. The reference image has first resolution. The object image has second resolution. The second resolution is lower than the first resolution. In other words, in a high-resolution processing, a high-resolution reference image is used, and resolution of the object image is increased.SELECTED DRAWING: Figure 1

Description

  The present technology relates to an image processing device. More specifically, the present invention relates to an image processing apparatus and an image processing method for increasing the resolution of a target image.

  2. Description of the Related Art In recent mobile devices, an authentication process is performed that permits only preregistered individuals to use the device. As such an authentication process, an iris authentication process that performs authentication based on the iris image of the eye is known. In this iris authentication, the size of the pupil changes depending on the brightness of the surroundings, so that there is a problem that the authentication accuracy decreases. Therefore, for example, an apparatus has been proposed in which the eyes of an authentication target person are continuously imaged, a high-resolution image is generated using a plurality of captured images, and iris authentication is performed (for example, see Patent Document 1). ..

JP, 2009-228925, A

  In the above-described conventional technique, a captured image in which the size of the pupil does not match the specified size among the plurality of captured images is set as the correction target image, and is relatively compressed or expanded in the center direction of the pupil to perform the correction. There is. However, in this conventional technique, since the alignment is performed with the captured image having a low resolution as a reference, there is a problem that the alignment accuracy is lowered.

  The present technology is created in view of such a situation, and an object thereof is to improve the accuracy of alignment of a target image and increase the resolution of the target image.

  The present technology has been made in order to solve the above-described problems, and a first aspect thereof is that a reference image having a first resolution is used and a second resolution lower than the first resolution is used. The image processing apparatus and the image processing method include a high resolution processing unit that performs a high resolution processing on a target image having the following to generate a high resolution image. This brings about the effect of using the reference image having the first resolution to increase the resolution of the target image having the second resolution lower than the first resolution.

  Further, in the first aspect, the resolution enhancement processing unit searches a reference image for an image region that is close to a predetermined block of the target image to generate a predetermined motion vector, and the predetermined unit. A distance generation unit that generates a distance between a specific pixel in the predetermined block and a pixel in the reference image that has moved according to the motion vector, and the specific pixel is set to the reference image when the distance satisfies a predetermined condition. An image generation unit that generates the high-resolution image by pasting may be provided. This brings about an effect that the specific pixel in the predetermined block is attached to the high resolution image according to the distance between the specific pixel in the predetermined block moved according to the motion vector and the pixel in the reference image.

  Further, in the first aspect, the image generation unit may perform the above-mentioned pasting as satisfying the predetermined condition when the distance falls within a predetermined range. Further, the image generation unit, when the resolution enhancement processing is performed a plurality of times for different target images, the distance falls within a predetermined range, and the distance is higher than that of the previous resolution enhancement processing. When the distance is small, the bonding may be performed assuming that the predetermined condition is satisfied.

  Further, in the first aspect, the predetermined block may be a block of a local motion vector in the target image. This brings about the effect of flexibly responding to the size of the pupil and the like.

  Further, in the first aspect, the image generation unit may perform the interpolation process on the pixels in the resolution-enhanced image that have not been combined. This brings about an effect that pixels of the high resolution image are generated even for the pixels that have not been combined.

  Further, in the first aspect, an authentication processing unit may be further provided, which performs an authentication process by comparing the reference image and the resolution-enhanced image. This authentication processing unit may perform iris authentication as authentication processing.

  Further, the first aspect may further include an image capturing unit that captures an image of a subject and generates a captured image. At this time, the target image is at least a part of the captured image. Further, in this case, the imaging unit may further image the subject according to the result of the authentication processing.

It is a figure showing an example of composition of an image processing device in an embodiment of this art. It is a figure showing an example of composition of high-resolution processing part 140 in an embodiment of this art. It is a figure showing an example of a mode at the time of registration of a specific person in an embodiment of this art. It is a figure showing an example of a mode at the time of attestation of a person in an embodiment of this art. It is a figure showing an example of a flow to an attestation processing in an embodiment of this art. It is a figure showing an example of a block of a motion vector in an embodiment of this art. It is a figure which shows the example of a mode of high resolution of the image by embodiment of this technique. It is a figure showing an example of relation between a distance and a regulation range in an embodiment of this art. It is a figure showing an example of a processing procedure of an image processing method in an embodiment of this art. It is a figure showing an example of a processing procedure of resolution improvement processing in an embodiment of this art. It is a figure showing an example of a processing procedure of pasting processing in an embodiment of this art.

Hereinafter, modes for carrying out the present technology (hereinafter, referred to as embodiments) will be described. The description will be given in the following order.
1. Configuration of image processing apparatus 2. Details of high resolution processing

<1. Configuration of image processing device>
[Image processing device]
FIG. 1 is a diagram illustrating a configuration example of an image processing device according to an embodiment of the present technology.

  The image processing apparatus increases the resolution of the target image using the reference image and authenticates whether the target image matches the reference image. For example, assuming a front camera on the display side of a mobile device, since the lens used has a wide angle, the number of pixels in which the iris is reflected on the sensor may be insufficient in comparison with the number of pixels generally required for authentication. There is. Therefore, in the image processing apparatus according to this embodiment, the resolution is increased by using a plurality of images by using the registered image stored in advance, so that the authentication performance is improved while coping with the change of the pupil. The image processing apparatus includes an image capturing section 110, a control section 120, a registered image storage section 130, a high resolution processing section 140, and an authentication processing section 150.

  The image capturing section 110 captures a subject and generates a captured image. Here, a person's face is assumed as the subject. At least a part of the captured image is a target image for high resolution. For example, as will be described later, the eye portion is cut out from the face of the person included in the captured image to become the target image.

  The registered image storage unit 130 stores the image of the eyes of a specific person who performs authentication as a registered image in advance. The resolution (first resolution) of the registered image is higher than the resolution (second resolution) of the image captured by the image capturing unit 110. This registered image is used as a reference image in the high resolution processing and the authentication processing.

  The high resolution processing unit 140 uses the registered image stored in the registered image storage unit 130 as a reference image to increase the resolution of the target image. In this resolution increasing process, as will be described later, the alignment is performed with the pixel of the reference image as a reference. As described above, the resolution of the reference image is higher than the resolution of the target image, and thus the generated image is higher than the resolution of the original target image.

  The authentication processing unit 150 compares the reference image with the high-resolution target image and performs the authentication process. That is, the authentication processing unit 150 authenticates whether the person imaged by the image capturing unit 110 is a person registered in advance in the registered image storage unit 130. That is, the authentication processing unit 150 performs biometric authentication, and as one method thereof, for example, performs iris authentication of the human eye.

  The control unit 120 controls the operation of each unit in the image processing apparatus. Specifically, the control unit 120 causes the image capturing unit 110 to capture a captured image required for the high resolution processing, and instructs the high resolution processing unit 140 to perform the high resolution processing. Then, according to the result of the authentication processing in the subsequent authentication processing unit 150, the image capturing unit 110 is caused to further image, and the high resolution processing unit 140 is instructed to perform the high resolution processing.

[High-resolution processing unit]
FIG. 2 is a diagram illustrating a configuration example of the high resolution processing unit 140 according to the embodiment of the present technology.

  The resolution enhancement processing unit 140 includes a search unit 141, a distance generation unit 142, and an image generation unit 143.

  The search unit 141 searches an image area that is close to the block of the target image in the reference image and generates a local motion vector (LMV). As will be described later, the block for generating the local motion vector is a local block (hereinafter, may be referred to as “LMV block”) in a region surrounding the iris in the target image.

  The distance generation unit 142 generates a distance between a specific pixel in the LMV block moved according to the local motion vector generated by the search unit 141 and a pixel in the reference image. That is, this distance is an index that indicates how close to the reference image the standard is when the distance is moved according to the local motion vector. Ideally, this distance should be zero.

  The image generation unit 143 is configured to, when the distance generated by the distance generation unit 142 satisfies a predetermined condition, combine specific pixels in a predetermined block of the target image with the high resolution image. Although the predetermined condition will be described later, it is necessary that a specific pixel in the LMV block falls within a specified range when moving according to the local motion vector. Further, when the processing is repeatedly performed on different target images, the one with a smaller distance is prioritized over the past one.

<2. Details of high resolution processing>
[Registration and Authentication]
FIG. 3 is a diagram illustrating an example of a state when a specific person is registered in the embodiment of the present technology.

  Here, it is assumed that a specific person is registered using the mobile device 100. As described above, the resolution of the registered image stored in the registered image storage unit 130 is required to be higher than the resolution of the captured image captured thereafter. Therefore, when registering the registration image, it is desirable that the portable device 100 be imaged close to the face, as indicated by a in the figure. As a result, the number of pixels occupied by the eyes in the image increases, as shown in b in the figure, and the resolution of the registered image can be substantially improved.

  FIG. 4 is a diagram illustrating an example of how a person is authenticated in the embodiment of the present technology.

  Here, it is assumed that the person who uses the mobile device 100 is authenticated. In smartphones and the like in recent years, it is assumed that the authentication process starts when the person who uses it holds it. That is, as shown by a in the figure, the authentication is performed without bringing the mobile device 100 close to the face. From the user's point of view, that is considered to be more convenient. In this case, as shown by b in the figure, the number of pixels occupied by the eyes in the image decreases, and the resolution decreases. Therefore, in the present embodiment, the authentication processing is performed after increasing the resolution of the captured image captured during authentication.

[Flow to authentication]
FIG. 5 is a diagram showing an example of a flow up to the authentication processing according to the embodiment of the present technology.

  When authenticating a person who uses the mobile device 100, the imaging unit 110 captures a captured image 620 including the face of the person to be authenticated. Then, as shown by a in the figure, an image 630 of the eye portion is cut out from the captured image 620.

  Then, for the image 630 of the eye portion, the position of the entire eye is aligned with the registered image 610 stored in the registered image storage unit 130. In this case, for example, a single motion vector (GMV: Global Motion Vector) is obtained between the image 630 of the eye part and the registered image 610, and the overall alignment of the eye is performed. That is, as shown in b in the figure, the image 630 is moved as a whole so that the size of the eyes and the blank portion are matched with the registered image 610.

  Using the image 631 obtained by these processes as a target image, the iris pattern is aligned with the position of the registered image 610 and the pixels are pasted together. In this case, for example, a local motion vector 636 between the LMV block 615 of the registered image 610 and the LMV block 635 of the image 631 is generated as shown in c in the figure. Then, when the LMV block 635 is moved according to the local motion vector 636, the distance between the specific pixel of the LMV block 635 and the pixel in the registered image 610 is generated. When this distance satisfies a predetermined condition, a specific pixel in the LMV block 635 is attached to the high resolution image to generate an image.

  In order to generate the local motion vector, for example, a general block matching method can be used. As an example, an LMV block is defined by a region of 4 × 4 pixels, and further 3 pixels around it are defined as a search range. The block size of the LMV block can be changed according to the ratio of the iris shown in the image.

  Then, the pixel values are compared between the LMV blocks 615 and 635. As a method of comparison, for example, it is assumed to use the sum of absolute differences. That is, the absolute difference value between each pixel of the LMV block 615 and each pixel of the LMV block 635 is calculated (16 pixels for a 4 × 4 pixel area). Then, the block is shifted by one pixel and the absolute difference value is calculated again. This is repeated within the search range, and finally it is judged that it is plausible to move to the position where the sum of absolute differences is the smallest. The motion (amount and direction) in this case is a local motion vector 636.

  As described above, when the resolution of the image 630 cut out from the captured image 620 is increased, the alignment of the eyes is performed with the registered image 610 as a reference to improve the alignment accuracy and the resolution improvement. Can be made And thereby, the authentication performance of the subsequent authentication process can be improved. That is, both the false rejection rate (FRR), which is the probability of falsely rejecting the person, and the false acceptance rate (FAR), which is the probability of falsely accepting another person, are both reduced. be able to.

[Motion vector block]
FIG. 6 is a diagram showing an example of a motion vector block according to the embodiment of the present technology.

  As described above, it is assumed that two types of motion vectors are obtained during eye alignment. First, in order to perform overall eye alignment, one motion vector (GMV) is obtained as a whole. Therefore, as shown in the figure, a block 640 that covers the entire eyeball region is used.

  Further, in order to align the iris pattern, a local motion vector (LMV) is obtained. Therefore, the LMV block 650 is used as shown in FIG. As a result, the fine iris pattern can be aligned with high accuracy. Further, assuming that a plurality of captured images are taken when performing authentication, the size of the pupil may change due to surrounding brightness during that time, in which case a false image called an artifact around the pupil. May occur. Artifacts greatly affect the subsequent authentication performance. In this respect, the use of LMV makes it possible to cope with a change in the size of the pupil and suppress the occurrence of artifacts.

[Success of high resolution]
FIG. 7: is a figure which shows the example of a mode of high resolution of the image by embodiment of this technique.

  When the registered image 610 and the captured image 620 have the same person, the motion vector is correctly obtained. Therefore, an iris pattern of the person is formed in the image obtained as the high-resolution image. For example, as shown by a in the figure, an iris pattern 661 of the person is formed.

  On the other hand, if the registered image 610 and the captured image 620 are different people, the motion vector cannot be obtained correctly. Therefore, the image obtained as the high-resolution image does not have a normal iris pattern. For example, as shown in b in the figure, the iris pattern 662 appears as a collapsed shape.

[distance]
FIG. 8 is a diagram showing an example of the relationship between the distance and the specified range in the embodiment of the present technology.

  As described above, in this embodiment, when the local motion vector (LMV) is obtained and the LMV block of the target image is moved in accordance with this local motion vector, specific pixels of the LMV block and the registered image 610 are registered. The distance generation unit 142 generates the distance to the pixel. Here, the specific pixel is, for example, a pixel located in the center of the LMV block.

  Then, when the generated distance satisfies a predetermined condition, a specific pixel in the LMV block is attached to the high resolution image. The predetermined conditions at this time will be described below.

  In the figure, a pixel 510 is a pixel of the registered image 610. Further, the specified range 520 is a range within a circle formed by a constant radius from the center of the pixel 510. Further, the pixels before movement when the LMV block of the target image is moved according to the local motion vector are represented by white circles, and the pixels after movement are represented by gray circles. That is, the arrow connecting the pixel before the movement and the pixel after the movement represents the local motion vector. Further, when processing is repeatedly performed on different target images, pixels that have been pasted together are represented by black circles. In this example, it is assumed that the prescribed range 520 is circular, but the present invention is not limited to this, and may be square, for example.

  When the pixel of the target image is moved according to the local motion vector and the specific pixel in the LMV block falls within the specified range, the specific pixel is pasted on the high resolution image. However, when the processing is repeatedly performed on different target images, the one with a smaller distance is prioritized over the past one.

  For example, as shown in Case 1, when the moved pixel is within the specified range 520 and the distance is smaller than the past one, the pixel of the target image is adopted and the bonding is performed. Be seen.

  On the other hand, as shown in Case 2, even if the pixel after the movement is within the specified range 520, if the distance is larger than the past, the pixel of the target image is not adopted and the bonding is performed. I don't know.

  Further, as shown in Case 3, if the pixel after the movement is not within the specified range 520, the pixel of the target image is not adopted even if the pasting has not been performed, and the pasting is performed. Absent.

[Processing procedure]
FIG. 9 is a diagram illustrating an example of a processing procedure of the image processing method according to the embodiment of the present technology. Here, a description will be given of a processing procedure in the case where the user picks up the face of the user and performs authentication when the user holds the portable device or the like including the image processing apparatus.

  First, the image capturing unit 110 captures an image of the user's face (step S901). Then, high resolution processing is performed on the captured image (step S902), and authentication processing is performed using the high resolution image to determine whether or not the person is a person registered in advance (step S903). If the authentication is successful (step S904: Yes), the series of processes is terminated and the user is permitted to use the portable device or the like.

  If the authentication has failed (step S904: No), the processes after the imaging process (step S901) are repeated. The number of times of this repetition may be set in advance. In that case, if the authentication is not successful even after repeating the set number of times, the user cannot use the mobile device or the like.

  FIG. 10 is a diagram illustrating an example of a processing procedure of the high resolution processing (step S902) according to the embodiment of the present technology.

  First, the registered image previously stored in the registered image storage unit 130 is acquired (step S921). Then, "1" is set to the variable L that counts the number of LMV blocks for local motion vector generation (step S922).

  A local motion vector is generated between the registered image and the block L of the captured image (step S923). Then, using the generated local motion vector, the pixel L of the block L is combined (step S924). Then, "1" is added to the variable L (step S925). If it is determined from the value of the variable L that the search for all local motion vectors has not been completed (step S926: No), the process is repeated for the next block L (step S923).

  When it is determined that all the local motion vectors have been searched (step S926: Yes), it is determined whether there is a pixel that has not been pasted (step S927). If there is a pixel that has not been pasted (step S927: Yes), interpolation processing is performed for that pixel (step S928). As the interpolation processing in this case, for example, interpolation using an average value of surrounding pixel values can be considered.

  FIG. 11 is a diagram illustrating an example of a processing procedure of the bonding processing (step S924) in the embodiment of the present technology.

  First, the distance between the specific pixel of the block L and the pixel of the registered image is measured (step S931). If this distance does not fall within the specified range 520 (step S932: No), the bonding is not performed (step S935). This is the case corresponding to case 3 described above.

  Even if the obtained distance falls within the specified range 520 (step S932: Yes), if the past distance is smaller (step S933: No), the bonding is not performed (step S935). .. This is the case corresponding to case 2 above.

  On the other hand, when the obtained distance falls within the specified range 520 (step S932: Yes) and is smaller than the past distance (step S933: Yes), the specific pixel of the block L is pasted (step S934). ). This is the case corresponding to case 1 above. If the pixel of the pasting destination has not been repeatedly processed, that is, if the pixel of the pasting destination pixel has not been pasted, the pasting is performed without considering the past distance.

  As described above, according to the embodiment of the present technology, it is possible to increase the resolution of the target image by performing the alignment based on the high-resolution registered image and pasting the pixels of the target image together. In particular, by using the local motion vector when comparing the registered image and the target image, it becomes possible to respond to changes in the size of the pupil, suppress the occurrence of artifacts, and accurately target the target image with high resolution. Can be converted. And thereby, the accuracy of an authentication process can be improved. Therefore, it is possible to perform biometric authentication with high accuracy without requiring the user to bring his or her face close to a mobile device or the like.

  It should be noted that the above-described embodiment shows an example for embodying the present technology, and the matters in the embodiment and the matters specifying the invention in the claims have a correspondence relationship. Similarly, the matters specifying the invention in the claims and the matters having the same names in the embodiments of the present technology have a correspondence relationship. However, the present technology is not limited to the embodiments and can be embodied by making various modifications to the embodiments without departing from the scope of the invention.

  Further, the processing procedure described in the above-described embodiment may be regarded as a method having these series of procedures, or as a program for causing a computer to execute the series of procedures or a recording medium storing the program. You can catch it. As the recording medium, for example, a CD (Compact Disc), an MD (MiniDisc), a DVD (Digital Versatile Disc), a memory card, a Blu-ray disc (Blu-ray (registered trademark) Disc), or the like can be used.

  It should be noted that the effects described in the present specification are merely examples and are not limited, and may have other effects.

In addition, the present technology may have the following configurations.
(1) A high-resolution image is generated by performing a high-resolution process on a target image having a second resolution lower than the first resolution using a reference image having a first resolution. An image processing apparatus including a resolution conversion processing unit.
(2) The high resolution processing unit is
A search unit that searches the reference image for an image region that is close to a predetermined block of the target image and generates a predetermined motion vector;
A distance generation unit that generates a distance between a specific pixel in the predetermined block moved according to the predetermined motion vector and a pixel in the reference image,
The image processing device according to (1), further comprising: an image generation unit that, when the distance satisfies a predetermined condition, bonds the specific pixel to the reference image to generate the high resolution image.
(3) The image processing device according to (2), wherein the image generation unit performs the bonding when the distance satisfies a predetermined condition when the distance is within a predetermined range.
(4) The image generation unit, when the resolution enhancement processing is performed a plurality of times for different target images, the distance is within a predetermined range, and is more than the previous resolution enhancement processing. The image processing apparatus according to (2), wherein when the distance is small, the bonding is performed assuming that the predetermined condition is satisfied.
(5) The image processing device according to any one of (2) to (4), wherein the predetermined block is a block of a local motion vector in the target image.
(6) The image processing device according to any one of (2) to (5), in which the image generation unit performs interpolation processing on pixels that have not been combined in the high-resolution image.
(7) The image processing device according to any one of (1) to (6), further including an authentication processing unit that performs authentication processing by comparing the reference image and the resolution-enhanced image.
(8) The image pickup unit further includes an image pickup unit that picks up an image of a subject and generates a picked-up image,
The image processing apparatus according to (7), wherein the target image is at least a part of the captured image.
(9) The image processing device according to (8), wherein the imaging unit further images the subject according to a result of the authentication processing.
(10) The image processing apparatus according to any one of (7) to (9), wherein the authentication processing unit performs iris authentication as the authentication processing.
(11) An image in which a reference image having a first resolution is used to perform a resolution enhancement process on a target image having a second resolution lower than the first resolution to generate a resolution-enhanced image Processing method.
(12)
The high resolution processing is
A search procedure for searching the reference image for an image region that is close to a predetermined block of the target image;
A distance generation procedure for generating a distance between a specific pixel in the predetermined block of the target image and a pixel in the reference image,
The image processing method according to (11), further comprising: an image generation procedure that bonds the specific pixel to the reference image to generate the high-resolution image when the distance satisfies a predetermined condition.

100 mobile device 110 image pickup unit 120 control unit 130 registered image storage unit 140 high resolution processing unit 141 search unit 142 distance generation unit 143 image generation unit 150 authentication processing unit

Claims (12)

  Using the reference image having the first resolution, the resolution increasing process for performing the resolution increasing process on the target image having the second resolution lower than the first resolution to generate the high resolution image. Image processing apparatus including a unit. The high resolution processing unit,
A search unit that searches the reference image for an image region that is close to a predetermined block of the target image and generates a predetermined motion vector;
A distance generation unit that generates a distance between a specific pixel in the predetermined block moved according to the predetermined motion vector and a pixel in the reference image,
The image processing apparatus according to claim 1, further comprising: an image generation unit that, when the distance satisfies a predetermined condition, bonds the specific pixel to the reference image to generate the high resolution image. The image processing apparatus according to claim 2, wherein the image generation unit performs the bonding when the distance is within a predetermined range, assuming that the predetermined condition is satisfied. The image generation unit, when the high resolution processing is executed a plurality of times for different target images, the distance falls within a predetermined range, and the distance is smaller than that of the previous high resolution processing. The image processing apparatus according to claim 2, wherein when the size is small, the pasting is performed assuming that the predetermined condition is satisfied. The image processing apparatus according to claim 2, wherein the predetermined block is a block of a local motion vector in the target image. The image processing apparatus according to claim 2, wherein the image generation unit performs an interpolation process on the pixels that have not been combined in the high-resolution image.   The image processing apparatus according to claim 1, further comprising an authentication processing unit that performs authentication processing by comparing the reference image and the high resolution image. An image pickup unit that picks up an image of a subject and generates a picked-up image,
The image processing apparatus according to claim 7, wherein the target image is at least a part of the captured image. The image processing apparatus according to claim 8, wherein the imaging unit further images the subject according to a result of the authentication processing. The image processing apparatus according to claim 7, wherein the authentication processing unit performs iris authentication as the authentication processing.   An image processing method for generating a high-resolution image by executing a high-resolution process on a target image having a second resolution lower than the first resolution using a reference image having a first resolution. The high resolution processing is
A search procedure for searching the reference image for an image region that is close to a predetermined block of the target image;
A distance generation procedure for generating a distance between a specific pixel in the predetermined block of the target image and a pixel in the reference image,
The image processing method according to claim 11, further comprising: an image generation procedure that bonds the specific pixel to the reference image to generate the high-resolution image when the distance satisfies a predetermined condition.

Images