CN1698279A - Pointing device having fingerprint image recognition function, fingerprint image recognition and pointing method, and method for providing portable terminal service using thereof - Google Patents

Abstract

A pointing device having a fingerprint recognition function, a fingerprint image recognition and pointing method, and a method for providing a portable terminal service using the same are disclosed. In the pointing device having a fingerprint recognition function and a fingerprint recognition method, fingerprint images having small sizes are mapped to generate a lare fingerprint image or a small fingerprint image extracted from the large fingerprint image so that the pointing device performs user recognition and pointer control. As a result, respective sensors for the user recognition and for the pointer control are not comprised in the pointing device, but only one kind of sensors for performing both functions of user recognition and pointer control is comprised in the pointing device according to an embodiment of the present invention. Also, it is possible to easily embody miniaturization of a portable terminal device because the user recognition wich requires a large fingerprint can be performed only with a small fingerprint image, thereby reducing manufacturing cost. Additionally, important information in the portable terminal device having the pointing device can be protected by selectively limiting kinds of service usable in the portable terminal device depending on user recognition.

Description

Pointing device with fingerprint image recognition function, fingerprint image recognition and pointing method, and method of using them for providing portable terminal service

technical field

The present invention generally relates to a pointing device having a fingerprint image recognition function and a fingerprint recognition method thereof, and more particularly, to a method for performing user recognition and point control by using a sensor without using a user recognition sensor and a point control sensor A pointing device and its fingerprint image recognition method.

Background technique

In general, a pointer as a pointing device refers to an XY tablet, a trackball, and a mouse that have been widely used in desktop computers, or refers to a touch screen panel or a touch pad that has been widely used in portable terminal devices such as laptop computers. . Recently, an optical mouse using light has been used.

Efforts to integrate biometrics into electronic and communication devices and their peripherals have recently increased. The most remarkable feature of biometrics is to alleviate troubles of loss, theft, forgetting or reappearance due to external factors in any case, and this feature is the most advantageous. When this feature is used, the audit function is fully implemented to track who violated the security. In particular, user identification technology using fingerprints has been actively commercialized, and it is easy to access and support such user identification technology because it identifies users by using human characteristics. As a result, various studies have been conducted and great developments have been made in this field.

Recently, technologies have been developed to introduce user identification using fingerprints to pointing devices. In the pointing device, an internal fingerprint recognition device recognizes a fingerprint from a finger surface through a predetermined window, and compares a previously registered fingerprint with the recognized fingerprint, and verifies the fingerprint when the comparison result is the same independently of the pointing function. FIG. 1 shows a fingerprint recognition optical mouse as an example. As shown in FIG. 1, the fingerprint recognition optical mouse 1 has the same shape and the same function as a general mouse, but includes a fingerprint recognition window 2 in the part touched by the right thumb. If the right thumb touches the fingerprint recognition window 2, an internal fingerprint recognition sensor (not shown) recognizes the thumb's fingerprint and compares the recognized fingerprint with previously registered fingerprints to determine user recognition.

In traditional fingerprint identification, the smallest fingerprint required for user identification is obtained. That is, in the case of the fingerprint recognition optical mouse of FIG. 1, a fingerprint of about 100*100 pixels is acquired for fingerprint recognition. Now, an optical mouse for detecting a fingerprint image of 96*96 pixels at a time is commercially available.

Additionally, fingerprint recognition can be introduced into pointing devices that use fingerprints. In other words, a technique of controlling the use of a fingerprint and simultaneously recognizing a user's pointer with a fingerprint has been provided in current pointing devices. However, in the prior art, a fingerprint acquisition sensor for fingerprint identification to acquire a larger fingerprint image for user identification and a fingerprint acquisition sensor for controlling a pointer to acquire a smaller fingerprint image are included in one pointing device .

FIG. 2 shows a portable terminal device including two fingerprint acquisition sensors. Fig. 2(a) shows a part of a portable computer (laptop) and Fig. 2(b) shows a part of a PDA. In the case of FIG. 2( a), a fingerprint acquisition sensor 3 for fingerprint recognition that recognizes a user's fingerprint to verify the user's fingerprint and a pointer for controlling a pointer displayed in a monitor of a laptop computer with a finger are included. Pointer control sensor 4. However, pointer controllers in laptop computers do not use fingerprint recognition but a change in capacitance caused by pressure from a finger or a stylus. In addition, in the case of FIG. 2( b ), the fingerprint acquisition sensor 5 and the pointer controller sensor 6 for user identification are respectively included. Generally, in the case of a laptop computer for controlling the movement of a cursor on a monitor based on movement information of a finger, a fingerprint image of about 20×20 pixels is sufficient to obtain movement information of a fingerprint for point control, but , for user identification, a fingerprint image of more than about 100 x 100 pixels is required. In the case of an actual laptop computer, products that acquire image data of about 100 x 100 pixels with a fingerprint capture sensor of about 5 x 5 mm are commercially available.

Although low power and high speed are realized by using small data of about 20×20 pixels for point control, a large fingerprint image of 100×100 pixels for fingerprint identification needs to be acquired and analyzed. As a result, fingerprint acquisition sensors 3 and 5 and pointer control sensors 4 and 6 for user identification are required. Because these two kinds of fingerprint acquisition sensors are included in the pointing device, the appearance of this pointing device is ugly and cannot solve the technical complexity of driving these two kinds of fingerprint acquisition sensors. Therefore, two types of fingerprint acquisition sensors are installed in the prior art, which has an adverse effect on miniaturization of electronic devices and devices to make thinner and simpler.

Therefore, there is a need for a method for performing user identification by using a fingerprint acquisition sensor for user identification in a portable terminal device and simultaneously performing point control by acquiring a plurality of small fingerprint images.

Description of drawings

FIG. 1 is a perspective view of a conventional optical mouse for fingerprint recognition.

FIG. 2 is a view showing an example of a portable terminal device including a conventional fingerprint sensor for fingerprint recognition and a conventional navigation pad for pointer control.

FIG. 3 is a view showing the structure of the pointing device according to the first embodiment of the present invention.

FIG. 4 is a view showing a process of calculating displacement data according to the present invention.

FIG. 5 is a view showing a process of mapping a fingerprint image according to the present invention.

FIG. 6 is a view showing the structure of a pointing device according to a second embodiment of the present invention.

FIG. 7 is a view showing a process of mapping fingerprint images obtained from a plurality of fingerprint acquiring devices shown in FIG. 6 .

8 is a flowchart showing fingerprint recognition processing in the pointing device according to the first or second embodiment of the present invention.

FIG. 9 is a detailed flowchart showing a process of mapping a fingerprint image in the virtual image space in FIG. 8 .

FIG. 10 is a flowchart showing a process of controlling a pointer in a pointing device according to the present invention.

FIG. 11 is a view showing the structure of a pointing device according to a third embodiment of the present invention.

FIG. 12 is a view showing a design example of a 3:1 reduction optical system installable in a minute space according to the present invention.

FIG. 13 is a view showing an example of a fingerprint image obtained from a fingerprint obtaining device by applying a reduction optical system according to the present invention.

FIG. 14 is a view showing a pointing device according to a fourth embodiment of the present invention.

FIG. 15 is a view showing a method for extracting a fingerprint image of m*n pixels from a fingerprint image of M*N pixels.

FIG. 16 is a flow chart illustrating a method for performing user identification and simultaneously performing pointing control according to the third or fourth embodiment of the present invention.

FIG. 17 is a view showing the structure of a pointing device according to a fifth embodiment of the present invention.

Fig. 18 is a flowchart showing the operation of the pointing device according to the fifth embodiment of the present invention.

FIG. 19 is a flowchart illustrating a method for restricting usage of a portable communication terminal device depending on a user by using a fingerprint recognition function according to the present invention.

FIG. 20 is a view showing an example of a portable terminal device including a pointing device according to the present invention.

Contents of the invention

technical topics

An object of the present invention is to improve a fingerprint recognition method so that user recognition and pointer control can be performed simultaneously using only one sensor without requiring separate fingerprint recognition sensors for user recognition and for pointer control.

technical solution

In one embodiment, a pointing device with a fingerprint image recognition function includes: at least one or more fingerprint acquisition means for acquiring fingerprint images of the finger surface according to a predetermined period; At least one or more feature points are extracted from the image; the movement detection device is used to calculate the displacement data between the feature points of the extracted fingerprint image to detect the movement information of the fingerprint image; mapping the fingerprint image in space; identification means, when the overall size of the mapped fingerprint image reaches a previously set size, comparing the entire mapped fingerprint image with a previously registered fingerprint image, and determining identification of the fingerprint; and computing means, Used to receive displacement data from a motion detection device and use the displacement data to calculate the direction and distance the pointer will move.

In one embodiment, a pointing device with a fingerprint identification function includes: a fingerprint acquisition device (first operation cycle), used to acquire a fingerprint image on the surface of a finger, and control the pointer through only one two-dimensional image acquisition; fingerprint identification A unit (second operation period) for comparing the feature points of the obtained fingerprint image with those of the previously registered fingerprint image to identify the user of the acquired fingerprint image; and a fixed-point control unit (third operation period), For detecting movement information based on partial data of the image acquired depending on the first operation cycle, and calculating displacement data of the fingerprint image based on the movement information to calculate the moving direction and distance of the pointer.

In one embodiment, a method for identifying a fingerprint for user identification includes the following steps: using a predetermined fingerprint acquisition sensor to acquire at least one or more fingerprint images according to a set period; extracting at least one fingerprint image from the acquired fingerprint image One or more feature points; mapping the first fingerprint image in a specific position in the virtual image space; calculating the feature points of the first fingerprint image and the second fingerprint acquired in the next period after the period in which the first fingerprint image has been acquired The displacement data between the feature points of the image; the second fingerprint image is mapped with the displacement data in the virtual image space; and when the overall size of the fingerprint image mapped in the virtual image space reaches the previously set size, The feature points are compared with the feature points of the previously registered fingerprint image, and the identification of the fingerprint is determined.

In one embodiment, there is provided a pointing method for a pointer control device having an image sensor smaller than a predetermined size required in fingerprint identification, the method comprising the steps of: using a control movable pointer A predetermined fingerprint acquisition sensor that acquires at least one or more fingerprint images of M×N pixels on the finger surface according to the first operation cycle; by extracting feature points from the obtained fingerprint images according to the second operation cycle and combining the extracted features Points are compared with the feature points of the previously registered fingerprint image to determine the identification of the user of the fingerprint image; extract the fingerprint image of m×n pixels from the obtained fingerprint image according to the third operation cycle; extract the m×n pixel fingerprint image by calculating the extracted m× n-pixel displacement data of the fingerprint image to detect movement information of each fingerprint image; and using the displacement data to calculate and output the direction and distance that the pointer will move.

In one embodiment, a pointing device with a fingerprint image recognition function includes: at least one or more fingerprint acquisition devices, used to obtain images of the finger surface according to a predetermined period or occasional requirements; movement detection devices, used to calculate and obtain The displacement data of the fingerprint image to detect the movement information of each fingerprint image; the computing device is used to receive the displacement data from the movement detection device, and uses the displacement data to calculate the direction and distance that the pointer will move; the storage device is used for mapping The fingerprint image obtained from the fingerprint acquisition device and the computing device and the displacement data of the fingerprint image; the CPU is used for analyzing and processing the data of the computing device and the image storage space.

Detailed ways

The present invention will be described in detail with reference to the accompanying drawings.

FIG. 3 is a view showing the structure of the pointing device according to the first embodiment of the present invention.

The pointing device in FIG. 3 includes a light emitting device 22, a light collecting device 23, a fingerprint acquisition device 24, a feature point extraction device 25, a storage device 26, a motion detection device 27, a mapping device 28, a virtual image space 29, an identification device 30 and an operation device 31 .

Referring to FIG. 3 , when the finger 20 to obtain a fingerprint image touches the transparent member 21 , the light emitting device 22 emits light on the surface of the finger 20 as the touching object. The light emitting device 22 includes at least one or more light emitting diodes.

The light collecting device 23 collects a fingerprint image generated by emitting light from the light emitting device 22 to the contact object. A light convex lens can be used as the light collecting means 23 .

The fingerprint acquiring device 24 detects the simulated fingerprint image collected by the light collecting device 23 and converts the simulated fingerprint image into a digital fingerprint image. The fingerprint acquisition device 24 includes an optical sensor array in which a plurality of CMOS image sensors (abbreviated as "CIS") are arranged two-dimensionally. Here, the fingerprint acquiring means 24 acquires a plurality of fingerprint images in a previously set cycle. The fingerprint acquiring device 24 is manufactured to be suitable for a small portable terminal device, so as to acquire a small fingerprint image. For example, a microsensor suitable for fixed-point control for capturing less than about 20×20 pixels is used as the fingerprint capturing device 24 . Here, devices known to those skilled in the art can be used as the light emitting device 22 , the transparent member 21 , the light collecting device 23 and the fingerprint acquiring device 24 .

The light emitted from the light emitting device 22 is radiated onto the surface of the finger 20 and reflected according to the pattern on the surface of the finger 20 . The light reflected from the bottom surface of the finger 20 forms a phase in the fingerprint taking device 24 through the light collecting device 23 . The image formed in the fingerprint acquisition device 24 is converted into a digital fingerprint image by the fingerprint acquisition device 24 . Acquisition of fingerprint images is performed in rapid succession on the time axis.

Feature point extracting means 25 extracts at least one or more feature points from each fingerprint image acquired by fingerprint acquiring means 24 in a predetermined cycle. These feature points include the length and direction of the lines in the fingerprint image and the location data where the lines separate or end.

The storage means 26 stores the fingerprint image acquired from the fingerprint acquiring means 24 and the information on the feature points extracted from the feature point extracting means 25 .

The movement detecting means 27 detects the degree of movement of each fingerprint image from the feature points of the fingerprint images stored in the storage means 26 . Here, the movement detecting means 27 detects the degree of movement of the fingerprint by calculating the displacement data (direction and distance) of the feature point from the movement of the fingerprint using a movement estimation method. In this way, the movement detecting means 27 detects the degree of movement of the fingerprint image by comparing the feature points of the fingerprint image acquired in the previously set period. The movement information and feature point extraction of the fingerprint image in fingerprint recognition and the acquisition of the fingerprint image are important factors because the movement of the fingerprint image and the reliability of fingerprint recognition are distinguished according to how reliably the feature points are extracted.

The mapping means 28 receives the displacement data (direction and distance) of the feature points of the fingerprint image, which is movement information from the movement detecting means 27 depending on the movement of the fingerprint image, and the mapping means 28 uses the displacement data to determine that the moving fingerprint image will be at The location of the map in virtual image space 29 . Next, the mapping means 28 maps each fingerprint image according to the determined position. When the fingerprint image is mapped by the mapping means 28, among the feature points acquired in the previous cycle and the current cycle, the same feature points are preferably mapped to overlap. In this way, the mapping device 28 two-dimensionally arranges the fingerprint images acquired each time in the virtual image space. Here, the virtual image space 29 has the size of a fingerprint image required for user identification. That is, the virtual image space 29 as a storage device for synthesizing the fingerprint image required in user identification preferably has the size of the fingerprint image required in user identification. For example, virtual image space 29 has dimensions less than about 100x100 pixels.

The recognition device 30 detects whether the size of the overall fingerprint image mapped in the virtual image space 29 is equal to the size of the virtual image space 29, and then if the size is the same, compares the entire mapped fingerprint image with the previously registered fingerprint image to authenticate the user.

The computing device 31 receives the displacement data from the movement detecting device 27, and uses the displacement data to calculate the direction, distance and degree of movement that the pointer will move. The computing device 30 is generally combined with a pointing device or a processor of a device having a pointing device. As a result, the processor is able to control the movement of the pointer on the screen of the display device in a desired direction and a desired distance.

In one embodiment, the fingerprint acquiring device 24 can be embodied in various ways. That is, the fingerprint acquiring device 24 can be embodied with a semiconductor device or with an optical system as described above. Here, a fingerprint acquisition device 24 using such an optical system has been commercialized as an authentication system for a long time and has advantages in terms of scratches, temperature and durability. However, such an optical system has a limitation of being used only in small portable terminal devices due to the size of the optical sensor and has a problem that information security and identification cannot be adopted. At the same time, the fingerprint acquisition device 24 using semiconductor devices has a clear image image and fast response speed when acquiring fingerprint images. In addition, since the miniaturization of the sensor is possible, the fingerprint acquisition device 24 using semiconductor devices has a wide range of applications and is highly competitive in cost.

In one embodiment, the acquisition of the fingerprint image can be performed with such an optical system or such a semiconductor system. For example, when the fingerprint image is acquired with a semiconductor system, the above-mentioned light emitting device 22 and light collecting device 23 are not required. As a result, FIG. 3 shows an example of a pointing device for acquiring a fingerprint image with an optical system, and a fingerprint image can be acquired with such a semiconductor system. Since the present invention is not characterized by the acquisition of the fingerprint image but by the processing method of the acquired fingerprint image, the fingerprint acquisition method can be performed with any system.

Hereinafter, the operation of the pointing device with the fingerprint recognition function will be described in detail. In one embodiment of the present invention, since the fingerprint recognition function and the point-fixing function are simultaneously executed in the pointing device, the fingerprint recognition function and the point-fixing function are described.

FIG. 4 is a view showing a process of calculating displacement data according to the present invention. Figure 4a shows the fingerprint image and its feature points acquired in the first cycle, and Figure 4b shows the fingerprint image and its feature points acquired in the second cycle. The fingerprint images of FIGS. 4 a and 4 b are images formed in the fingerprint acquisition device 24 . In one embodiment, as an example, a fingerprint image from which 5 feature points (denoted as M) are extracted is shown.

The fingerprint image of FIG. 4b is obtained by shifting the fingerprint image of FIG. 4a to the right by 3 pixels (ΔX=3) and downward by 3 pixels (ΔY=-3) within a predetermined period. The movement detecting means 27 acquires the movement of the fingerprint image by calculating the displacement data (direction and distance) of the extracted feature points.

The mapping device 28 uses the feature point displacement data calculated by the moving detection device 27 to map the fingerprint image acquired each time to the corresponding position in the virtual image space 29 . The mapping process will be described in detail with reference to FIG. 5 .

FIG. 5 is a view showing a process of mapping a fingerprint image according to the present invention.

In one embodiment, the use of a miniaturized fingerprint capture device 24 for capturing fingerprint images smaller than 20 x 20 pixels is used to capture fingerprints of the size required in user identification (i.e., less than about 100 x 100 pixels). Image processing is described.

Fig. 5 a shows the fingerprint image acquired according to the previously set period using a miniaturized fingerprint acquisition device 24 of less than 20 × 20 pixels, and Fig. 5 b shows a virtual image space 29 of less than 100 × 100 pixels where the fingerprint image is mapped . In FIG. 5 , for convenience of description, it is assumed that the predetermined pattern is not the shape of the fingerprint but the figure of the pattern.

When the fingerprint acquiring device 24 acquires the first fingerprint image 41 of 20*20 pixels at time _T0 , the feature point extracting device 25 extracts at least one or more feature points from the acquired first fingerprint image 41 and stores the feature points in storage device 26 . In Fig. 5a, 6 feature points (indicated as black dots) are extracted on the first fingerprint image 41 . The mapping device 28 maps the first fingerprint image 41 on a predetermined position in the virtual image space 29 . Here, the mapping device 28 preferably maps the acquired fingerprint image on the center of the virtual image space 29 . Thereafter, when the fingerprint acquisition device 24 obtains the second fingerprint image 42 at time _T1 , the feature point extraction device 25 extracts at least one or more feature points (number: 8) from the second fingerprint image 42 and stores the feature points in storage device 26 .

The movement detection means 27 uses the movement information of the first fingerprint image 41 and the second fingerprint image 42 to calculate displacement data (direction and distance). Displacement data were calculated by the method described in Figure 4. The mapping means 28 maps the second fingerprint image 42 in a position in the virtual image space 29 corresponding to the calculated displacement data in the movement detection means 27 . Then, when the fingerprint acquiring means 24 acquires the third fingerprint image 43 at time _T2 , the feature point extracting means 25 extracts at least one or more feature points (number: 9) from the third fingerprint image 43 . The extracted feature points are stored in the storage device 26, and displacement data about the second fingerprint image 42 and the third fingerprint image 43 are calculated using the extracted feature points. The mapping means 28 maps the third fingerprint image 43 in a position in the virtual image space 29 corresponding to the calculated displacement data. As described above, fingerprint image acquisition, feature point extraction, displacement data calculation, and mapping operations are repeatedly performed according to a predetermined cycle until the overall size of the mapped fingerprint images 41, 42, 43, ..., n is the size specified in the user identification. The required size, that is to say, the size of the virtual image space 29. In this way, a fingerprint image having a large size required in user identification can be obtained by a plurality of fingerprint images each having a small size.

During the mapping operation, when the fingerprint image acquired in the current cycle is mapped in the virtual image space 29, it is preferable to map the feature points so that at least a part of the feature points of the fingerprint image captured in the previous cycle can be compared with the fingerprint image captured in the current cycle. The feature points of the fingerprint image overlap. For example, when the second fingerprint image 42 is mapped in the virtual image space 29 in FIG. 5 b , at least a part of the feature points of the second fingerprint image 42 are mapped to overlap with at least a part of the feature points of the first fingerprint image 41 . Likewise, when the third fingerprint image 43 is mapped in the virtual image space 29 , at least a part of the feature points of the third fingerprint image 43 is mapped to overlap with at least a part of the feature points of the second fingerprint image 42 . In FIG. 5 b , reference numeral 48 denotes the portion where the first fingerprint image 41 overlaps the second fingerprint image 42 , and reference numeral 49 denotes the portion where the second fingerprint image 42 overlaps the third fingerprint image 43 .

Meanwhile, the second fingerprint image 42 is obtained by moving the finger 20 for a predetermined time (T1-T0) after acquiring the first fingerprint image 41 . In FIG. 3 , when the finger 20 moves any distance in any direction when touching the transparent member 21 , the fingerprint image is acquired according to the set period. In this case, the direction of movement of the finger 20 is opposite to that of the fingerprint image.

As described above, the fingerprint images acquired each time in the set period are mapped in the virtual image space 29 . When the overall size of the mapped fingerprint image is equal to the size of the virtual image space 29, the identification device 30 compares the entire fingerprint image mapped in the virtual image space 29 with the previously registered fingerprint image, and then determines identification. In this case, the identification is preferably determined by matching feature points of the fingerprint image. The identification means 30 authenticates the user if the two fingerprint images are identical, but refuses to authenticate the user if not. As a result, the user can restrict the use of the terminal device so that only the user can use or suppress information that the user intends to be leaked in the device.

FIG. 6 is a view showing the structure of a pointing device according to a second embodiment of the present invention.

The pointing device in FIG. 6 includes more light emitting devices and fingerprint acquisition devices than in FIG. 3 . In FIG. 6, a plurality of fingerprint acquiring devices 24-1, 2, 3 acquire a plurality of fingerprint images each time according to a predetermined period. Compared with FIG. 3, when the fingerprint acquiring means 24 acquires a fingerprint image in each cycle in the pointing device of FIG. Multiple fingerprint images are acquired in each cycle in the pointing device of FIG. 6 . FIG. 6 shows a pointing device including 3 light emitting devices 23-1, 2, 3 and 3 fingerprint acquiring devices 24-1, 2, 3.

FIG. 7 is a view showing a process of mapping fingerprint images acquired from a plurality of fingerprint acquiring devices 24-1, 2, 3 shown in FIG. 6 .

Fig. 7 shows the use of 3 miniaturized fingerprint acquisition devices to acquire fingerprint images of about 20×20 pixels and the use of fingerprint images of about 20×20 pixels to acquire fingerprints with the required size (about 100×100 pixels) in user identification. pixel) image processing. Fig. 7a shows fingerprint images of 20×20 pixels acquired by the fingerprint acquiring devices 24-1, 2, 3 according to the previously set cycle. Fig. 7b shows the processing of the fingerprint image shown in Fig. 7a corresponding to the location of the virtual image space 29 of 100 x 100 pixels.

When comparing the mapping process of FIG. 7 with the mapping process of FIG. 5, the mapping process of FIG. Periodically map the three fingerprint images acquired each time in the virtual image space 29 .

Referring to Figures 7a and 7b, the fingerprint acquiring devices 24-1, 2, 3 simultaneously acquire three 20×20 pixel fingerprint images (the first fingerprint image set 61) at time _T0 . Next, the feature point extracting means 25 extracts at least one or more feature points from the first fingerprint image set 61 , and stores the extracted feature points in the storage means 26 . The first fingerprint image set 61 in Fig. 7a has a total of 12 feature points (shown as black dots). The mapping means 28 maps the first set of fingerprint images 61 in a specific location in the virtual image space 29 . Here, the first fingerprint image set 61 is preferably mapped in the center of the virtual image space 29 . Thereafter, the fingerprint acquiring means 24-1, 2, 3 acquire 3 fingerprint images (second fingerprint image set 62) at the next time T1. The feature point extraction means 25 extracts at least one or more feature points (number: 9) from the second fingerprint image set 62, and stores the feature points in the storage means 26. The movement detection device 27 uses the movement information of the first set of fingerprint images 61 and the second set of fingerprint images 62 to calculate displacement data (direction and distance). Displacement data were calculated by the same method described in Figure 4. The mapping means 28 maps the second set of fingerprint images 62 in positions corresponding to the displacement data calculated by the movement detection means 27 . The mapping operation is repeatedly performed until the overall size of the mapped set of fingerprint images 61 , 62 , . . . , n becomes the size required for user identification, that is, the size of the virtual image space 29 . As a result, a fingerprint image having a large size required in user identification can be obtained by a plurality of fingerprint images each having a small size. Here, the second fingerprint image set 62 includes fingerprint images obtained from the three fingerprint acquisition devices 24 - 1 , 2 , and 3 by moving the finger for a predetermined time T ₁ -T0 after acquiring the first fingerprint image set 61 .

Here, as described in FIG. 5, when the fingerprint image set acquired in the current cycle is mapped in the virtual image space 29, it is preferable to map the feature points so that a part of the feature points of the fingerprint image set acquired in the current cycle can be compared with Part of the feature points of the fingerprint images acquired in the previous cycle overlap.

When the set of fingerprint images is mapped in the entire virtual image space 29, the identification device 30 determines identification by comparing the entire fingerprint image mapped in the virtual image space 29 with previously registered fingerprint images. When the two fingerprint images are identical, the identification device 30 authenticates the user, but when they are not identical, the identification device 30 refuses to authenticate the user.

Meanwhile, the pointing device according to an embodiment of the present invention controls a pointer through movement of a fingerprint image acquired from a finger surface. Point processing in the pointing device is described below. The operations of the light emitting device 22 , the light collecting device 23 , the fingerprint acquiring device 24 , the feature point extracting device 25 , the storage device 26 and the movement detecting device 27 are the same as those described above. However, when performing the point-fixing function, the operation device 31 receives displacement data about the feature points of the fingerprint image or fingerprint image set calculated in the movement detection device 27, and uses the displacement data to calculate the direction and distance that the pointer will move on the monitor . That is, as shown in FIG. 4, the arithmetic device 31 calculates the required direction and the required distance in which the pointer will move. Here, although the movement detection device 27 calculates the displacement data using the feature points of the acquired fingerprint image, the displacement data can be directly calculated using the digital fingerprint image data.

8 is a flowchart showing fingerprint recognition processing in the pointing device according to the first or second embodiment of the present invention.

If n is set to 1 after the pointing device is initialized (S802), each fingerprint acquiring means 24, 24-1, 2, 3 acquires n (n=1) fingerprint images of 20*20 pixels (S803). Here, when multiple fingerprint acquiring devices 24-1, 2, 3 are used as shown in FIG. 6, the pointing device can simultaneously acquire multiple fingerprint images (fingerprint image sets). As a result, although the size of the fingerprint image acquired each time from each fingerprint acquiring device is about 20*20 pixels, the overall size of the fingerprint image acquired each time can be adjusted by controlling the number of fingerprint acquiring devices.

Feature point extracting means 25 extracts at least one or more feature points from n fingerprint images acquired by fingerprint acquiring means 24, 24-1, 2, 3, and stores the feature points in storage means 26 (S804).

Next, the mapping means 28 uses the extracted feature points to map n fingerprint images in the virtual image space 29 (S805).

The identifying means 30 identifies whether the size of the entire fingerprint image mapped in the virtual image space 29 becomes the previously set size (S806). Here, the set size indicates the minimum size required in user identification. That is, although each fingerprint image acquired from each fingerprint acquiring device 24, 24-1, 2, 3 has a size of about 20×20 pixels, the size of the fingerprint image is enough to obtain the movement information of the fingerprint image but not enough. Obtain information for user identification. That is, a fingerprint image of 20*20 pixels is enough to obtain the movement information of the fingerprint image, but a fingerprint image of about 100*100 pixels is required for user identification through the fingerprint image. As a result, the set size of the fingerprint image is approximately 100×100 pixels, which is the size of the virtual image space 29 .

If the size of the overall fingerprint image mapped in the virtual image space 29 in step S806 is less than the previously set size, that is, the size of the virtual image space 29, then the variable n is incremented by 1 (S807), then continue to obtain the fingerprint image (S803～S806) . The fingerprint image acquisition process continues until the size of the overall fingerprint image mapped in the virtual image space 29 reaches the previously set size.

When the size of the overall fingerprint image mapped in the virtual image space 29 reaches the previously set size in step S806, the identification device 30 extracts at least one or more feature points from the overall fingerprint image mapped in the virtual image space 29 (S808).

The identification means 30 compares the feature points of the overall fingerprint image extracted in step S808 with the feature points of the previously registered fingerprint images (S809).

The recognition device 30 determines whether the feature points compared in step S808 are the same (S810), and if the feature points are the same, the recognition device 30 authenticates the user (S811), but refuses to authenticate the user if not (S812).

FIG. 9 is a detailed flowchart showing the process of mapping the fingerprint image in the virtual image space (S805) in FIG. 8. Referring to FIG.

First, the first fingerprint image is mapped in a specific position in the virtual image space 29 (S901). Here, the first fingerprint image (or set of fingerprint images) is preferably mapped at the center of the virtual image space 29 .

Then, when the second fingerprint image is obtained by the fingerprint acquisition device 24, 24-1, 2, 3 in the next cycle, the mobile detection device 27 receives the second fingerprint image (S902), to calculate the second fingerprint image and the first fingerprint Displacement data (distance and direction) of the image (S903). Here, the second fingerprint image is a fingerprint image obtained within a predetermined time interval according to the movement of the fingerprint. The displacement data in step S903 is calculated using the movement information of the feature points of the first fingerprint image and the second fingerprint image. Thereafter, the mapping device 28 maps the second fingerprint image to the corresponding position in the virtual image space 29 according to the displacement data calculated by the motion detection device 27 ( S904 ).

Fingerprint acquisition, displacement data calculation and mapping operations are performed continuously n times until the size of the overall fingerprint image reaches the previously set size, ie, the size of the virtual image space 29 (S905-S908).

As shown in Figures 8 and 9, the fingerprint images of about 20×20 pixels obtained n times according to the set cycle are synthesized into one large fingerprint image to have the size required in user identification, for example about 100×100 pixels . An image having a size required in user identification can be obtained by synthesizing fingerprint images acquired at each position and their relative motion information.

FIG. 10 is a flowchart showing a process of controlling a pointer in a pointing device according to the present invention.

If the finger 20 touches the transparent member 21 (S1001), the nth fingerprint image is obtained by the fingerprint acquisition device 24, 24-1, 2, 3 (S1002). Then, the (n+1)th fingerprint image is obtained by the fingerprint obtaining means 24, 24-1, 2, 3 according to the previously set period (S1003). The movement detecting means 27 calculates the degree of movement from the nth fingerprint image to the (n+1)th fingerprint image, that is, displacement data (S1004). The arithmetic means 31 calculates the coordinate value of the pointer, that is, the direction and distance of movement, using the displacement data (S1005). Next, corresponding to the coordinate value of the pointer calculated by the arithmetic means 31, the processor (shown) moves the pointer in conjunction with the arithmetic means 31 (S1006).

In this way, in FIGS. 3 and 6, the pointing device maps a plurality of fingerprint images each having a size suitable for pointer control and acquired by fingerprint acquisition means 24, 24-1, 2, 3, and extends it to Have a size suitable for user identification. As a result, user identification and pointer control can be simultaneously performed using one fingerprint recognition sensor.

FIG. 11 is a view showing the structure of a pointing device according to a third embodiment of the present invention.

The pointing device in FIG. 11 includes a light emitting device 22 , a light collecting device 23 , a fingerprint acquisition device 34 , a fingerprint identification unit 100 and a pointing control unit 200 .

The fingerprint identification unit 100 includes a feature point extraction device 35 and an identification device 36 , and the fixed point control unit 200 includes a fingerprint image extraction device 37 , a movement detection device 38 and a computing device 39 . Here, the pointing device according to the embodiment of the present invention may further include a casing (not shown), and the casing includes a light-emitting device 22, a light-collecting device 23, a fingerprint device 34, a feature point extraction device 35, a motion detection device 38 and an arithmetic device 39, and includes a transparent member 21, wherein the surface of the finger touches the fingerprint acquisition device 34 at a predetermined distance. More preferably, the pointing device of FIG. 11 is properly installed in the portable terminal device.

In the pointing device of FIG. 11 , when the finger 20 touches the transparent member 21 , the light emitting device 22 emits light to the surface of the finger 20 . The light emitting device 22 includes at least one or more light emitting diodes (abbreviated "LEDs").

The light collecting device 23 collects the light reflected from the surface of the finger 20 after the light is emitted from the light emitting device 22 to the finger 20 . A common optical convex lens can be used as the light concentrating device 23 .

The fingerprint acquiring device 34 acquires a fingerprint image of the finger surface for controlling the pointer using the light collected by the light collecting device 23 . The fingerprint acquiring device 34 converts the simulated fingerprint image collected by the light collecting device 23 into a digital fingerprint image to obtain a fingerprint image of M×N pixels. Here, the size of M×N pixels acquired by the fingerprint acquiring means 34 represents a size required in user identification. That is, the size of M×N pixels represents the size at which user identification is performed on the fingerprint image by using the fingerprint image acquired at one time. The fingerprint acquisition device 34 includes an optical sensor array in which a plurality of CMOS image sensors (abbreviated as "CIS") are arranged two-dimensionally. Here, the fingerprint acquiring means 34 acquires fingerprint images in a previously set cycle. The fingerprint capture device 34 is made suitable for small portable devices and uses a CIS for capturing large fingerprint images exceeding about 100x100 pixels. In this way, like the fingerprint capture device 34 for user identification, the CIS can be used to capture fingerprint images with various sizes, typically ranging from 90x90 pixels to 400x400 pixels. Therefore, the size of the fingerprint image acquired by the fingerprint acquiring device 34 of the third embodiment of the present invention is different from that of the fingerprint image acquired by the fingerprint acquiring device 24 of the first or second embodiment of the present invention.

The light generated from the light emitting device 22 is emitted to the surface of the finger 20 and reflected according to the pattern of the surface of the finger 20 . The light reflected from the bottom surface of the finger forms an image in the fingerprint capturing device 34 through the light collecting device 23 . The image formed in the fingerprint acquisition device 34 is converted into a digital fingerprint image by the fingerprint acquisition device 34 . This fingerprint image acquisition is performed in rapid succession on the time axis.

The fingerprint recognition unit 100 extracts feature points from a fingerprint image acquired by the fingerprint acquirer 34 at an operation cycle different from that of the fingerprint acquirer 34, and performs user identification by comparing the extracted feature points with feature points of a previously registered fingerprint image. . The fingerprint recognition unit 100 compares the extracted feature points with those of a previously registered fingerprint image typically once to three times per second. That is, fingerprint recognition processing for authenticating a user is performed by receiving 1-3 fingerprint images per second, extracting feature points from the received fingerprint images, and comparing the extracted feature points with feature points of previously registered fingerprint images . More preferably, the fingerprinting process is performed every second. The fingerprint identification unit 100 includes a feature point extracting means 35 for extracting feature points from the acquired fingerprint image and a feature point for comparing the feature points of the fingerprint image extracted by the feature point extracting means 35 with the feature points of the previously registered fingerprint image An identification means 36 for performing user identification.

The fixed-point control unit 200 extracts a fingerprint image of m×n pixels (M, N>m, n) from the fingerprint image acquired by the fingerprint acquisition device 34 in an operation cycle different from that of the fingerprint identification unit 100, to detect the fingerprint image of the fingerprint image. mobile information. The pointing control unit 200 uses the detected movement information to calculate displacement data, and uses the calculated displacement data to calculate the direction and distance that the pointer will move. Preferably, the fixed-point control unit 200 detects movement information of the feature points of the fingerprint image, and calculates displacement data of the feature points according to the movement information. Corresponding to the displacement data of the feature points, the fixed point control unit 200 calculates the moving direction and distance of the pointer.

The fixed-point control unit 200 extracts fingerprint images of about 20×20 pixels from the fingerprint images acquired in the previously set period, calculates displacement data of each fingerprint image and uses the displacement data to calculate two-dimensional coordinates (ΔX, ΔY), That is, the two-dimensional direction and distance the pointer will move. The fixed-point control unit 200 extracts fingerprint images about 800-1200 times per second, and calculates the displacement data of each fingerprint image extracted according to the corresponding period.

The fingerprint identification unit 100 and the pointing control unit 200 respectively operate according to different operation cycles to perform user identification and pointer control operations. That is, when the user of the pointing device controls the pointer using the fingerprint image, the fingerprint recognition unit 100 performs fingerprint verification of the user independently of the pointing control process. As a result, during pointer-controlled navigation, fingerprint verification is performed periodically without additional fingerprint recognition processing.

Hereinafter, operations of the fingerprint recognition unit 100 and the pointing control unit 200 are described in detail.

The feature point extraction means 35 extracts at least one or more feature points from the fingerprint image acquired each time according to the previously set period. These feature points include the length and direction of the lines in the fingerprint image and the location data where the lines separate or end.

The identification means 36 compares the feature points of the fingerprint image extracted from the feature point extraction means 35 with the feature points of the previously registered fingerprint images to perform user identification based on the identification of the two fingerprint images. Here, the recognition device 36 may include a comparison device (not shown) for combining the obtained fingerprint image with the previously registered fingerprint image global information and local feature information or matching the feature points of the two fingerprint images to compare the two. fingerprint image. Identification means 36 uses comparison means to determine the authentication of the two fingerprint images.

If the feature points of the previously registered fingerprint image are identical to the feature points of the fingerprint image extracted from the feature point extracting means 35, the identifying means 36 performs user identification, or rejects the user identification if not identical.

The fingerprint image extraction unit 37 extracts a fingerprint image of m×n pixels (where M, N>m, n) from the fingerprint image of M×N pixels acquired by the fingerprint acquisition unit 34 . The size of m x n pixels indicates the size used in pointer control. A size of 20x20 pixels for pointer control in a pointing device is sufficient for fingerprint images that are typically extracted from small fingerprint images. The fingerprint image extraction means 37 extracts a fingerprint image from about 15*15 pixels to about 80*80 pixels. As a result, the fingerprint image extracting means 37 extracts a fingerprint image of about 20 x 20 pixels for pointer control from the fingerprint image of about 100 x 100 pixels acquired by the fingerprint acquiring means 34 for user identification. Here, the size of the fingerprint image used is just an example of the present invention. That is, the acquired size of MxN pixels is preferably suitable for user identification, and the extracted size of mxn pixels is preferably suitable for pointer control.

The movement detection device 38 acquires the degree of movement of each fingerprint image acquired each time according to the set period. Here, the movement detecting means 38 preferably detects the degree of movement of the fingerprint by calculating displacement data (direction and distance) related to feature points of the fingerprint image acquired in a set period and using a movement estimation method. More preferably, the movement detecting means 38 detects the degree of movement of the fingerprint image by calculating displacement data about the feature points of the fingerprint image acquired in a set period. Here, the displacement data of the fingerprint image is preferably calculated by calculating the moving distance and direction from the feature point of the fingerprint image acquired in the previous cycle to the feature point of the fingerprint image acquired in the current cycle. Movement information of a fingerprint image and feature point extraction and fingerprint acquisition in fingerprint recognition are important factors because movement of a fingerprint image and reliability of fingerprint recognition are distinguished according to how reliably feature points can be extracted.

The computing device 39 receives the movement degree of the fingerprint image from the motion detection device 38, that is, displacement data, and uses the received displacement data to calculate two-dimensional coordinates (ΔX, ΔY), that is, the direction and distance/movement degree that the pointer will move.

The arithmetic means 39 is generally combined with a pointing device or a processor of a device having a pointing device. As a result, the processor can control the movement of the pointer on the screen of the display device according to the coordinates calculated in the arithmetic means 39 .

The pointing device according to one embodiment of the present invention may further include display means (not shown) for displaying previously stored information. In this way, when the pointing device further includes a display device, the display device receives a signal according to the fingerprint recognition by the fingerprint recognition unit 100 to display the recognition result. When the identification of the user is successfully performed in the fingerprint identification unit 100, information for performing all functions of the corresponding terminal is displayed on the display means. However, when user identification is rejected, restrictive information is displayed, which can only perform specific functions of the terminal.

A technique of restrictively allowing use of a corresponding terminal through user identification will be mentioned later.

The fingerprint acquiring device 34 in the third embodiment of the present invention can be embodied by semiconductor devices as shown in the first and second embodiments of the present invention.

Meanwhile, when the fingerprint capturing device 34 is embodied as an optical system, a small fingerprint capturing device, ie, a "reduced optical system", can be used to obtain a large fingerprint image for user identification. In other words, the size of the fingerprint acquiring device 34 can be miniaturized by reducing the size of the actual fingerprint by 1/2˜1/4 and acquiring the reduced fingerprint image.

The principle and processing of acquiring a fingerprint image using a reduced optical system will now be described in detail.

FIG. 12 is a view showing a design example of a 3:1 reduction optical system installable in a minute space according to the present invention. As shown in FIG. 12, an aspheric mirror 42 is used to mount an optical device in a small space. If the optical system is configured as shown in FIG. 12 , the size of the actual object represented by the left arrow 41 is reduced to approximately 1/3 the size represented by the right arrow 43 . If the image of the object represented by the left arrow 41 passes through the aspheric mirror 42, an inverted image of about 1/3 size will be formed in the fingerprint acquisition device 34 on the right. The reduction optical system is embodied by applying the above-mentioned principle so that the size of the actual fingerprint is reduced to a size of 1/n (where n is a real number from 1 to 5).

FIG. 13 is a view showing an example of a fingerprint image obtained from a fingerprint acquiring device by applying the reduction optical system of FIG. 12 . Fig. 13a shows a fingerprint image acquired from the fingerprint acquisition device 34 at an optical system of 1:1, and Fig. 13b shows a fingerprint image acquired from the fingerprint acquisition device 34 at a reduction optical system of 4:1.

Generally, about two valleys are formed per 1 mm in a human fingerprint. As a result, when using the fingerprint acquisition device 34 with 20×20 pixels, the recognition pixel of the fingerprint acquisition device 34 is 0.5mm, and the number of fingerprints acquired by the fingerprint acquisition device 34 is only 2, as shown in FIG. 13a. In this way, when the number of grooves of the acquired fingerprint image becomes small, the accuracy of user identification decreases, and the performance of pointer control also decreases. As a result, the size of the sensor needs to be large for adequate data collection.

In order to overcome the above-mentioned problems, more data can be obtained by applying a reduction optical system to the fingerprint acquisition device 34 without increasing the size of the sensor. That is, in the embodiment of the present invention, the size of the fingerprint with an interval of 0.5 mm is obtained by reducing the size of the fingerprint by 1/n (where n is a real number from 1 to 5). More specifically, the size of the fingerprint is reduced to 1/2 to 1/4 the size. As a result, it is possible to obtain more data than in FIG. 13a using a fingerprint acquisition device 34 having the same dimensions as in FIG. 13a. As shown in FIG. 13b, when the size of the fingerprint is reduced to 1/4 size using the reduction optical system, the fingerprint spacing of 0.5mm can be reduced to about 0.125mm. Therefore, 4 to 16 times more fingerprint information can be obtained using the fingerprint acquiring device 34 having the same size as that of FIG. 13a. In other words, when fingerprint images are obtained by reducing fingerprints having an average interval of 0.5mm to 1/2˜1/4 size, the size of the fingerprint acquiring device 34 can be miniaturized to 1/4˜1/16. As a result, the fingerprint image for user identification and pointer control can be obtained using the low-power and low-cost miniaturized fingerprint acquisition device 34, and the miniaturized fingerprint acquisition device 34 is advantageous when applied to a small portable terminal device.

FIG. 14 is a view showing a pointing device with a fingerprint recognition function according to a fourth embodiment of the present invention.

Compared with FIG. 11 , the pointing device of FIG. 14 further includes a storage device 60 . The storage device 60 stores the fingerprint image obtained from the fingerprint obtaining device 34 . In the pointing device of FIG. 14 , if the fingerprint image obtained from the fingerprint obtaining device 34 is stored in the storage device 60 according to the previously set cycle each time, then the fingerprint identification unit 100 and the pointing control unit 200 use the image stored in the storage device 60. fingerprint image to perform user identification and pointer control, respectively. That is to say, when user identification and pointer control are respectively performed in the fingerprint recognition unit 100 and the pointing control unit 200, it immediately receives the fingerprint image acquired in the pointing device of FIG. The fingerprint image acquired by the acquisition device 34 is first stored in the storage device 60, and then only the fingerprint image with the size required for pointer control is used to perform pointer control, so that the pointer can also be embodied in low cost, low energy consumption and high speed navigation information generation. Here, of course, user identification and pointer control are simultaneously performed using the fingerprint image stored in the storage device 60 .

In this way, by extracting a fingerprint image having a size required in pointer control from the fingerprint image acquired by the fingerprint acquiring means 34 for user identification in the pointing device of FIGS. 11 and 14, user identification and pointer control can be Use a fingerprint recognition sensor to do both.

FIG. 15 is a view showing a method for extracting a fingerprint image of m*n pixels from a fingerprint image of M*N pixels.

The fingerprint acquiring device 34 acquires the fingerprint image 71 of M×N pixels according to a predetermined cycle. The fingerprint image 71 of M*N pixels has a size sufficient for user identification. Preferably, the fingerprint image 71 has a size from 90x90 pixels to 400x400 pixels. In addition, the fingerprint image extracting means 37 extracts a fingerprint image 72 of mxn pixels from the fingerprint image 71 of MxN pixels. Here, the fingerprint image extracting means 37 extracts the center portion of the fingerprint image 71 of M*N pixels. The size of m x n pixels represents the size of the fingerprint image 72 that can be controlled by the pointer. Fingerprint image 72 has dimensions from 15x15 pixels to 80x80 pixels.

In the pointing device of FIG. 14, a fingerprint image 71 of M*N pixels is stored in the storage device 60, and user identification is performed using the fingerprint image 71 of M*N pixels. Meanwhile, pointer control is performed using the fingerprint image 72 of m×n pixels extracted from the fingerprint image 71 of M×N pixels.

In the case where the pointer is adjusted using the fingerprint image 72 of m×n pixels, when the surface of the finger 20 moves ΔX and ΔY from the first position to the second position, with respect to the m×n image extracted from the fingerprint image extracting device 37 The data of the pixel's fingerprint image is sent to the movement detection device 38. Here, fingerprint images are sent at a rate of about 800-1200 times per second. As a result, the displacement data of the fingerprint image depending on the movement of the finger 20 is calculated and converted into a velocity, and the moving direction and distance of the pointer are also calculated and converted into a velocity. Here, since the above processing speed is required for stable pointing operation in the pointing device according to the embodiment of the present invention, it is preferable to select the smallest image size to reduce the amount of processing and calculation.

At the same time, the overall fingerprint image 71 of M×N pixels required for user identification is sent to the fingerprint identification unit 100 . The fingerprint image 71 is sent at a rate of 1 to 3 times per second, where general identification processing can be performed. The fingerprint recognition unit 100 is configured to be included in the processing device of the portable terminal device so that the fingerprint recognition unit 100 can perform the function.

The process of calculating the displacement data of the fingerprint image 72 using the feature points of the fingerprint image 72 is the same as that of FIG. 4 .

FIG. 16 is a flow chart illustrating a method for performing user identification and simultaneously performing pointing control according to the third or fourth embodiment of the present invention.

In the initialization state (S1601), if the surface of the finger 20 touches the transparent member 21 (S1602), the fingerprint image of M×N pixels is acquired by the fingerprint acquisition device 34 according to the first operation cycle (S1603).

The fingerprint recognition unit 100 and the fixed-point control unit 200 simultaneously perform user recognition (S1620) and pointer control (S1630) using the fingerprint image 71 obtained by the fingerprint acquisition device 34. In the case of the pointing device of FIG. 14 , user identification (S1620) and pointer control (S1630) are stored in the storage device 60 except that the fingerprint image 71 obtained is stored in the storage device 60 and the fingerprint is used using the fingerprint image 61 stored in the storage device 60. Same as the pointing device of Figure 11.

In the user identification (S1620), the feature point extraction device 35 extracts at least one or more feature points from the fingerprint image of M*N pixels according to the second operation cycle, to send the feature points to the recognition device 36 (S1604). The recognition means 36 compares the feature points extracted from the fingerprint image of M*N pixels with the feature points of the previously registered fingerprint images (S1605). The recognition means 36 determines from the comparison result whether the feature points of the two fingerprint images are the same (S1606). If the feature points of the two fingerprint images are the same, the identification means 36 authenticates the user (S1607), and if not, rejects the user identification (S1608).

Next, in the pointer control process (S1630), the fingerprint image extracting means 3 extracts the fingerprint image 72 of m×n pixels from the fingerprint image of M×N pixels according to the third cycle to send the extracted fingerprint image to The detection device 38 is moved (S1609). In this embodiment, m and n range from 15 to 80 to have a size suitable for pointer control of the extracted fingerprint image 72 . The motion detecting device 38 calculates the displacement data of the fingerprint image 72 of m×n pixels to send the displacement data to the computing device 39 (S1610). Here, the movement detecting means 38 calculates displacement data by calculating the degree of movement (ie, distance and direction) from the fingerprint image acquired in the previous cycle to the fingerprint image acquired in the current cycle. Preferably, displacement data depending on the degree of movement of the feature points of the extracted fingerprint image 72 is calculated. The operation means 39 uses the displacement data calculated in the movement extraction means 38 to calculate the coordinates at which the pointer will move (S1611). Corresponding to the coordinates of the pointer, the processor (not shown) of the terminal device moves the pointer.

As described in FIG. 16 , the pointing device according to the embodiment of the present invention can simultaneously perform user identification and pointer control by using the fingerprint image 71 acquired from the fingerprint acquisition sensor 34 . User identification processing (S1620) and pointer control processing (S1630) are executed in previously set different operation cycles, and these two processings S1620 and S1630 are executed separately. That is, when the user adjusts the pointer using the fingerprint (S1630), the user identification process (S1620) is naturally performed. As a result, no fingerprint recognition processing is required for the user, and the fingerprint is automatically recognized during pointer control, so that the range of services available to the user can be adjusted to the corresponding device according to the fingerprint recognition result.

FIG. 17 is a view showing the structure of a pointing device according to a fifth embodiment of the present invention.

Compared with the above-described embodiments of the present invention, the fifth embodiment is characterized by not including the process of extracting feature points from a fingerprint image during the fingerprint recognition function. That is, the location where the acquired fingerprint image is stored in the storage device based on the extracted feature points is not determined. On the contrary, the mapping position of the fingerprint image is determined according to the moving distance, that is, the displacement data and stored in the storage device.

Hereinafter, the structure of the pointing device of FIG. 17 is described in detail.

The pointing device in FIG. 17 includes a transparent member 21 , a light emitting device 22 , a light collecting device 23 and a fingerprint acquiring device 34 . However, description of the structure of these elements is omitted because it is the same as that of the third or fourth embodiment.

The fingerprint image obtained by the fingerprint acquisition device 34 is immediately input into the fixed-point control unit 200 including the fingerprint image extraction device 37 , the movement detection device 38 and the computing device 39 . Since the operations of specific elements of the fixed-point control unit 200 are the same as those of the above-mentioned third or fourth embodiment, specific descriptions are omitted.

However, compared with the third or fourth embodiment, the operation of the fifth embodiment is characterized in that the fingerprint image of m×n pixels extracted by the fingerprint image extracting means 37 is stored in the storage means 40 and the storage location is based on the The data of the displacement value calculated by the arithmetic unit 39 is used for mapping. Specifically, it is assumed that the i-th fingerprint image extracted from the fingerprint image extraction means 37 is stored in a specific location of the storage means 40 . If the (i+1)th fingerprint image is extracted from the fingerprint image extraction means 37, the displacement data ΔX and ΔY obtained by the movement detection means 38 and the arithmetic means 39 are received, and the (i+1)th fingerprint data is stored in the The specific location where the i-th fingerprint data is stored is moved to the location of the displacement data.

The operation of storing fingerprint data in the storage device 40 is performed by a method of periodically storing data according to a predetermined time interval or when a specific command is received.

In addition, the fifth embodiment of the present invention includes a CPU 50 for controlling the operation of storing fingerprint data in the above-mentioned storage device 40, controlling the movement of the pointing device and executing the graph by receiving the displacement data ΔX and ΔY from the arithmetic device 39. The fingerprinting operation described in 18.

Fig. 18 is a flowchart showing the operation of the pointing device according to the fifth embodiment of the present invention.

If the system is initialized and the surface of the finger 20 is in contact with the transparent member 21 (S1810), a fingerprint image of m×n pixels is acquired by the fingerprint acquisition device 34 and the fingerprint image extraction device 37 (S1820).

The acquired fingerprint image is stored in the storage device 40 (S1830), and displacement data and pointer coordinates on the fingerprint image are calculated (S1840, S1850). The calculation result is provided to the storage device 40 and used to map the fingerprint image.

The calculation result is also supplied to the CPU 50 and used to control the operation of the pointing device (S1860, S1870).

At the same time, the CPU 50 receives the fingerprint image from the storage device 40. If the received fingerprint image is not a fingerprint image of M×N pixels, the CPU 50 receives the fingerprint image from the storage device 40 again (S1880). If the received fingerprint image is the same as the fingerprint image of M*N pixels, the fingerprint image is compared with the previously stored fingerprint image (S1890), and identification of the two fingerprint images is determined (S1899). Then, user identification or identification rejection operation is performed according to the result.

FIG. 19 is a flowchart illustrating a method for restricting usage of a portable communication terminal device depending on a user by using a fingerprint recognition function according to the present invention.

In one embodiment, a portable terminal device having a fingerprint recognition function recognizes a user's fingerprint to perform fingerprint recognition on the user (S1900). The portable terminal device identifies whether a person intending to use the terminal device is the user himself or not through fingerprint recognition (S1910). When the user is the user himself, the terminal device displays the entire menu that the terminal device can provide (S1920), so that the user can use all functions (services) (S1930). However, when the user is not the user himself, the terminal device only displays a specific menu that can be allowed to be used by the person (S1940). In this way, important information such as credit card and financial information can be protected by restricting the use of the terminal device by individuals who are not identified through fingerprint identification.

If a user other than the user himself intends to use an impermissible function (S1950), the terminal device displays a message indicating that the corresponding function cannot be used (S1960). Then, after several seconds, the terminal device displays again the menu that the person is allowed to use (S1940). Therefore, in a case where a person is not a previously registered user, the user's personal information, control information of the system, and paid services such as electronic commerce can be protected by restricting access of other users to protect the personal information.

Fingerprint recognition processing can be used as background processing to alleviate the inconvenience caused by the use of the portable communication terminal device. Here, when a user uses a two-dimensional pointer with a finger, background processing automatically performs steps of collecting, analyzing, and authenticating data required in fingerprint recognition without notifying the user of these steps. Required processing is performed when the user uses processing previously designated for protection in the system or when performing an approval transaction, in the case of the user himself using data obtained from background processing. However, when the user is not the user himself, the processing is rejected to protect the owner's information or property.

That is, face recognition processing is performed in conjunction with background processing. When the user is the user himself, the next steps are successively performed, but when the user is not the user himself, the subsequent use is rejected. As a result, desired stability can be ensured without impairing user's convenience. Here, the same sensor components in fingerprint enrollment and authentication as the two-dimensional pointing device are used, and the pointing device is configured to simultaneously perform data collection and software processing.

In the above-described embodiments, if the user registers his or her fingerprint by himself, the registered data is saved in the nonvolatile memory and used repeatedly until the user himself changes the data.

In another embodiment of the present invention, when a user uses a portable communication terminal device, instead of a fingerprint recognition sensor for fingerprint recognition, a pointing device with a fingerprint verification sensor function can identify the portable terminal device user through the user's fingerprint recognition Whether it is the user himself. For this operation, when the user moves his or her finger, the portable terminal device collects two-dimensional movement information, and generates a two-dimensional fingerprint with a size required for personal identification by synthesizing the collected movement information and the fingerprint image in the corresponding position. image, and extract feature points from the corresponding fingerprint image. Then, the portable terminal device registers the extracted feature points or compares the extracted feature points with registered data for personal identification.

FIG. 20 is a view showing an example of a portable terminal device including a pointing device according to the present invention. Portable terminal devices include cellular phones, PDAs or smart phones.

In the portable terminal device, the outer surface of the transparent member 230 is exposed, and when a finger is placed on the outer surface of the transparent member 230, a fingerprint image having a size required for user identification is obtained through fingerprint acquisition. If a fingerprint image is obtained, the existing menu window is converted into a service screen 240 as shown in FIG. 20 .

In the portable terminal service, the service can be used by selecting and clicking a menu on the service screen 240 not using a movement key such as a mouse of a general-purpose computer but using the pointer 250 . In addition, the portable terminal device includes at least one or more function buttons 220 for performing other functions or inputting performance commands.

While the invention is susceptible to various modifications and alternative forms, specific embodiments are shown by way of example in the drawings and detailed description. It should be understood, however, that the invention is not intended to be limited to the particular forms disclosed. On the contrary, the invention covers all modifications, equivalents and alternatives falling within the spirit and scope of the invention as defined in the appended claims.

industrial application

In the pointing device with fingerprint recognition function and the fingerprint recognition method according to one embodiment of the present invention, a fingerprint image with a small size is mapped to generate a large fingerprint image or a small fingerprint image extracted from a large fingerprint image for pointing The device performs user identification and pointer control. As a result, sensors for user identification and for pointer control are not included in the pointing device, but one sensor for performing both functions of user identification and pointer control is included in the pointing device according to one embodiment of the present invention. in the device. In addition, since user identification requiring a large fingerprint can be performed using only a small fingerprint image, a miniaturized portable terminal device can be easily realized, thereby reducing manufacturing cost.

In addition, it is possible to protect important information in the terminal device by restricting the kinds of services usable in the terminal device according to user identification.

Claims (53)

1. A pointing device with fingerprint image recognition function comprising: At least one or more fingerprint acquisition devices are used to acquire fingerprint images of the finger surface according to a predetermined period; Feature point extracting means for extracting at least one or more feature points from the acquired fingerprint image; A movement detection device, used to calculate the displacement data between the feature points of the extracted fingerprint image, to detect the movement information of the fingerprint image; a mapping device for mapping the fingerprint image in the internal virtual image space according to the movement information; identification means, when the overall size of the mapped fingerprint image reaches a previously set size, the identification means compares the entire mapped fingerprint image with a previously registered fingerprint image, and determines recognition of the fingerprint; and computing means for receiving the displacement data from the movement detecting means and using the displacement data to calculate the direction and distance the pointer will move. 2. The pointing device as claimed in claim 1, further comprising a housing, said housing comprising a fingerprint acquisition device, a feature point extraction device, a movement detection device, a mapping device, an identification device and an operation device, and including a finger surface contacted therewith A transparent member with a planar surface, the transparent member has a predetermined distance from the fingerprint acquisition device. 3. The pointing device according to claim 1 or 2, wherein the fingerprint acquisition means is a CMOS image sensor. 4. The pointing device according to claim 1 or 2, wherein the dimensions of the acquired fingerprint image and virtual image space are m*n pixels and M*N pixels respectively, and m and n are smaller than M and N respectively. 5. The pointing device as claimed in claim 1 or 2, wherein the movement detection means calculates the moving distance and direction from the feature point of the fingerprint image acquired in the previous cycle to the feature point of the fingerprint image acquired in the current cycle. 6. The pointing device as claimed in claim 1 or 2, wherein the mapping means maps the fingerprint image in the virtual image space, so that when there is the same feature point between the feature points of the n-1th fingerprint image and the nth fingerprint image, Overlap the same feature points. 7. The pointing device as claimed in claim 1 or 2, wherein the identification means determines whether the feature points of the entire mapped fingerprint image are consistent with the feature points of the previously registered fingerprint image by matching the feature points of the entire mapped fingerprint image The feature points of the previously registered fingerprint images are the same, and recognition of the fingerprint is judged based on the determination result. 8. A portable terminal device comprising the pointing device as claimed in claim 1 or 2 and performing user's fingerprint recognition and control of the pointer. 9. A method for identifying a fingerprint for user identification, comprising the steps of: a fingerprint image acquisition step for acquiring at least one or more fingerprint images according to a set period using a predetermined fingerprint acquisition sensor; A feature point extraction step for extracting at least one or more feature points from the acquired fingerprint image; The first mapping step is used to map the first fingerprint image in a specific position in the virtual image space; A displacement data calculation step for calculating displacement data between the feature points of the first fingerprint image and the feature points of the second fingerprint image acquired in the next period after the period in which the first fingerprint image has been acquired; a second mapping step for mapping the second fingerprint image in virtual image space using the displacement data; and A fingerprint recognition step for comparing feature points of the entire mapped fingerprint image with feature points of previously registered fingerprint images when the overall size of the fingerprint image mapped in the virtual image space reaches a previously set size, and determining the correctness of the fingerprint image recognition. 10. The method according to claim 9, wherein the dimensions of the acquired fingerprint image and virtual image space are m×n pixels and M×N pixels respectively, and m and n are smaller than M and N respectively. 11. The method as claimed in claim 9, wherein the displacement data calculation step is to calculate the moving distance and direction from the feature point of the first fingerprint image to the feature point of the second fingerprint image. 12. The method of claim 9, wherein the second mapping step is to map the second fingerprint image in a position corresponding to the calculated displacement data from the mapped first fingerprint image in the virtual image space. 13. The method according to claim 9 or 12, wherein the second mapping step is to map the second fingerprint image in the virtual image space, so that when there is the same feature point between the feature points of the first fingerprint image and the second fingerprint image , overlapping the same feature points. 14. The method according to claim 9 or 12, wherein the fingerprint image acquisition step is to acquire multiple fingerprint images each time by using multiple fingerprint acquisition sensors. 15. The method of claim 14, wherein the plurality of captured fingerprint images are images of adjacent fingerprints. 16. The method of claim 9, wherein the step of fingerprinting comprises: The first step is to determine whether the size of the overall fingerprint image mapped in the virtual image space reaches the previously set size; The second step is to extract at least one or more feature points from the overall fingerprint image when the overall fingerprint image reaches the previously set size according to the determination result; A third step, comparing the extracted feature points with those of the previously registered fingerprint image; and The fourth step is to determine the identification of the fingerprint according to the comparison result. 17. A pointing device with fingerprint identification function, comprising: Fingerprint acquisition means (first operating cycle) for acquiring a desired fingerprint image of the finger surface that controls the pointer by only one two-dimensional image acquisition; A fingerprint identification unit (second operating cycle) for extracting at least one or more feature points from the acquired fingerprint image, and comparing the acquired feature points with feature points of a previously registered fingerprint image to identify the acquired fingerprint the user of the image; and The fixed-point control unit (the third operation period) is used to detect movement information according to the partial data of the image acquired in the first operation period, and calculate the displacement data of the fingerprint image according to the movement information, so as to calculate the movement direction and distance of the pointer. 18. The pointing device according to claim 17, wherein the fingerprint identification unit comprises: feature point extraction means for extracting at least one or more feature points from the fingerprint image acquired by the fingerprint acquisition means; and Identification means for comparing the feature points of the fingerprint image extracted by the feature point extracting means with those of previously registered fingerprint images to determine the user's identification of the fingerprint image. 19. The pointing device as claimed in claim 18, wherein the identifying means determines whether the feature points of the previously registered fingerprint image are identical to the extracted feature points by matching the extracted feature points with the feature points of the previously registered fingerprint images, And the identification of the user is performed according to the determination result. 20. The pointing device of claim 17, wherein the pointing control unit comprises: Fingerprint image extraction device, for extracting m * n pixels (herein, m and n are integers, m < M , n<N) fingerprint image; A motion detection device, used to calculate the displacement data of the extracted m×n pixel fingerprint image, to detect the motion information of each fingerprint image; and The computing device is used for receiving the displacement data from the mobile detection device and calculating the moving direction and distance of the pointer according to the displacement data. 21. The pointing device as claimed in claim 20, wherein the movement detection means calculates the moving distance and the distance from the feature point of the fingerprint image acquired in the previous cycle to the feature of the fingerprint image acquired in the current cycle to calculate the displacement of the fingerprint image data. 22. The pointing device of claim 20, wherein M and N range from 90 to 400, and m and n range from 15 to 80. 23. The pointing device according to claim 17, wherein the fingerprint acquiring means is a CMOS image sensor. 24. The pointing device of claim 17, wherein the fingerprint acquisition means is an active capacitive sensor. 25. The pointing device as claimed in claim 17, wherein the second operation cycle is 1-3 times/second. 26. The pointing device as claimed in claim 17, wherein the third operation cycle is 800˜1200 times/second. 27. The pointing device of claim 17, wherein the fingerprint recognition unit and the pointing control unit operate simultaneously according to the second operation cycle and the third operation cycle, respectively. 28. The pointing device of claim 17, further comprising: light emitting means for emitting light towards the surface of the finger controlling the pointer; and A light-gathering device for gathering fingerprint images reflected from the finger surface, The fingerprint image collected by the light collecting device is obtained by the fingerprint obtaining device. 29. The pointing device according to claim 28, wherein the light concentrating device is located between the light emitting device and the fingerprint acquisition device, The ratio of the distance between the light emitting device and the light concentrating device to the distance between the light concentrating device and the fingerprint acquisition device is n:1, and n is a real number from 1 to 5. 30. The pointing device according to claim 28, wherein the light collecting means is an aspheric mirror. 31. A portable terminal device comprising the pointing device as claimed in claim 17 and configured to simultaneously perform fingerprint recognition and pointer control. 32. The portable terminal device as claimed in claim 31 , further comprising display means for displaying previously stored information, Wherein when the user identification is performed in the fingerprint identification unit, the display device displays the entire information or the execution functions allowed to the user, and when the user identification is not performed, the display device only displays the previously set limited range of information and available functions. 33. A pointing method having a fingerprint recognition function, the pointing method being a pointing method of a pointer control device having an image sensor whose size is smaller than that of a predetermined picture image required in fingerprint recognition, said method Include the following steps: a fingerprint image acquisition step for acquiring at least one or more fingerprint images of M×N pixels on the surface of the finger controlling the movable pointer according to the first operation cycle using a predetermined fingerprint acquisition sensor; an identifying step for determining identification of the user of the fingerprint image by extracting feature points from the acquired fingerprint image and comparing the extracted feature points with feature points of previously registered fingerprint images according to the second operation cycle; A fingerprint image extraction step for extracting a fingerprint image of m×n pixels from the acquired fingerprint image according to the third operation cycle; A movement detection step for detecting movement information of each fingerprint image by calculating displacement data of each extracted fingerprint image of m×n pixels; and An operation step that uses displacement data to calculate and output the direction and distance the pointer will move. 34. The fixed point method as claimed in claim 33, wherein the identifying step comprises: When performing user identification, outputting a signal using previously set entire information; and When user identification is not performed, a signal using previously set limited information is output. 35. The fixed point method of claim 33, wherein M and N range from 90 to 400, and m and n range from 15 to 80. 36. The fixed-point method according to claim 33, wherein the third operation cycle is 800-1200 times/second. 37. The fixed-point method as claimed in claim 33, wherein the movement detection step calculates the moving distance and direction of the feature point from the k-1th fingerprint image to the kth fingerprint image to calculate the displacement data of each fingerprint image. 38. The fixed point method as claimed in claim 33, wherein the image extracting step, the movement detecting step and the calculating step are respectively performed from the recognizing step according to the third operation cycle. 39. The fixed-point method as claimed in claim 33, further comprising the steps of: a light emitting step for emitting light toward the surface of the finger controlling the pointer; and a gathering step for gathering the fingerprint image produced on the surface of the finger using an aspheric mirror, The fingerprint acquisition step is to acquire the fingerprint images gathered in the gathering step. 40. The pointing method according to claim 39, wherein the gathering step gathers the fingerprint image by reducing the size of the fingerprint on the finger surface by 1/n by adjusting the aspheric mirror. 41. The fixed point method of claim 40, wherein n is a real number from 1 to 5. 42. A pointing device with fingerprint recognition function, comprising: At least one or more fingerprint acquisition devices, used to acquire images of the finger surface according to predetermined periodic or occasional requirements; computing means for calculating displacement data using the acquired fingerprint image to calculate the direction and distance in which the pointer will move using the displacement data; storage means for mapping the fingerprint image obtained from the fingerprint acquisition means corresponding to the amount of movement in the displacement data received from the arithmetic means; and CPU is used to analyze and process the data of computing device and storage device. 43. The pointing device according to claim 42, wherein the CPU is formed with the arithmetic means and the storage means as one component. 44. The pointing device according to claim 42, wherein the CPU is formed as a structure separate from the arithmetic means and the storage means. 45. The pointing device of any one of claims 42 to 44, further comprising: light emitting means for emitting light towards the surface of the finger; and A light-gathering device for gathering fingerprint images reflected from the finger surface, The fingerprint image collected by the light collecting device is obtained by the fingerprint obtaining device. 46. The pointing device according to claim 45, wherein the light concentrating device is located between the light emitting device and the fingerprint acquisition device, The ratio of the distance between the light emitting device and the light concentrating device to the distance between the light concentrating device and the fingerprint acquisition device is n:1, and n is a real number from 1 to 5. 47. The pointing device as claimed in any one of claims 42 to 44, wherein the CPU further includes a function of processing fingerprint identification in software, for combining the fingerprint image stored in the storage device with the fingerprint image received from the fingerprint acquisition device fingerprint images for comparison. 48. The pointing device according to claim 42, wherein the storage means periodically stores the fingerprint image according to a predetermined period or stores the fingerprint image only when the storage means receives a request from the CPU. 49. A method for providing services to a portable terminal device with a fingerprint recognition function, the method comprising: The first step is to obtain the fingerprint image of the user through a pointing device having a fingerprint identification function, and compare the acquired fingerprint image with the previously registered fingerprint image to determine the identification of the user; and The second step is to classify the usage rights of the user according to the determination result of the first step to display a menu corresponding to the usage rights. 50. The method as claimed in claim 49, further comprising a third step of enabling a user to use a desired function of the menu displayed in the second step. 51. The method of claim 50, wherein the second step is to display the entire menu when the user identification is performed, or to display a specific menu that is allowed to be used when the user identification is not performed. 52. The method as claimed in claim 51, wherein when the unrecognized user intends to use other functions permitted by the menu, a message indicating that the use is not permitted is displayed and the use-permitted menu is displayed again. 53. The method of claim 49, wherein the first step is performed as a background process to automatically capture a fingerprint image by using a pointing device while the user is navigating with a finger, without performing additional processing to capture the user's fingerprint. fingerprint.

Images