JP5353196B2 - Moving measuring device and moving measuring program - Google Patents

Description

  The present invention relates to a movement measurement device and a movement measurement program. More specifically, the present invention relates to a movement measurement apparatus and a movement measurement program suitable for a pointing device for controlling a cursor position displayed on a screen.

  Electronic devices such as personal computers (hereinafter referred to as PCs) include a mouse, a touch pad (a touch sensor disposed near a keyboard such as a notebook PC), and a touch panel (a touch sensor superimposed on a display such as a mobile phone). For example, an input device called a pointing device for controlling a cursor position displayed on a screen is widely used.

  For example, Patent Document 1 discloses a technique of detecting a relative movement amount between a desk and an apparatus by photographing a desk surface with a mouse having an optical lens and a photodetector array.

  In addition, as described in Patent Document 2, a technique for detecting the movement of a subject from a plurality of images taken in time series with an imaging apparatus including a plurality of lenses is also known.

JP 2004-213673 A JP 2008-34948 A

  In view of this, a configuration is considered in which a user's hand or finger is photographed using an imaging device including a plurality of lenses, and the movement is recognized to be a pointing device.

  However, for example, the device described in Patent Document 1 is that the reflected image becomes dark in order to detect that the mouse is lifted from the desk surface (that is, the distance from the imaging device to the subject has changed). We are using. This is due to the fact that the imaging field of view is illuminated with the mouse in contact with the desk surface. For example, if the mouse is turned over, the ceiling illumination etc. will enter the field of view. There is a problem because it is not always dark.

  Further, in order to detect that the height of the subject has changed, a proximity sensor is provided. However, such a configuration including a separate sensor is costly and increases the possibility of failure. Furthermore, although the degree of correlation is used, there is a problem that the depth of field of the optical system needs to be sufficiently shallow in order to respond with a slight lift.

  In other words, unlike an optical mouse sensor that recognizes the movement of a desk surface, when shooting outward to recognize a user's finger, an object other than the subject to be recognized is captured or external light is incident. There is a problem of doing. In addition, it is necessary to correctly detect the subject even when the apparatus itself moves like the optical mouse sensor. In order to solve these problems, it is required to accurately detect the distance between the apparatus and the subject without being affected by the external environment of the apparatus or the movement of the apparatus itself. In the technique described in Patent Document 2, a plurality of lenses are used to widen the field of view or to read a narrow field of view at a high frame rate, and the purpose is to detect the distance between the apparatus and the subject. is not.

  Furthermore, in recent years, downsizing of notebook PCs and the like has progressed, and mobile communication devices such as mobile phones have been increasingly operated by touch panels instead of conventional button operations, so that the size of pointing devices is also desired to be reduced. Yes.

  SUMMARY OF THE INVENTION An object of the present invention is to provide a mobile measuring device that can accurately recognize an operation by a user without being affected by the external environment of the device or the movement of the device itself, with a small and simple configuration.

In order to achieve this object, the movement measuring apparatus according to claim 1 is an imaging optical system including a lens array in which two or more lenses are arranged in a substantially planar shape and an object image is formed on the image plane. An imaging means configured to capture a subject image by the imaging optical system, a first single-eye image that is image data of an imaging surface area formed by the first lens, The first individual image is obtained by either the first lens or the second lens, and the second single-eye image that is image data of the imaging surface area formed by the second lens simultaneously with the single-eye image. The amount of movement of the subject is calculated based on at least three single-eye images with the third single-eye image that is image data of the imaging surface area formed with a predetermined time difference from the eye image and the second single-eye image. and a moving amount calculation means for, three single-eye For two ommatidium images of the image, while shifting the one ommatidium image pixel by pixel, the movement amount calculating means for calculating a movement amount of the luminance difference is smallest between ommatidium images, pieces were taken at the same time A first shift amount calculating unit that calculates a shift amount between eye images; a second shift amount calculating unit that calculates a shift amount between single-eye images photographed at a predetermined time difference; and a first shift amount calculating unit. Determining means for determining whether or not the first shift amount calculated in the step is within a predetermined range, and the second shift amount calculated by the second shift amount calculating means in accordance with the determination result of the determining means Output means, and the determination means determines the height of the subject from the imaging means based on the first shift amount, and the output means determines the second height according to the determination result of the determination means. Output the shift amount of as a shift amount with different attributes, , When the first shift amount is within a predetermined range is one that does not output the amount of movement.

According to a second aspect of the present invention, in the movement measuring device according to the first aspect, the compound eye image is further synthesized based on the single- eye images photographed by all the lenses of the imaging means and output. The image acquisition means and a selection means capable of appropriately switching between the composite image acquisition means and the movement amount calculation means are provided.

According to a third aspect of the present invention, there is provided a moving measurement program including an imaging optical system and a light receiving element, each of which includes two or more lenses arranged in a substantially planar shape and a lens array for forming a subject image on the image plane. A single-eye image is shifted pixel by pixel as a shift amount between single-eye images captured by an imaging means that is configured by a pixel array and picks up a subject image by an imaging optical system. However, the first shift amount calculation process for calculating the shift amount that minimizes the luminance difference between the single-eye images, and one single-eye image as the shift amount between the single-eye images captured at a predetermined time difference one pixel at a time While shifting, the second shift amount calculation process for calculating the shift amount that minimizes the luminance difference between the single-eye images and the first shift amount calculated by the first shift amount calculation process are within a predetermined range. Whether or not A determination process, in accordance with the determination result of the determining process, there is to be executed and an output process of outputting the second shift amount as the movement amount calculated by the second shift amount calculation processing, to the computer, determination In the process, the height of the subject from the imaging unit is determined based on the first shift amount, and in the output process, the second shift amount is set as a movement amount having a different attribute according to the determination result of the determination process. If it is output and the first shift amount falls within a predetermined range, the movement amount is not output .

  According to the present invention, the height of the subject (distance from the imaging device to the subject) and the subject's height are determined using a plurality of single-eye images taken simultaneously using a compound-eye imaging device and a single-eye image taken at a predetermined time difference. By simultaneously measuring the amount of movement over time, it is possible to realize a device that can be recognized as a user operation by extracting only movements such as the user tracing the device with a finger without being affected by the movement of the outside of the device or the device itself. be able to. Furthermore, such a compound eye imaging device can be realized in a small size and at low cost by using a microlens array.

  Hereinafter, the configuration of the present invention will be described in detail based on the embodiment shown in FIGS. The movement measuring apparatus according to the present invention includes an imaging optical system including a lens array in which two or more lenses are arranged in a substantially planar shape and forms a subject image on an image plane, and a pixel array of a light receiving element. , An imaging unit (compound eye imaging unit 10) that captures a subject image by the imaging optical system, a first single-eye image that is image data of an imaging surface area formed by the first lens, and a first single-eye The second single-eye image, which is image data of the imaging surface area imaged by the second lens at the same time as the image, and the first single-eye image and the first lens by either the first lens or the second lens. The movement amount calculation means for calculating the movement amount of the subject based on at least three single-eye images with the third single-eye image that is image data of the imaging surface area formed with a predetermined time difference from the two single-eye images. (First shift amount calculation unit 20, previous frame Recording unit 30, a second shift amount calculation unit 40, those having a height determining unit 50, the output setting unit 60) and.

1. First Embodiment FIG. 1 shows an example of a functional block diagram of a movement measuring apparatus according to a first embodiment. The movement measurement apparatus 1 of the present embodiment includes at least a compound eye imaging unit 10, a first shift amount calculation unit (first shift amount calculation unit) 20, a previous frame recording unit 30, a second shift amount calculation unit (second Shift amount calculation means) 40, height determination section (height determination section) 50, and output setting section (output means) 60, and the amount of movement of the subject based on the image data photographed by the compound eye imaging section 10. And the movement amount is transmitted as output information to a host device (not shown). Each block excluding the compound eye imaging unit 10 is configured by hardware resources such as a microprocessor, RAM, and ROM. In addition, the mobile measurement device 1 performs communication with a host device connected by a communication means (not shown). Hereinafter, each block will be described.

  The compound eye imaging unit 10 is a unit that captures and acquires a compound eye image, and includes, for example, a lens array 3, a light shielding member 4, an imaging element 5, and a cover glass 6 as shown in FIG.

  The lens array 3 is for forming a subject image, and two lenses (hereinafter referred to as a first lens 3a and a second lens 3b) are arranged in a plane substantially orthogonal to the lens optical axis. Is.

  The light blocking member 4 is for preventing crosstalk on the image plane of the light beam that has passed through the lenses 3a and 3b of the lens array 3 and suppressing noise such as ghost and flare.

  The image pickup device 5 is a means for picking up a compound eye image that is a set of images (hereinafter referred to as single-eye images) formed by the lenses 3a and 3b of the lens array 3, and specifically, for example, a light receiving device. A CMOS image sensor or the like in which 5a is two-dimensionally arranged is used. Here, the imaging device 5 is configured to incorporate a circuit that adjusts the gain of a photoelectric conversion signal by the light receiving device 5a or converts an analog signal to a digital signal, and outputs a captured image as digital image data. Hereinafter, the single-eye image formed by the lens 3a is referred to as a first single-eye image (single-eye image 1), and the single-eye image formed by the lens 3b is referred to as a second single-eye image (single-eye image 2). Say.

  Further, the outer surface of the compound-eye imaging unit 10 is covered with a cover glass 6 for protection, and the subject 2 is photographed through the cover glass 6.

  The first shift amount calculation unit 20 calculates a shift amount (hereinafter also referred to as a first shift amount) between two single-eye images (first single-eye image and second single-eye image) that are taken simultaneously. It is means to do. Here, it is known that parallax (compound eye parallax) due to the positional relationship between each lens and the subject exists between single-eye images. Hereinafter, a mechanism for generating parallax between single-eye images will be described with reference to FIG.

  FIG. 3 shows an image of the subject 2 existing at the height h of the subject from the center of each lens 3a, 3b on the imaging surface 7 by each lens 3a, 3b. The case where it appears as an eye image is shown.

  Here, the point p on the subject 2 appears in q1 and q2 on each single-eye image. The difference between the image position q1 on the first single-eye image and the image position q2 on the second single-eye image is called parallax. That is, when the relative positions of the lenses 3 a and 3 b and the imaging surface 7 are fixed, it can be said that the parallax changes according to the height of the subject 2.

  Therefore, the height h of the subject can be measured by obtaining the parallax between a plurality of single-eye images. The shift amount between single-eye images can be obtained by, for example, the processing described below.

  While shifting one individual image left and right (left and right in FIG. 3) pixel by pixel, the luminance difference between the individual images is calculated, and the shift amount that minimizes the luminance difference is obtained. Here, the luminance difference E is obtained by Equation 1, and the luminance difference E indicates that the smaller the value is, the more similar the images are. That is, in Equation 1, sx and sy when the luminance difference E is the minimum are parallaxes, and are the required shift amounts (hereinafter also referred to as shift amount 1 and first shift amount). The direction (symbol) and range of the parallax are determined by the lens arrangement and the height of the assumed subject. Therefore, the luminance difference with 10 different shift amounts per pixel within a predetermined range, for example, 0 to 10 pixels. To obtain the minimum value.

  Note that the calculation of the shift amount is not limited to the processing according to Equation 1. For example, various arithmetic processing such as obtaining a sum of squares instead of the absolute value in Equation 1 or calculating a normalized correlation coefficient may be performed. Is possible.

  The previous frame recording unit 30 is a means for recording only one single-eye image for one frame. In the present embodiment, the first frame image is recorded in the previous frame recording unit 30, but any one of the individual images may be recorded (that is, the individual image in FIG. 1). The eye image 1 and the single eye image 2 may be reversed). Hereinafter, in order to distinguish the first single-eye image of the previous frame recorded by the previous frame recording unit 30 from the first single-eye image to be processed by the first shift amount calculation unit 20 or the like. Also referred to as a third individual image.

  The second shift amount calculation unit 40 is means for calculating the shift amount between the third single-eye image and the first single-eye image recorded in the previous frame recording unit 30.

  The second shift amount calculation unit 40 calculates a luminance difference at each shift amount by a method similar to that of the first shift amount calculation unit 20 described above, and obtains a minimum value (see Equation 1 above). That is, in Equation 1, sx and sy when the luminance difference E is the minimum are the shift amounts to be obtained (hereinafter also referred to as shift amount 2 and second shift amount). Here, the second shift amount is caused not by the parallax but by the movement of the subject (finger), and the shift direction is generated in the two-dimensional direction, up, down, left and right. In the present embodiment, the minimum value is obtained by a total of nine shifts, one pixel vertically and horizontally from the center pixel. However, the present invention is not limited to this. Further, since the second shift amount is determined by the finger moving speed and the photographing interval, for example, by obtaining the minimum value in a wider range, it is possible to cope with a fast movement.

  In addition, it is also preferable to operate the arithmetic processing of the first shift amount calculation unit 20 and the second shift amount calculation unit 40 described above in a time division manner according to the performance of the calculation device.

  The height determination unit 50 determines the first shift amount calculated by the first shift amount calculation unit 20 based on a threshold value, and determines the height of the subject. This is because, as described above, the parallax increases as the height of the subject, that is, the distance between the subject and the image sensor becomes shorter. Here, the relationship between the threshold value set for the first shift amount and the determination results (case 0 to 3) is shown in FIG.

  In this embodiment, the first shift amount is compared with three threshold values (th0, th1, th2, where th0> th1> th2), and the four types shown in Table 1 (case 0 to 3) are determined. I am doing so.

  Here, since the surface of the device (compound eye imaging unit 10) is covered with the cover glass 6, the subject 2 does not enter within a certain distance. Therefore, for example, by setting the maximum threshold th0 to a height slightly corresponding to the outside of the cover glass 6, when the subject 2 is in contact with the apparatus surface, the first shift amount is greater than th0. It can be determined to be large (case 0).

  The output setting unit 60 determines the output information based on both the second shift amount calculated by the second shift amount calculation unit 40 and the determination result (case 0 to 3) of the height determination unit 50.

  Table 2 shows the output information corresponding to the determination result.

  The output information in the present embodiment is a digital signal composed of three numerical values: a movement amount type (1 or 2), a movement amount in the x direction, and a movement amount in the y direction. The movement amount type 1 indicates the movement amount in the first and second directions, and the movement amount type 2 indicates the movement amount in the third and fourth directions.

  As shown in Table 2, when the determination result is case 0, the movement amount in the x and y directions (shift amount 2) is output to the host device as the movement amount in the first and second directions. Here, the amount of movement in the first and second directions can be used as the amount of movement of the cursor on the display screen of the host device, as with a normal mouse.

  That is, when the subject (finger) is in contact with the surface of the apparatus, the movement amount is the cursor movement amount.

  When the determination result is case 1, the movement amount is not output regardless of the image shift amount. As a result, the input can be stopped immediately when the user lifts his / her finger slightly from the surface of the device. Further, when the user's finger is unintentionally slightly out of the range of case 0 or case 2, it is possible to prevent an erroneous operation such as moving the cursor by mistake during a scroll operation, for example.

  When the determination result is case 2, the movement amount in the x and y directions (shift amount 2) is output to the host device as the movement amount in the third and fourth directions. Here, the amount of movement in the third and fourth directions is used as a scroll amount similar to that of a commercially available mouse with a scroll wheel, or by appropriately thresholding the amount of movement, and so on. Can be used.

  When the determination result is “case 3”, the movement amount is not output regardless of the shift amount of the image as in the case 1. Thereby, when the user does not exist in the vicinity of the apparatus, it is possible to prevent an unintended input from being generated by the user due to a distant subject or the movement or rotation of the apparatus itself.

  Next, the details of the operation will be described by taking as an example the case where the above-described movement measuring apparatus 1 is applied to a portable terminal device that is used as a pointing device such as a touch pad.

  Here, it is assumed that three height thresholds (a> b> c) are set for the first shift amount, and the host apparatus handles outputs in each region as shown in Table 3.

  (When not in use) If the finger or the like is not brought within a certain distance (c> shift amount 1) from the device, the device does not react even if the device is pointed toward anything or the device itself is moved. This is because the amount of movement is not output when the parallax is equal to or less than the threshold value c regardless of the content of the image data output from the compound-eye imaging unit 10 corresponding to the above-described case3.

  (At the time of contact) When the surface of the apparatus is stroked with a finger (shift amount 1> a), the cursor moves in the direction of finger movement. This corresponds to the case 0 described above.

  (Finger return operation) When the finger is moved away from the device by a certain distance (a> shift amount 1> b), the device does not react. This is because it corresponds to the case 1 described above, and in this case, it can be determined that the finger returning operation as shown in FIG. 5 is being performed. By doing so, the user can repeatedly move the cursor in one direction while lifting and returning the finger.

  (Scrolling operation) When the finger or hand is moved up, down, left or right with the finger or hand held away from the device to some extent (b> shift amount 1> c), the screen display content of the host device scrolls up, down, left or right. . Corresponds to case 2 of the state j rainy season.

  In the present embodiment, the determination of the first shift amount is classified into four cases using three threshold values. However, the present invention is not limited to this, and various modifications may be made according to user interface requirements and the like. Is possible.

  For example, the movement amount may be output only when the height is below a certain level. When multiple types of input such as cursor movement and scrolling are not necessary, the possibility of erroneous input can be reduced by setting one threshold and dividing the case into two cases.

  Further, for example, the movement amount may be output only in a specific height range (for example, 2 cm to 5 cm). In this case, for example, it is possible to realize an operation that does not react even when a finger touches it for holding the housing, and reacts only when a hand is held away from the device by a certain distance or more.

  As described above, according to the movement measuring apparatus 1, it is possible to limit the height at which the movement amount measurement functions effectively by a simple arithmetic circuit called threshold processing of the shift amount. In addition, by using the motion detection function only at a height enough to touch the device, it is possible to provide the user with a feeling of use similar to that of a contact sensor such as a touch pad. Furthermore, since information of a plurality of attributes can be input depending on the height of the subject, information that can be input such as cursor movement and scrolling can be increased, and a more sophisticated pointing device can be realized. Furthermore, erroneous operations can be reduced by setting a no-reaction area between the height ranges effective as input operations.

2. Second Embodiment Next, a second embodiment of the present invention will be described. Note that the description of the same points as in the first embodiment will be omitted.

  FIG. 6 shows a functional block diagram of the movement measuring apparatus 1 in the second embodiment. The movement measurement apparatus 1 according to the second embodiment includes an imaging device 90 and a host device 100. The imaging device 90 of this embodiment has a compound eye imaging unit 10 described later, and is connected to the host device 100 via a communication unit (not shown). The host apparatus 100 includes the above-described movement amount calculation means (the first shift amount calculation unit 20, the previous frame recording unit 30, the second shift amount calculation unit 40, the height determination unit 50, and the output setting unit 60), and will be described later. A composite image acquisition unit (composite image acquisition unit) 70 and a selection unit (selection unit) 80 are included. As the host device 100, for example, a personal computer including an arithmetic device, a main storage device, an auxiliary storage device, or the like can be used.

  The movement measurement apparatus 1 of the first embodiment executes each calculation such as shift amount calculation in the movement measurement apparatus 1, and the movement amount type and movement amount obtained as a result of the calculation as output information for the host device. However, the movement measurement apparatus 1 according to the second embodiment only acquires an image in the imaging apparatus 90, transfers the image data as it is to the host apparatus 100, and performs other calculations for the host apparatus. It is configured to execute within 100. Such a configuration is suitable when the host device 100 has high computing performance like a personal computer.

  An example of the compound eye imaging unit 10 of the present embodiment is shown in FIG. The compound eye imaging unit 10 has a lens array 3 composed of a large number of lenses, captures a compound eye image of a subject image formed by the lens array 3, and outputs it as digital image data.

  In addition, the movement measuring apparatus 1 of the second embodiment not only has a mode (hereinafter referred to as a pointing mode) in which the above-described movement amount calculation means operates as a pointing device, but also uses a composite image acquisition unit 70 to perform a vein authentication camera. And a mode that operates as a general camera for photographing a person or a person (hereinafter referred to as a camera mode).

  The composite image acquisition unit 70 synthesizes a compound eye image based on image data of a single-eye image captured by all the lenses included in the compound eye imaging unit 10. The processing by the composite image acquisition unit 70 may be a known or new method, and is not particularly limited. For example, the processing method described in Japanese Patent Application Laid-Open No. 2008-97327 is applied. be able to.

  The selection unit 80 can switch between the pointing mode and the camera mode. That is, the compound eye imaging unit 10 of the present embodiment can output an image of only a specific area on the imaging surface in accordance with an instruction from the host device 100.

  FIG. 8 shows a schematic diagram of the imaging areas in the pointing mode and the camera mode. A small square in FIG. 8 corresponds to a single-eye image region (consisting of a plurality of pixels), and the whole indicates the entire imaging region of the imaging device. In the camera mode, for example, the entire image pickup device, for example, in the pointing mode, for example, image data of only a region corresponding to two single-eye images near the center is captured and transmitted to the host computer. Note that the selection unit 80 is possible, for example, by setting the mode of the CMOS image sensor.

  Here, the time taken for imaging by the image sensor is dominant in the image data transfer processing time, and the transfer time is proportional to the number of pixels. Therefore, by enabling switching of the size of the imaging area as in this embodiment, a high-resolution image can be acquired using all pixels in the camera mode, and only a small number of pixels can be acquired at high speed in the pointing mode. In addition, it can take images (many shots per unit time) and follow the fast movement of the subject.

  By providing both a pointing mode and a camera mode in this way, a single sensor has both high-speed (low resolution) and high-resolution (low speed) imaging functions. The camera can be shared with ordinary cameras for taking pictures of people and people, and it is possible to realize cost reduction and downsizing rather than mounting individual sensors.

3. Next, the movement measurement program according to the present invention will be described. The movement measurement program is stored in, for example, an auxiliary storage device of the host device 100, and the host device 100 functions as the movement measurement device 1 when the program is read and executed by the CPU. The movement measurement program preferably operates as a device driver in an operating system that manages the resources of the entire host device 100.

  In the present embodiment, it operates as two devices, a pointing device and a camera, and provides these functions to the system. Further, by functioning as a device driver, the pointing and image input functions can be used in various applications executed on a computer.

  Further, in the present embodiment, the movement measurement program operates in the pointing mode by default, temporarily shifts to the camera mode in response to a user instruction such as pressing the camera button, and the image acquisition that is the operation in the camera mode is completed. , Automatically return to pointing mode. The operation in both modes will be described with reference to the flowcharts of FIGS.

3-1. Pointing Mode An example of processing (movement amount calculation processing) executed by the movement measurement program in the pointing mode is shown in the flowchart of FIG. The movement amount calculation process is periodically repeated using a timer interruption.

  First, image data is input from the compound eye imaging unit 10 (S11). Specifically, image data for two single-eye images is input.

  Next, shift amount calculation 1 (S12: first shift amount calculation process) is executed, and shift amount 1 (simultaneous parallax) is calculated. The processing content is the same as that by the first shift amount calculation unit 20.

  Next, height determination (S13: determination processing) is performed. The processing content is the same as that by the height determination unit 50 described above. That is, in the above threshold setting example, when the determination results are cases 0 and 2 (Table 1) (S13: valid), the process proceeds to the next shift amount calculation 2 (S14), and when the determination results are cases 1 and 3 (S13). : Invalid) shifts to single-eye image recording (S16).

  Next, shift amount calculation 2 (S14: second shift amount calculation process) is executed to calculate shift amount 2 (subject movement amount). The processing content is the same as that by the second shift amount calculation unit 40 described above.

  Next, movement amount output (S15: output processing) is executed. The processing content is the same as that by the output setting unit 50 described above. That is, the movement amount type, the x-direction movement amount, and the y-direction movement amount obtained based on the height determination result and the shift amount 2 are transmitted to the host device.

  Next, single-eye image recording (S16) is executed. The processing content is the same as that by the previous frame recording unit 30 described above. That is, the single image 1 is recorded in the memory for calculation of the next frame. The above is the process executed by the movement measurement program in the pointing mode, and the process is repeated at regular intervals.

3-2. Camera Mode An example of processing (composite image acquisition processing) executed by the movement measurement program in camera mode is shown in the flowchart of FIG. In the synthesized image acquisition process, a compound eye image is synthesized and output as a single piece of image data.

  First, image data is input from the compound eye imaging unit 10 (S21). In the camera mode, all single-eye images are input.

  Next, reference image setting (S22) is executed. That is, a region of each single-eye image is extracted, and a single-eye image serving as a reference for parallax detection and reconstruction is set.

  Next, parallax detection (S23) is performed. That is, the parallax of each single-eye image with respect to the reference single-eye image is detected.

  Next, the reconstruction calculation (S24) is executed in consideration of the parallax from each single-eye image data. That is, since each single-eye image has a parallax corresponding to the height of the object, the position on the single-eye image corresponding to each pixel position of the virtual composite image is obtained using the detected parallax, The pixel value of the position is obtained by interpolation and used as the pixel value of the composite image. In addition, when the corresponding position exists in the area | region of a several single-eye image, what is necessary is just to use the average of the pixel value calculated | required from each single-eye image as the pixel value of a synthesized image.

  Next, the reconstructed composite image is output as an image acquired by the camera (S25). By this processing, information of each individual eye image can be integrated and a composite image with high resolution can be obtained. Such a composite image can be used for various purposes such as a general camera or a vein authentication image input unit similar to the conventional art.

  The above-described embodiment is a preferred embodiment of the present invention, but is not limited thereto, and various modifications can be made without departing from the gist of the present invention.

It is an example of the functional block diagram of the movement measuring device concerning the present invention. It is an example of the schematic block diagram of a compound eye imaging part. It is a figure for demonstrating the parallax which arises between single-eye images. It is a figure for demonstrating the threshold value set to a shift amount. It is a figure for demonstrating finger return operation. It is an example of the functional block diagram of the movement measurement apparatus of 2nd Embodiment. It is an example of the schematic block diagram of the compound eye imaging part of 2nd Embodiment. It is a figure for demonstrating the imaging region for every operation mode. It is a flowchart which shows an example of the process which a movement measurement program performs in pointing mode. It is a flowchart which shows an example of the process which a movement measurement program performs in camera mode.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Movement measuring device 2 Subject 3 Lens array 4 Light-shielding member 5 Imaging element 6 Cover glass 10 Compound eye imaging unit 20 First shift amount calculation unit 30 Previous frame recording unit 40 Second shift amount calculation unit 50 Height determination unit 60 Output Setting unit 70 Composite image acquisition unit 80 Selection unit 90 Imaging device 100 Host computer

Claims (3)

Two or more lenses are arranged in a substantially planar shape, and are composed of an imaging optical system composed of a lens array for forming a subject image on the image plane and a pixel array of a light receiving element, and the subject image by the imaging optical system Imaging means for imaging
A first single-eye image that is image data of the imaging surface region imaged by the first lens, and image data of the imaging surface region that is imaged by the second lens simultaneously with the first single-eye image. Imaging in which a second single-eye image and the first single-eye image and the second single-eye image are formed with a predetermined time difference by either the first lens or the second lens. Movement amount calculation means for calculating the movement amount of the subject based on at least three single-eye images with the third single-eye image that is image data of the surface area ;
For two ommatidium images of the previous SL three ommatidium images, one with the ommatidium image shifted by one pixel, the movement amount calculating means for calculating a movement amount of the luminance difference is smallest between ommatidium images Is
First shift amount calculating means for calculating a shift amount between single-eye images taken at the same time;
A second shift amount calculating means for calculating a shift amount between single-eye images photographed at a predetermined time difference;
Determining means for determining whether or not the first shift amount calculated by the first shift amount calculating means is within a predetermined range;
Output means for outputting, as a movement amount, the second shift amount calculated by the second shift amount calculation means according to the determination result of the determination means,
The determination means determines the height of the subject from the imaging means based on the first shift amount,
The output means outputs the second shift amount as a movement amount having a different attribute according to the determination result of the determination means, and when the first shift amount falls within a predetermined range, A movement measuring apparatus characterized by not outputting the movement amount . Furthermore, a composite image acquisition unit that combines and outputs a compound eye image based on single-eye images captured by all the lenses included in the imaging unit;
The movement measuring apparatus according to claim 1, further comprising: a selecting unit that can appropriately switch between the synthetic image acquiring unit and the movement amount calculating unit. Two or more lenses are arranged in a substantially planar shape, and are composed of an imaging optical system composed of a lens array for forming a subject image on the image plane and a pixel array of a light receiving element, and the subject image by the imaging optical system For single-eye images taken by the imaging means for imaging
A first shift amount calculation process for calculating a shift amount that minimizes the luminance difference between the single-eye images while shifting one single-eye image pixel by pixel as a shift amount between the single-eye images captured simultaneously;
Second shift amount calculation processing for calculating a shift amount that minimizes a luminance difference between single-eye images while shifting one single-eye image pixel by pixel as a shift amount between single-eye images captured at a predetermined time difference When,
A determination process for determining whether or not the first shift amount calculated in the first shift amount calculation process is within a predetermined range;
In response to said determination process of the determination result, there is to be executed and an output process of outputting the second shift amount calculated by the second shift amount calculation processing as the movement amount to the computer,
In the determination process, the height of the subject from the imaging unit is determined based on the first shift amount,
In the output process, according to the determination result of the determination process, the second shift amount is output as a movement amount having different attributes, and when the first shift amount falls within a predetermined range, A movement measurement program that does not output the movement amount .

Images