RU2514086C2 - Device for organisation of 3d user interface - Google Patents

Abstract

FIELD: measurement equipment.

SUBSTANCE: invention relates to equipment of optic-electronic measurement systems and devices of information input into calculation devices and systems. The device comprises optically coupled first and second emitters, "surface of shadow location", a television camera and a calculation device, besides, the emitters generate light beams with different spectral maxima γ₁ and γ₂, and the television camera has appropriate separate spectral channels for reception of these emissions, at the same time coordinates of half-shadows generated from a finger of a man operator located on the way of these light beams, are input into the calculation device, which by their values calculates its spatial position.

EFFECT: increased efficiency of a device.

2 cl, 4 dwg

Description

The invention relates to techniques for optoelectronic measuring systems and devices for inputting information into computing devices and systems. It can be used in information and reference and payment terminals to organize new types of interaction with their software, to facilitate design during three-dimensional manipulation of objects in computer-aided design (CAD) systems and other interactive and passive systems.

A number of devices of similar application are known. These are, for example, 3Dconnexion devices [1, 2], which are optical-mechanical systems of three-dimensional positioning, with a ball or joystick. They are designed to work in CAD or in applications that require control of the movement of something in the virtual space (for example, Google Earth, 3ds Max, Autodesk Inventor, Compass-3D, etc.). The obvious disadvantage of this system is the need for physical contact of the user with the joystick.

Of non-contact optical-electronic systems of a similar purpose, a device proposed by Apple is known [3]. This device contains a laser, which with the help of 2 movable, optically conjugated mirrors scans the working area in which the fingers of a human operator can be located. The optical radiation reflected from these fingers is incident on a photodetector, the output of which is connected to the input of the phase detector. The output signal of the phase detector is proportional to the phase shift between the signal modulating the laser beam in amplitude and the signal from the output of the photodetector. Thus, as a result of scanning the working area, you can get a set of distances to the highlighted points on the fingers of a human operator. This information is further used to form finger images and is displayed on the monitor screen. The computing device determines the moments of coincidence of the position of the fingers in the working area with the coordinates of virtual objects with which the human operator interacts. The disadvantage of this device is its complexity associated with the need to measure small phase shifts, and the presence of mechanical sweep.

Closest to the technical implementation with the claimed is "Device for three-dimensional manipulation" [4]. This device contains the first and second emitters, a surface diffusely scattering the radiation incident on it, and optically coupled to these emitters and hereinafter referred to as the “shadow location surface”, onto which the light fluxes from the first and second emitters alternately fall. In addition, a television camera is included in the device, whose field of view is the “shadow location surface” optically conjugated to it. The working area of the device is formed by the intersection of the propagation cones of the radiation flux incident on the "surface of the shadow location" from the first and second emitters. At the same time, when the operator’s finger is inserted into the work area, two shadows are formed alternately, according to the signals of the computing device controlling the inclusion of emitters, on the “shadow location surface”. Their images using a television camera are entered into a computing device, which determines the spatial coordinates of the finger and its tilt angles. This information is then entered into the computer to which this device is connected.

The disadvantage of this device is the reduced performance associated with the need for alternating the formation of shadows from two radiation sources and synchronous with this process of entering into the computing device the values of the video signals separately when the first and second emitters are turned on.

The aim of the invention is to increase speed and simplify the device.

To do this, in the known device, which contains the first and second emitters, an “shadow location surface” optically conjugated to them, made in the form of a television camera surface diffusely scattering incident radiation, the field of view of which is “a shadow location surface”, and a computing device for an input video signal is output from the television camera, the above-mentioned first and second radiators are arranged so that they form light fluxes ψ ₁ and ψ ₂ with different spectral maxi umami γ ₁ and γ _2, and the TV camera must be at least dvuhspektralnoy (color), and the value of γ ₁ shall belong to one and another range γ ₂ of the spectral sensitivity of the television camera. When an operator’s finger appears on the “shadow location surface” in the working area, two penumbraes are formed, the first associated with the absence of light flux from the first emitter, the second with the absence of light flux from the second emitter. Images of the “surface of the shadow location” are entered into the computing device, in the two spectral ranges of the television camera and then the boundaries of both penumbraes are found, which determine the spatial coordinates of the operator’s finger.

Figure 1 shows the functional diagram of the proposed device,

where 1, 2 are the first and second emitters forming the light flux ψ ₁ and ψ ₂ with wavelengths γ ₁ and γ _2, respectively;

3 - a television camera with a viewing angle f;

4 - computing device;

5 - “surface of the shadow location”;

The device operates as follows.

The emitters 1 and 2 represent, in a sufficient approximation, point sources of radiation, for example, powerful LEDs of the red and green color ranges, with their optical axes passing through the “surface of the shadow location”. In order for their radiation to be used as efficiently as possible, i.e. not go beyond the "surface of the shadow location", they can be equipped with focusing optics. These emitters create two radiation fluxes ψ ₁ and ψ ₂ with a maximum in the spectra of γ ₁ and γ ₂ , in particular, for red and green LEDs, their values are 620-630 nm [5] and 520-530 nm [6], respectively.

The light flux φ ₁ (partial reflection of the flux ψ ₁ ) and φ ₂ (partial reflection of the flux ψ ₂ ) reflected from the “surface of the shadow location” is recorded by a television camera 3, which is a standard color image camera with RGB output signals (red, green, and blue luminance components) in analog or digital form and which then go to the input of the computing device 4. The working area of the device is formed by the intersection of the propagation cones of the streams ψ ₁ and ψ ₂ falling on the surface of the “shadow location” ii ”5. In this case, in the case of the appearance of the finger of a human operator on the“ surface of the shadow location ”, two partial shadows are formed.

The foregoing is illustrated by the optical-geometric scheme depicted in figure 2,

Where 6, 7 are partial shadows formed on the “surface of the shadow location”.

8 - a lens of a television camera;

9 - photodetector matrix as part of a television camera 3;

10, 11 - image penumbra 6 and 7, respectively;

12 - a brush of a human operator.

The image of the "surface of the shadow location" on the photodetector 9 has the following features:

- in region 10 there are no spectral components of the light flux generated by the emitter 1 and there are spectral components of the light flux generated by the emitter 2;

- in region 11 there are no spectral components of the light flux generated by the emitter 2 and there are spectral components of the light flux generated by the emitter 1;

- the rest of the image "surface of the shadow location" there are light fluxes generated by both the emitter 1 and the emitter 2.

Thus, if the 1st emitter is a red LED, and the second emitter is a green LED, then in the video signal of the television camera 3 areas 10 and 11 can be detected based on the following rule:

$$\left(\mathrm{one}\right)e\mathrm{from}l\mathrm{and}:\{\begin{array}{l}{U}_{noise}^R<R\left(t\right)<U_{signal}^R,t\mathrm{about}\text{<figure-callout id="10" label="area" filenames="00000013.png" state="{{state}}">area</figure-callout> 10 detected;}\\ U_{noise}^G<G\left(t\right)<U_{signal}^G,t\mathrm{about}\text{<figure-callout id="11" label="area" filenames="00000013.png" state="{{state}}">area</figure-callout> 11 detected}\text{.}\end{array}$$

where R (t) and G (t) are the current values of the color R and G components of the video signal;

, $$U_{noise}^G$$

- пороговый уровень шумовых составляющих видеосигнала (пиковые значения паразитных засветок) в спектральных диапазонах R и G изображения «поверхности теневой локации»; $$U_{noise}^R$$

, $$U_{noise}^G$$

- the threshold level of noise components of the video signal (peak values of spurious illumination) in the spectral ranges R and G of the image "surface of the shadow location";

, $$U_{signal}^G$$

- пороговый уровень полезных значений видеосигнала (минимальные значения полезного сигнала) в спектральных диапазонах R и G изображения «поверхности теневой локации»; $$U_{signal}^R$$

, $$U_{signal}^G$$

- the threshold level of the useful values of the video signal (the minimum values of the useful signal) in the spectral ranges R and G of the image "surface of the shadow location";

Calculation formulas that allow you to determine the spatial coordinates of the finger of a human operator can be derived based on the geometric diagram shown in figure 3,

where I ₁ (x _I1 , y _I1 , z _I1 ) - coordinates of the optical center of the first emitter;

I ₂ (x _I2 , y _I2 , z _I2 ) - coordinates of the optical center of the second emitter;

5 - “surface of the shadow location”;

6.7 - partial shade;

a, b - characteristic points belonging to the surface of the finger of a human operator (the tip of the finger and any of the points spaced from the tip of the finger, respectively);

0XYZ - basic coordinate system;

a ₁ (x _a1 , y _a1 , 0) - the coordinates of the projection of point a on the 0XY plane, and the projecting line leaves the optical center of the first emitter;

a ₂ (x _a2 , y _a2 , 0) - the coordinates of the projection of point a on the plane 0XY, and the projecting line leaves the optical center of the second emitter;

b ₁ (x _b1 , y _b1 , 0) - the coordinates of the projection of point b on the 0XY plane, and the projecting line leaves the optical center of the first emitter;

b ₂ (x _b2 , y _b2 , 0) - the coordinates of the projection of point b on the 0XY plane, and the projecting line leaves the optical center of the second emitter.

The coordinates of the points a ₁ , a ₂ , b ₁ , b ₂ can be obtained from the coordinates of their images on the surface of the photodetector matrix 9, by multiplying by the magnitude of the linear increase in the optical system [7].

The equation of a line passing through 2 given points (X ₁ , Y ₁ , Z ₁ ) and (X ₂ , Y ₂ , Z ₂ ) has the form [8]:

$$\left(2\right)\begin{array}{cc}& \end{array}\frac{X-X_{\mathrm{one}}}{X_2-X_{\mathrm{one}}}=\frac{Y-Y_{\mathrm{one}}}{Y_2-Y_{\mathrm{one}}}=\frac{Z-Z_{\mathrm{one}}}{Z_2-Z_{\mathrm{one}}}.$$

And can be represented as a system:

$$\left(3\right)\begin{array}{cc}& \end{array}\{\begin{array}{l}\frac{X-X_{\mathrm{one}}}{X_2-X_{\mathrm{one}}}=\frac{Y-Y_{\mathrm{one}}}{Y_2-Y_{\mathrm{one}}},\\ \frac{Y-Y_{\mathrm{one}}}{Y_2-Y_{\mathrm{one}}}=\frac{Z-Z_{\mathrm{one}}}{Z_2-Z_{\mathrm{one}}}.\end{array}$$

Passing to the coordinate values used in the text I ₁ , I ₂ , a ₁ , a ₂ , b ₁ , b ₂ , we obtain the following (overdetermined) systems of equations.

For point a:

$$\left(\mathrm{four}\right)\begin{array}{cc}& \end{array}\{\begin{array}{l}\frac{x_a-x_{I\mathrm{one}}}{x_{a\mathrm{one}}-x_{I\mathrm{one}}}=\frac{y_a-y_{I\mathrm{one}}}{y_{a\mathrm{one}}-y_{I\mathrm{one}}},\\ \frac{y_a-y_{I\mathrm{one}}}{y_{a\mathrm{one}}-y_{I\mathrm{one}}}=\frac{z_a-z_{I\mathrm{one}}}{z_{a\mathrm{one}}-z_{I\mathrm{one}}},\\ \frac{x_a-x_{I2}}{x_{a2}-x_{I2}}=\frac{y_a-y_{I2}}{y_{a2}-{\mathrm{<figure-callout\; id="2"\; label="y\; I"\; filenames="00000013.png,00000015.png"\; state="\{\{state\}\}">y\; </figure-callout>}}_I},\\ \frac{y_a-y_{I2}}{y_{a2}-{\mathrm{<figure-callout\; id="2"\; label="y\; I"\; filenames="00000013.png,00000015.png"\; state="\{\{state\}\}">y\; </figure-callout>}}_I}=\frac{z_a-z_{I2}}{z_{a2}-{\mathrm{<figure-callout\; id="2"\; label="z\; I"\; filenames="00000013.png,00000015.png"\; state="\{\{state\}\}">z\; </figure-callout>}}_I}.\end{array}$$

For point b:

$$\left(5\right)\begin{array}{cc}& \end{array}\{\begin{array}{l}\frac{x_b-x_{I\mathrm{one}}}{x_{b\mathrm{one}}-x_{I\mathrm{one}}}=\frac{y_b-y_{I\mathrm{one}}}{y_{b\mathrm{one}}-y_{I\mathrm{one}}},\\ \frac{y_b-y_{I\mathrm{one}}}{y_{b\mathrm{one}}-y_{I\mathrm{one}}}=\frac{z_b-z_{I\mathrm{one}}}{z_{b\mathrm{one}}-z_{I\mathrm{one}}},\\ \frac{x_b-x_{I2}}{x_{b2}-x_{I2}}=\frac{y_b-y_{I2}}{y_{b2}-{\mathrm{<figure-callout\; id="2"\; label="y\; I"\; filenames="00000013.png,00000015.png"\; state="\{\{state\}\}">y\; </figure-callout>}}_I},\\ \frac{y_b-y_{I2}}{y_{b2}-{\mathrm{<figure-callout\; id="2"\; label="y\; I"\; filenames="00000013.png,00000015.png"\; state="\{\{state\}\}">y\; </figure-callout>}}_I}=\frac{z_b-z_{I2}}{z_{b2}-{\mathrm{<figure-callout\; id="2"\; label="z\; I"\; filenames="00000013.png,00000015.png"\; state="\{\{state\}\}">z\; </figure-callout>}}_I}.\end{array}$$

The solutions of systems 4 and 5 with respect to unknown x _a , y _a , z _a and x _b , y _b , z _b give the coordinates of points a and b in the OXYZ system.

The direction cosines of the vector passing through the points a and b can be obtained from the formulas [6]:

$$\left(6\right)\begin{array}{cc}& \end{array}\{\begin{array}{c}cos\alpha =\frac{(x_a-x_b)}{|\; |\; |a|\; |\; |},\\ cos\beta =\frac{(y_a-y_b)}{|\; |\; |a|\; |\; |},\\ cos\gamma =\frac{(z_a-z_b)}{|\; |\; |a|\; |\; |},\end{array}$$

Where $$|\; |\; |a|\; |\; |=\sqrt{{(x_a-x_b)}^2+{(y_a-y_b)}^2+{(z_a-z_b)}^2}.$$

In the second embodiment of this device, the "shadow location surface" is an optically transparent, diffusely scattering radiation material (for example, frosted glass). In this case, the television camera 3 is located under the "surface of the shadow location", and is optically paired with this surface, as shown in figure 4, and registers the radiation incident on it.

The computing device can be performed on the basis of a productive processor oriented to stream processing of video information, for example, of the TMS320DM8148 type based on DaVinci technology [9]. In addition, in the case of using a television camera with a standard USB interface, it can be connected directly to a personal computer, on which spatial position calculations will be implemented.

Literature

1. http://www.3dconnexion.com/

2. US Patent No. 8063883 B2 of November 22, 2011

3. US Patent No. 8018579 B 1 of September 13, 2011.

4. RF patent No. 2362216 dated 05/12/2008 (prototype).

5. http://www.transistor.ru/pdf/arlight/powerleds/ARPL Emitter 3W-RED-140-RER3E.pdf.

6.http: //www.transistor.ru/pdf/arlight/powerleds/ARPL_Emitter-3 W-GREEN-140-GNH3E.pdf.

7. Reference designer of optical-mechanical devices. Under the total. ed. V.A. Panova. Leningrad, Engineering, 1980

8. M. Ya. Vygodsky. Handbook of Higher Mathematics. Moscow, ACT, 2006

9.http: //www.ti.com/lsds/ti/dsp/platform/davinci/device.page.

Claims (2)

1. A device for organizing a three-dimensional user interface containing optically coupled first and second emitters, a television camera and a “shadow location surface”, as well as a computing device, the input of which is connected to the output of the television camera, characterized in that the first and second emitters form light fluxes incident on the "surface location of the shadow" with different spectral maxima of γ ₁ and γ ₂ and the television camera has a respective individual spectral channels for receiving these zlucheny, wherein the image penumbras on the "surface location of the shadow" produced human operator finger located in the path of both light fluxes is detected with a television camera and input to the computing device on which coordinate values of the penumbra calculates the spatial position of the finger. 2. The device according to claim 1, characterized in that the "surface of the shadow location" is made optically transparent, diffusely scattering radiation, while the first and second emitters are located one at a time, and the television camera is on the other side of this surface.

Images