[go: up one dir, main page]

CN104035552B - Three-dimensional manipulating method and three-dimensional manipulating device - Google Patents

Three-dimensional manipulating method and three-dimensional manipulating device Download PDF

Info

Publication number
CN104035552B
CN104035552B CN201310075672.4A CN201310075672A CN104035552B CN 104035552 B CN104035552 B CN 104035552B CN 201310075672 A CN201310075672 A CN 201310075672A CN 104035552 B CN104035552 B CN 104035552B
Authority
CN
China
Prior art keywords
user
operation behavior
dimensional
coordinate
finger
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related
Application number
CN201310075672.4A
Other languages
Chinese (zh)
Other versions
CN104035552A (en
Inventor
李勇
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Samsung Guangzhou Mobile R&D Center
Samsung Electronics Co Ltd
Original Assignee
Samsung Guangzhou Mobile R&D Center
Samsung Electronics Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Samsung Guangzhou Mobile R&D Center, Samsung Electronics Co Ltd filed Critical Samsung Guangzhou Mobile R&D Center
Priority to CN201310075672.4A priority Critical patent/CN104035552B/en
Publication of CN104035552A publication Critical patent/CN104035552A/en
Application granted granted Critical
Publication of CN104035552B publication Critical patent/CN104035552B/en
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Landscapes

  • Position Input By Displaying (AREA)

Abstract

本发明提供一种三维操作方法和三维操作装置,本发明的三维操作方法包含步骤:基于分别设置在屏幕周边的距离传感器和角度传感器测量的信息分析和判断用户的操作行为;根据用户的操作行为的判断结果而判定操作类型,量化操作参数;根据操作类型和操作参数,在用户界面执行三维操作。

The present invention provides a three-dimensional operation method and a three-dimensional operation device. The three-dimensional operation method of the present invention includes the steps of: analyzing and judging the user's operation behavior based on the information measured by the distance sensor and the angle sensor respectively arranged around the screen; Determine the type of operation based on the result of the judgment and quantify the operating parameters; perform three-dimensional operations on the user interface according to the type of operation and the operating parameters.

Description

三维操作方法和三维操作装置Three-dimensional operation method and three-dimensional operation device

技术领域technical field

本发明涉及具有三维显示器的移动终端领域,尤其涉及可以在三维显示器上实现三维操作的方法和三维操作装置。The invention relates to the field of mobile terminals with a three-dimensional display, in particular to a method and a three-dimensional operation device capable of realizing three-dimensional operations on the three-dimensional display.

背景技术Background technique

随着三维(3D)技术的不断发展和应用,手机上也随之出现很多3D效果的游戏及应用,屏幕可以展示3D效果图,使用户切实体验立体的显示效果。With the continuous development and application of three-dimensional (3D) technology, many games and applications with 3D effects appear on mobile phones. The screen can display 3D renderings, so that users can truly experience the three-dimensional display effect.

但现有的触摸屏都是需要物体直接触摸到屏幕上,才能实现某些操作。比如,需要手指触碰到屏幕上,在屏幕上完成一段滑动轨迹,根据滑动轨迹判断用户的行为,进而实现对手机的操作及控制。However, existing touch screens require objects to be directly touched on the screen to achieve certain operations. For example, it is necessary to touch the screen with a finger, complete a sliding track on the screen, judge the user's behavior according to the sliding track, and then realize the operation and control of the mobile phone.

由于触摸屏的局限性,现有技术只适合2D模式下的操作及控制,无法解决3D空间中的展示及控制。换句话说,目前的这种单纯触摸方式的平面触屏无法完成对立体效果展示的控制功能。Due to the limitations of the touch screen, the existing technology is only suitable for operation and control in 2D mode, and cannot solve the display and control in 3D space. In other words, the current flat touch screen with a simple touch method cannot complete the control function for displaying a three-dimensional effect.

发明内容Contents of the invention

本发明鉴于触摸屏上无法完成三维操作而提出,其目的在于提供一种即使不触碰屏幕,也能完成对三维显示器的三维操作的三维操作方法和三维操作装置。The present invention is proposed in view of the inability to complete three-dimensional operations on a touch screen, and aims to provide a three-dimensional operation method and a three-dimensional operation device that can complete three-dimensional operations on a three-dimensional display without touching the screen.

本发明的三维操作方法,包含步骤:基于检测部件所测量的距离信息和角度信息分析和判断用户的操作行为;根据用户的操作行为的判断结果而判定操作类型,量化操作参数;根据操作类型和操作参数,在用户界面执行三维操作。The three-dimensional operation method of the present invention includes the steps of: analyzing and judging the user's operation behavior based on the distance information and angle information measured by the detection component; judging the operation type according to the judgment result of the user's operation behavior, and quantifying the operation parameters; according to the operation type and Manipulate parameters to perform 3D operations in the user interface.

并且,所述检测部件包括三个距离传感器和角度传感器。Also, the detection component includes three distance sensors and an angle sensor.

并且,所述检测部件进一步包括一个距离传感器和角度传感器作为校验传感器。Moreover, the detection component further includes a distance sensor and an angle sensor as calibration sensors.

并且,所述检测部件包括雷达。Also, the detection component includes radar.

并且,所述操作参数包含运动速度、运动方向以及与屏幕所成的角度。In addition, the operating parameters include movement speed, movement direction, and an angle with the screen.

并且,所述用户的操作行为分为单向运动、多向运动和转动。In addition, the user's operation behavior is divided into one-way movement, multi-directional movement and rotation.

并且,所述单向运动的用户的操作行为通过如下步骤进行判断:判断手指上各点的坐标在某一时刻的变化趋势是否相同;判断三个坐标平面中是否至少有一个平面上的坐标投影为线形;将用户的操作行为封装成类。In addition, the user's operation behavior of the one-way movement is judged by the following steps: judging whether the coordinates of each point on the finger have the same change trend at a certain moment; judging whether there is a coordinate projection on at least one of the three coordinate planes It is linear; it encapsulates the user's operation behavior into a class.

并且,所述多向运动的用户的操作行为通过如下步骤进行判断:判断手指个数;模拟出手指在空间的轨迹变化;将用户的操作行为封装成类。Moreover, the user's operation behavior of the multi-directional movement is judged through the following steps: judging the number of fingers; simulating the track changes of the fingers in space; encapsulating the user's operation behavior into classes.

并且,所述转动的用户的操作行为通过如下步骤进行判断:初步判断用户的手势是否做旋转运动;判断手指个数;分析用户手势的旋转趋势;分析用户手指在空间中的运动轨迹;将用户的操作行为封装成类。And, the operation behavior of the rotating user is judged through the following steps: preliminary judgment of whether the user's gesture is rotating; judging the number of fingers; analyzing the rotation trend of the user's gesture; analyzing the trajectory of the user's finger in space; The operation behavior is encapsulated into a class.

本发明的三维操作装置,包括三维显示器、控制器、存储器以及检测部件。The three-dimensional operation device of the present invention includes a three-dimensional display, a controller, a memory and a detection component.

并且,所述多个距离和角度传感器设置在所述三维显示器的周边。Also, the plurality of distance and angle sensors are arranged around the three-dimensional display.

并且,所述控制器根据所述多个距离和角度传感器的信息分析和判断用户的操作行为,并将所判断的用户的操作行为归类到一种操作类型。Moreover, the controller analyzes and judges the user's operation behavior according to the information of the plurality of distance and angle sensors, and classifies the judged user's operation behavior into one type of operation.

并且,所述三维显示器根据所述操作类型显示三维画面。In addition, the three-dimensional display displays a three-dimensional image according to the operation type.

通过本发明所提供的三维操作方法和三维操作装置,可利用空间的手势去操作三维画面,对3D效果提供了完美支持,让UI变得更加唯美,更加感性。Through the three-dimensional operation method and the three-dimensional operation device provided by the present invention, spatial gestures can be used to operate the three-dimensional picture, which provides perfect support for the 3D effect and makes the UI more beautiful and perceptual.

附图说明Description of drawings

图1为本发明的三维操作方法的流程图。Fig. 1 is a flow chart of the three-dimensional operation method of the present invention.

图2为在三维显示器上安装距离和角度传感器的示意图。Fig. 2 is a schematic diagram of installing distance and angle sensors on a three-dimensional display.

图3为用于说明求出手指上的某一点的坐标的立体坐标系。FIG. 3 is a diagram illustrating a three-dimensional coordinate system for obtaining the coordinates of a certain point on the finger.

图4为用于说明单向运动的用户的操作行为的判断过程的流程图。FIG. 4 is a flow chart for explaining a process of judging a user's operation behavior of a unidirectional movement.

图5为用于说明多向运动的用户的操作行为的判断过程的流程图。FIG. 5 is a flow chart for explaining the process of judging the user's operation behavior of the multi-directional movement.

图6为用于说明转动行为的用户形成的判断过程的流程图。FIG. 6 is a flowchart for explaining a user-generated judgment process of a turning behavior.

具体实施方式detailed description

以下,参照附图详细说明本发明的实施方式。为了便于说明,本发明的移动终端以手机为例进行说明。本发明的手机至少包括三维显示器、控制器、存储器和显示器。Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. For ease of description, the mobile terminal of the present invention is described by taking a mobile phone as an example. The mobile phone of the present invention at least includes a three-dimensional display, a controller, a memory and a display.

首先,参考图1说明本发明的三维操作方法。First, the three-dimensional manipulation method of the present invention will be described with reference to FIG. 1 .

如图1所示,本发明的三维操作方法包括:As shown in Figure 1, the three-dimensional operation method of the present invention comprises:

步骤101:基于分别设置在屏幕周边的距离传感器和角度传感器所测量的距离信息和角度信息分析和判断用户的操作行为。Step 101: Analyzing and judging the user's operation behavior based on the distance information and angle information measured by the distance sensor and the angle sensor respectively arranged around the screen.

步骤102:根据用户的操作行为的判断结果,判断操作类型。Step 102: According to the judgment result of the user's operation behavior, judge the operation type.

在此,用户的操作行为通过软件接口被归纳为不同的操作类型,再通过相关的软件接口提供给用户界面供使用。这里,操作类型主要有移动、放大、缩小、旋转等。Here, the user's operation behavior is summarized into different operation types through the software interface, and then provided to the user interface for use through the relevant software interface. Here, the operation types mainly include moving, zooming in, zooming out, rotating, and the like.

步骤103:根据操作类型,在用户界面实现三维显示。Step 103: Realize three-dimensional display on the user interface according to the operation type.

下面具体说明判断用户的操作行为的方法。The method for judging the user's operation behavior will be described in detail below.

用户的三维操作行为会有很多种,若单纯从坐标变化的角度进行量化分析,大致可根据坐标变化将用户的操作行为分为单向运动、多向运动和转动这三类。There are many kinds of three-dimensional operation behaviors of users. If quantitative analysis is performed purely from the perspective of coordinate changes, user operation behaviors can be roughly divided into three types: one-way movement, multi-directional movement and rotation according to coordinate changes.

为了判断和分析用户的操作行为,本发明在三维显示器的四角安装了距离和角度传感器。图1表示在三维显示器上安装距离和角度传感器的示意图。如图1所示,在三维显示器的四角安装距离和角度传感器,通过传感器可以测量出手指上的某一点与传感器的直线距离和该点与传感器的连线与三维显示器所成的角度。也即,通过距离传感器可测量出手指和传感器的距离和角度等信息,再通过立体几何算法计算出手和三维显示器的多点相对位置,根据这些信息对用户的操作行为进行判断。In order to judge and analyze the user's operation behavior, the present invention installs distance and angle sensors at the four corners of the three-dimensional display. Figure 1 shows a schematic diagram of installing distance and angle sensors on a 3D display. As shown in Figure 1, distance and angle sensors are installed at the four corners of the 3D display, and the sensor can measure the linear distance between a point on the finger and the sensor and the angle between the point and the sensor and the 3D display. That is to say, information such as the distance and angle between the finger and the sensor can be measured through the distance sensor, and then the multi-point relative positions of the hand and the 3D display can be calculated through the three-dimensional geometry algorithm, and the user's operation behavior can be judged based on this information.

以单点为例,具体的计算方法如下。Taking a single point as an example, the specific calculation method is as follows.

首先假设手机屏幕处于xy坐标内,屏幕的长为L宽为W。如图2所示建立坐标系,手指上的某一点在坐标系中的相对坐标为d(x,y,z)。根据传感器的测量数据,我们可以测量出d与屏幕上传感器的距离l1、l2、l3,以及他们与手机屏幕所成的角度a,b,c。First assume that the screen of the mobile phone is in xy coordinates, and the length of the screen is L and the width is W. Establish a coordinate system as shown in Figure 2, and the relative coordinates of a point on the finger in the coordinate system are d(x, y, z). According to the measurement data of the sensor, we can measure the distances l 1 , l 2 , and l 3 between d and the sensor on the screen, and the angles a, b, and c formed between them and the screen of the mobile phone.

根据上述的测量数据可以列出以下方程式:According to the above measurement data, the following equation can be listed:

z=l1sina………………1)z=l 1 sina………………1)

x2+y2=(l2cosb)2………………2)x 2 +y 2 =(l 2 cosb) 2 ………………2)

(W-x)2+y2=(l2cosa)2………………3)(Wx) 2 +y 2 =(l 2 cosa) 2 ………………3)

(L-y)2+x2=(l2cosc)2………………4)(Ly) 2 +x 2 =(l 2 cosc) 2 ………………4)

根据上述方程式就可以计算出x,y,z的值为:According to the above equation, the values of x, y, and z can be calculated as:

z=l1sina………………7)z=l 1 sina……………7)

虽然测量用户手指上各点的距离信息和角度信息仅用三个距离传感器和角度传感器即可,但由于传感器在测量上可能受到噪声、环境等干扰产生一定的误差,因此本发明还设置了第四个传感器作为校验传感器,根据第四个传感器的返回数据,利用上述的公式重新计算,选取测量准确的坐标点,达到去噪的效果。Although only three distance sensors and angle sensors can be used to measure the distance information and angle information of each point on the user's finger, but because the sensors may be disturbed by noise and the environment to produce certain errors in measurement, the present invention also sets the first The four sensors are used as calibration sensors. According to the return data of the fourth sensor, the above formula is used to recalculate, and the coordinate points with accurate measurement are selected to achieve the effect of denoising.

根据上面的方法就可以计算出手指上各点与手机屏幕的相对坐标值D1(x1,y1,z1)、D2(x2,y2,z2)…DN(xN,yN,zN),再根据这些坐标就可以描绘出用户在这一时刻的手势。用户的手指不断在动,每一时刻的数据都不同,用time1至timeM表示不同时刻,并用D1至DN代表所有被选取的参考点,这样就可以得到下面的一组坐标数据:According to the above method, the relative coordinate values D 1 (x 1 , y 1 , z 1 ), D 2 (x 2 , y 2 , z 2 )...D N (x N ) of each point on the finger and the screen of the mobile phone can be calculated ,y N ,z N ), and then based on these coordinates, the user's gesture at this moment can be described. The user's fingers are constantly moving, and the data at each moment is different. Use time 1 to time M to represent different moments, and use D 1 to D N to represent all selected reference points, so that the following set of coordinate data can be obtained:

time1 time 1 time2 time 2 ...... timeM time M D1 D 1 x11,y11,z11 x 11 ,y 11 ,z 11 x12,y12,z12 x 12 ,y 12 ,z 12 ...... x1M,y1M,z1M x 1M ,y 1M ,z 1M D2 D2 x21,y21,z21 x 21 ,y 21 ,z 21 x22,y22,z22 x 22 ,y 22 ,z 22 ...... x2M,y2M,z2M x 2M ,y 2M ,z 2M D3 D3 x31,y31,z31 x 31 ,y 31 ,z 31 x31,y31,z31 x 31 ,y 31 ,z 31 ...... x3M,y3M,z3M x 3M ,y 3M ,z 3M ...... ...... ...... ...... ...... DN D N xN1,yN1,zN1 x N1 ,y N1 ,z N1 xN2,yN2,zN2 x N2 ,y N2 ,z N2 ...... xNM,yNM,zNM x NM ,y NM ,z NM

根据上面的数据,就可以对每一点或每一时刻的坐标值进行建模,描绘出所有点在空间中的变化轨迹,再根据坐标变化的轨迹去判断用户的操作行为。According to the above data, the coordinate value of each point or each moment can be modeled, and the change trajectory of all points in space can be described, and then the user's operation behavior can be judged according to the coordinate change trajectory.

由于D1至DN的坐标都是三维空间的立体坐标,为了根据坐标变化判断出用户的操作行为,需要将坐标在三个坐标面上进行投影操作,根据轨迹在坐标面上的投影对运动轨迹进行判断。再根据不同的运动轨迹各自的运动规律进行量化分析,最终确定用户的行为。Since the coordinates of D 1 to D N are three-dimensional coordinates in three-dimensional space, in order to judge the user's operation behavior according to the coordinate change, it is necessary to perform projection operations on the coordinates on three coordinate planes, and to control the movement according to the projection of the trajectory on the coordinate plane. track to judge. Quantitative analysis is then carried out according to the respective motion rules of different motion trajectories, and finally the user's behavior is determined.

在此,所谓投影就是找出D(x,y,z)点到某个平面的垂直照射点,例如,将D(x,y,z)对三个坐标平面进行投影操作,所得到的投影点坐标分别为d(x,y,0)、d(0,y,z)、d(x,0,z)。Here, the so-called projection is to find the vertical illumination point from D(x, y, z) to a certain plane. For example, the projection operation of D(x, y, z) on three coordinate planes, the obtained projection The point coordinates are d(x,y,0), d(0,y,z), d(x,0,z) respectively.

下面以实施例说明判断用户的操作行为的方法。The method for judging the user's operation behavior is described below with an embodiment.

实施例1:单向运动Example 1: One-way movement

单向运动在现有的触摸屏技术中是应用的最为广泛的一种运动趋势,现有触摸屏技术中可以通过记录手指在屏幕上滑动的轨迹来完成对用户的操作行为的判断。但是在本发明中,手指或手掌是在空间中进行滑动,不直接接触触摸屏,现有的判断技术无法实现。One-way movement is the most widely used movement trend in the existing touch screen technology. In the existing touch screen technology, the judgment of the user's operation behavior can be completed by recording the trajectory of the finger sliding on the screen. However, in the present invention, fingers or palms slide in space without directly touching the touch screen, which cannot be realized by existing judgment techniques.

本发明中单向运动的用户的操作行为的判断方法如图4所示。The method for judging the user's operation behavior of one-way movement in the present invention is shown in FIG. 4 .

单向运动的特点是一根或多跟手指做平行于某一坐标面或者斜向运动操作的情况。在这种情况下,所有参考点在time1至timeM时间内的运动轨迹大体一致。若对整个运动轨迹微分,选取所有点的坐标在某一时刻的变化趋势进行抽样计算,其规律大体符合如下公式:One-way motion is characterized by one or more fingers operating parallel to a certain coordinate plane or obliquely. In this case, the motion trajectories of all reference points from time 1 to time M are roughly the same. If the entire trajectory is differentiated, the change trend of the coordinates of all points at a certain moment is selected for sampling calculation, and the law generally conforms to the following formula:

由于手指划过的速度不同,不能保证整个轨迹的变化趋势是定值,但微分到单点的变化趋势是一致的。Due to the different speeds of the finger swipe, it cannot be guaranteed that the change trend of the entire trajectory is a constant value, but the change trend of the differential to a single point is consistent.

因此,本发明的单向运动的判断方法首先判断手指上各点的坐标在某一时刻的变化趋势是否相同(S401)。Therefore, the method for judging one-way motion of the present invention first judges whether the coordinates of each point on the finger have the same changing trend at a certain moment (S401).

另外,将D1至DN的坐标投影到三个坐标平面上时,至少在一个坐标平面上为线形。In addition, when the coordinates of D 1 to D N are projected onto three coordinate planes, at least one coordinate plane is linear.

因此,本发明的单向运动的判断方法再判断三个坐标平面中是否至少有一个平面上的坐标投影为线形(S402)。Therefore, the method for judging one-way motion of the present invention further judges whether the coordinate projection on at least one of the three coordinate planes is linear ( S402 ).

同时满足上述两点的情况下,可以判定手指或手掌在做单向的线性运动,将用户的操作行为封装成类(S403)。When the above two points are satisfied at the same time, it can be determined that the finger or the palm is making a unidirectional linear movement, and the user's operation behavior is encapsulated into a class ( S403 ).

本发明还可以根据运动的轨迹、坐标的变化,计算出运动的速度、方向以及与坐标平面所成角度等操作参数,并将对应于用户的操作行为的操作类型与各种操作参数提供给UI层,供UI层显示对应的三维画面。The present invention can also calculate the operation parameters such as the speed, direction and angle with the coordinate plane according to the trajectory of the movement and the change of the coordinates, and provide the operation type and various operation parameters corresponding to the user's operation behavior to the UI layer, for the UI layer to display the corresponding 3D images.

实施例2:多向运动Example 2: Multi-directional movement

多向运动也是比较常见的用户的操作行为,在现有的触摸屏技术中,可以通过最初时刻的触点个数及触摸的轨迹来判断手的运动轨迹,再根据轨迹对用户的行为进行判断。Multi-directional movement is also a relatively common user operation behavior. In the existing touch screen technology, the trajectory of the hand can be judged by the number of contacts at the initial moment and the trajectory of the touch, and then the user's behavior can be judged according to the trajectory.

但是在三维空间中,与单向运动一样,用户的手指运动轨迹是在三维空间中形成的,并没有触碰到触摸屏。However, in the three-dimensional space, like the one-way movement, the user's finger movement trajectory is formed in the three-dimensional space, and does not touch the touch screen.

图5用于说明多向运动的用户的操作行为的判断过程的流程图。FIG. 5 is a flow chart for explaining the judging process of the user's operation behavior of the multi-directional movement.

若要判断用户的操作行为,首先要判断在屏幕的测量范围内有几根手指(S501)。To determine the user's operation behavior, it is first necessary to determine how many fingers are within the measurement range of the screen (S501).

判断手指个数可通过如下方法实现,即,对time1时刻的所有点的坐标进行建模,即将time1时刻的D1至DN的所有坐标都投影到三个坐标平面上,此时在每个坐标平面上得到的轨迹应该是多条类似直线的线性轨迹,三个坐标平面中线性轨迹最多的平面上的轨迹个数应该就是手指的个数。再通过投影点的轨迹还原出空间中手指在这一时刻的位置。Judging the number of fingers can be achieved by the following method, that is, modeling the coordinates of all points at time 1 , that is, projecting all coordinates of D 1 to D N at time 1 onto three coordinate planes, at this time The trajectories obtained on each coordinate plane should be multiple linear trajectories similar to straight lines, and the number of trajectories on the plane with the most linear trajectories among the three coordinate planes should be the number of fingers. Then restore the position of the finger in space at this moment through the trajectory of the projected point.

然后,模拟出用户手指在空间中的轨迹变化(S502)。可通过对time1至timeM时刻的D1至DN的坐标进行投影操作,就可以通过在坐标平面上的坐标点的变化,模拟出空间中的轨迹变化。Then, the track change of the user's finger in space is simulated (S502). By performing a projection operation on the coordinates of D 1 to D N from time 1 to time M , the trajectory change in space can be simulated through the change of coordinate points on the coordinate plane.

然后,根据轨迹变化判断用户的行为,如手指聚合、分散等运动方式,将用户的操作行为封装成类。Then, judge the user's behavior according to the trajectory change, such as finger movement such as aggregation and dispersion, and encapsulate the user's operation behavior into classes.

实施例3:转动Example 3: Turn

转动是在二维空间中所没有的用户的操作行为,是3D空间所特有的一种运动趋势。转动是一种相对自由的坐标变化趋势。用户的手可以以手腕为轴以任何姿势在屏幕前进行转动。每一参考点的运动轨迹都类似一段不规则弧形,无法简单的从单一坐标变化趋势上去判断手在转动。Rotation is an operation behavior of the user that does not exist in the two-dimensional space, and is a movement trend unique to the 3D space. Rotation is a relatively free coordinate change trend. The user's hand can rotate in front of the screen in any posture around the wrist. The trajectory of each reference point is similar to an irregular arc, and it is impossible to judge the hand rotation simply from the trend of a single coordinate change.

图6为用于说明判断用户手势的转动行为的过程的流程图。FIG. 6 is a flow chart for explaining the process of judging the rotation behavior of the user's gesture.

首先,本发明初步判断用户的手势是否做旋转运动(S601)。其判断方法为,对所有点的坐标进行投影分析。具体来讲,将time1至timeM时刻的D1至DN的坐标(即所有点的坐标数据)分别投影到三个坐标面上,此时若至少有一个平面上的投影轨迹呈圆形或椭圆形,就可以初步判断用户的手势类似一个球面,在做旋转运动。First, the present invention preliminarily judges whether the user's gesture is rotating (S601). The judgment method is to perform projection analysis on the coordinates of all points. Specifically, the coordinates of D 1 to D N from time 1 to time M (that is, the coordinate data of all points) are respectively projected onto three coordinate planes. At this time, if the projection trajectory on at least one plane is circular Or ellipse, it can be preliminarily judged that the user's gesture is similar to a spherical surface, making a rotating motion.

其次,判断手指个数(S602)。利用time1时刻的所有点的坐标,确定屏幕范围内的手指个数及形状。Next, determine the number of fingers (S602). Use the coordinates of all points at time 1 to determine the number and shape of fingers within the screen range.

然后,分析用户手势的旋转趋势(S603)。当用户的手进行旋转时,timeM-1至timeM这个过程中,每根手指上各点的向量变化应该是一致的。向量分析方法如下:Then, analyze the rotation tendency of the user's gesture ( S603 ). When the user's hand rotates, during the process from time M-1 to time M , the vector changes of each point on each finger should be consistent. The vector analysis method is as follows:

对time1至timeM时刻的D1至DN的坐标进行分析,计算出time1至time2、time2至time3、…、timeM-1至timeM各点的向量值。Analyze the coordinates of D 1 to D N from time 1 to time M , and calculate the vector values of each point from time 1 to time 2 , time 2 to time 3 , ..., time M-1 to time M.

假设点D1和D2在同一根手指上,则两点在time1至time2时刻向量的变化是相同的,既满足以此类推可以得到公式:Assuming that points D 1 and D 2 are on the same finger, the changes of the vectors of the two points from time 1 to time 2 are the same, satisfying By analogy, the formula can be obtained:

为了减少测量误差,可以对同一根手指上的多个点进行向量叉积来得到拟合后的结果。在同一时间段内各个手指上的采样点向量变化趋势相同,则可确定用户的手势在做旋转运动。In order to reduce the measurement error, a vector cross product can be performed on multiple points on the same finger to obtain the fitted result. In the same period of time, the vectors of the sampling points on each finger have the same change trend, so it can be determined that the user's gesture is performing a rotational movement.

然后,分析每根手指在空间中的运动轨迹,计算旋转参数(S604)。Then, analyze the trajectory of each finger in space, and calculate the rotation parameter (S604).

根据上面的数据,可以将用户手势的变化趋势近似地看做一种类似球面的旋转运动。这样可以分别计算出time1至timeM时刻球面旋转的球心、半径和旋转轴等信息,并且计算出各个时刻球心及旋转轴的向量变化趋势。According to the above data, the changing trend of user gestures can be approximately regarded as a spherical rotation motion. In this way, information such as the center, radius, and axis of rotation of the spherical surface from time 1 to time M can be calculated separately, and the vector change trends of the center of the sphere and the axis of rotation can be calculated at each time.

以单一时刻为例,下面来介绍一下具体的计算方法:Taking a single moment as an example, let’s introduce the specific calculation method as follows:

<球心及半径的计算方法><Calculation method of sphere center and radius>

从理论上可知,只要有在同一个圆上的4个点就可以求出球心的坐标和半径。假设球半径为R,球心坐标的近似值为O(x0,y0,z0),球面上的第i点的坐标为Di(xi,yi,zi),则每一点的坐标与球半径存在如下关系式:It can be seen from theory that as long as there are 4 points on the same circle, the coordinates and radius of the center of the sphere can be obtained. Assuming that the radius of the sphere is R, the approximate value of the coordinates of the center of the sphere is O(x 0 , y 0 , z 0 ), and the coordinates of the i-th point on the sphere are D i ( xi , y i , z i ), then each point’s There is the following relationship between the coordinates and the radius of the ball:

(x0-xi)2+(y0-yi)2+(z0-zi)2=R2………………8)(x 0 -x i ) 2 +(y 0 -y i ) 2 +(z 0 -z i ) 2 =R 2 ………………8)

这样选取四个点的坐标既可以计算出球心坐标值及半径。但这只是理论值,由于用户的转动动作是类似的球面运动,各点坐标的测量选取也有一定误差,所以要对多个点的计算结果进行拟合。In this way, the coordinates of the center of the sphere and the radius can be calculated by selecting the coordinates of the four points. But this is only a theoretical value. Since the user's rotation is a similar spherical motion, there are certain errors in the measurement and selection of the coordinates of each point, so it is necessary to fit the calculation results of multiple points.

这里有很多种方法可供选择,例如,最常用的方法就是穷举,将所有的观察点的坐标都带入方程式计算出多个球心坐标,再进行去噪拟合,这种方法虽然简单易懂,但计算量非常大,影响手机性能。There are many methods to choose from here. For example, the most commonly used method is exhaustion. Put the coordinates of all observation points into the equation to calculate multiple coordinates of the center of the sphere, and then perform denoising and fitting. Although this method is simple It is easy to understand, but the amount of calculation is very large, which affects the performance of mobile phones.

下面主要来介绍一种平差计算的方法,以求得球心坐标和球半径的最可靠值。The following mainly introduces a method of adjustment calculation to obtain the most reliable values of the coordinates of the center of the sphere and the radius of the sphere.

对上面的公式中针对球心坐标和球半径求偏微分,并以其修正值δx、δy、δz和δR代替微分,得到:Calculate the partial differential for the coordinates of the center of the sphere and the radius of the sphere in the above formula, and replace the differential with its corrected values δ x , δ y , δ z and δ R to obtain:

2(x0-xix+2(y0-yiy+2(z0-ziz=2R0δR………………9)2(x 0 -x ix +2(y 0 -y iy +2(z 0 -z iz =2R 0 δ R ……………9)

在此,R0为半径拟合值。Here, R 0 is the fitted value of the radius.

假设按第i点的坐标值与球心的近似坐标计算的球半径为球半径的观测值Rt,其改正值为VRt,则Assuming that the spherical radius calculated according to the coordinate value of the i-th point and the approximate coordinates of the center of the sphere is the observed value R t of the sphere radius, and its correction value is V Rt , then

R=Rt+VRt=R0R………………12)R=R t +V Rt =R 0R ………………12)

VRt=R0-RtR………………13)V Rt =R 0 -R tR ………………13)

上式中,δR已求得,可以用前面的公式代入,得到第i点的球半径观测值方程式:In the above formula, δ R has been obtained, and can be substituted by the previous formula to obtain the equation of the observed value of the spherical radius at the i-th point:

则方程式的一般形式为:Then the general form of the equation is:

VRt=atδx+btδy+ctδz+lt………………15)V Rt =a t δ x +b t δ y +c t δ z +l t ……………15)

由m个观测值的方程式和3个未知数δx、δy、δz组成的方程式为The equation consisting of the equation of m observed values and three unknowns δ x , δ y , δ z is

[aa]δx+[ab]δy+[ac]δz+[al]=0………………16)[aa]δ x +[ab]δ y +[ac]δ z +[al]=0……………16)

[ab]δx+[bb]δy+[bc]δz+[bl]=0………………17)[ab]δ x +[bb]δ y +[bc]δ z +[bl]=0……………17)

[ac]δx+[bc]δy+[cc]δz+[cl]=0………………18)[ac]δ x +[bc]δ y +[cc]δ z +[cl]=0……………18)

设法方程式的协因数矩阵为The cofactor matrix of the managed equation is

则未知数的解为Then the solution of the unknown is

δx=-[al]Q11-[bl]Q12-[cl]Q13………………19)δ x =-[al]Q 11 -[bl]Q 12 -[cl]Q 13 ……………19)

δy=-[al]Q12-[bl]Q22-[cl]Q23………………20)δ y =-[al]Q 12 -[bl]Q 22 -[cl]Q 23 ……………20)

δz=-[al]Q13-[bl]Q23-[cl]Q33………………21)δ z =-[al]Q 13 -[bl]Q 23 -[cl]Q 33 ……………… 21)

球心平差后的坐标为:The coordinates after spherical adjustment are:

x=x0x………………22)x=x 0x ……………… 22)

y=y0y………………23)y=y 0y …………… 23)

z=z0z………………24)z=z 0z ……………24)

平差后的球半径为:The adjusted spherical radius is:

………………25) …………… 25)

式中,分子为观测值方程式常数项之和,分母为观测值的个数。In the formula, the numerator is the sum of the constant terms of the observation value equation, and the denominator is the number of observation values.

根据上述方法,就可以较为精确地求出单一时刻的球面旋转的球心及半径,当然计算方法有很多种,本发明仅以此为例。According to the above method, the center and radius of the spherical surface rotation at a single moment can be calculated more accurately. Of course, there are many calculation methods, and the present invention only uses this as an example.

<旋转轴计算方法><Rotation axis calculation method>

球面的转轴是经过球心的一条直线,该轴线与旋转平面的法向量平行。换言之,只要求得旋转平面的法向量,再代入球心坐标,即可将旋转轴唯一确定。此时,旋转平面的法向量如何计算就成了问题的关键。The axis of rotation of the sphere is a straight line passing through the center of the sphere, which is parallel to the normal vector of the plane of rotation. In other words, only the normal vector of the rotation plane is required, and then substituted into the coordinates of the center of the sphere, the rotation axis can be uniquely determined. At this point, how to calculate the normal vector of the rotating plane becomes the key to the problem.

这里,所谓旋转平面是指,在time1时刻的所有点坐标还原到坐标系中形成一个类似球面的形状,选取接近各个手指指尖的坐标点D1至Dt为采样点,这些采样点可以近似的拟合在同一平面内,该平面就是这一时刻的旋转平面。Here, the so-called rotating plane means that the coordinates of all points at time 1 are restored to the coordinate system to form a spherical shape, and the coordinate points D1 to Dt close to the fingertips of each finger are selected as sampling points, and these sampling points can be The approximate fit is in the same plane, which is the plane of rotation at this moment.

根据空间中的两点可以确定一条直线,两条交叉的直线就可以确定一个平面的原理,只要有不在同一直线上的四个点就可以确定一个平面,并且计算出他的法向量。A straight line can be determined according to two points in space, and a plane can be determined by two intersecting straight lines. As long as there are four points that are not on the same straight line, a plane can be determined and its normal vector can be calculated.

假设有D1至Dt个点,大致分散在四到五个区域内,只要在不同区域内分别选取,就可以确定多个旋转平面并计算出多个法向量,再将这些法向量做向量叉乘运算,如果两个平面是平行的,则向量叉乘的结果为0。在本发明中,可以以使用多向量拟合的方法,来确定旋转平面的法向量。Assuming that there are D 1 to D t points, which are roughly scattered in four to five areas, as long as they are selected in different areas, multiple rotation planes can be determined and multiple normal vectors can be calculated, and then these normal vectors can be used as vectors Cross product operation, if the two planes are parallel, the result of vector cross product is 0. In the present invention, a multi-vector fitting method can be used to determine the normal vector of the rotation plane.

计算出单一时刻的模拟球面运动的球心、半径、旋转轴等信息后,就可以用同样的方法对time2至timeM时刻的坐标进行建模,计算出每一个时刻的球面旋转信息,如下表所示:After calculating the center, radius, and axis of rotation of the simulated spherical motion at a single moment, you can use the same method to model the coordinates from time 2 to time M , and calculate the spherical rotation information at each moment, as follows As shown in the table:

根据这些信息,就可以进一步计算球心变化向量,旋转轴的向量变化等信息,可以对整个手势的变化轨迹做一个量化的建模分析。Based on this information, you can further calculate information such as the change vector of the center of the sphere, the vector change of the rotation axis, etc., and perform a quantitative modeling analysis on the change trajectory of the entire gesture.

最后,将用户的操作行为信息封装成类,返回给应用程序接口。Finally, the user's operation behavior information is encapsulated into a class and returned to the application program interface.

上述几种用户的操作行为只是从坐标变化的角度做一个简单的划分,因为人手的动作可以千变万化,还可以根据坐标变化的速度和时间等信息,对运动轨迹进行更细致的分析。The above-mentioned types of user operation behaviors are simply divided from the perspective of coordinate changes, because the movements of human hands can be varied, and the movement trajectory can be analyzed in more detail based on information such as the speed and time of coordinate changes.

此外,虽然本发明以设置在三维显示器四角上的距离和角度传感器为例进行了说明,但本发明并不限定于此,本发明可以通过设置在手机屏幕周边的一个雷达测量用户手指上的某一点与雷达的直线距离以及该点与雷达的连线与三维显示器所成的角度。即,本发明可通过雷达代替距离传感器和角度传感器测量距离信息和角度信息。在此,本发明对雷达的设置数量并不进行限定,例如可以在手机屏幕周边设置两个雷达,一个雷达用于测量距离信息和角度信息,另一个雷达用于校验所测量的距离信息和角度信息准确度。In addition, although the present invention has been described by taking the distance and angle sensors arranged on the four corners of the three-dimensional display as an example, the present invention is not limited thereto. The straight-line distance between a point and the radar and the angle between the line connecting the point and the radar and the 3D display. That is, the present invention can measure distance information and angle information by using a radar instead of a distance sensor and an angle sensor. Here, the present invention does not limit the number of radar settings. For example, two radars can be set around the screen of the mobile phone, one radar is used to measure distance information and angle information, and the other radar is used to verify the measured distance information and angle information. Accuracy of angle information.

本发明主要是实现三维显示器的3D控制,应用十分广泛。可用于在3D游戏及3D效果图的演示当中,通过用户的手势分析用户的操作行为,封装成统一的UI接口,应用可以利用这些接口实现不同效果的UI风格,给用户完全不同的用户体验。将本发明真正运用到手机上的时候,其应用之处可以说数不胜数。例如:The invention mainly realizes the 3D control of the three-dimensional display and is widely used. It can be used in the demonstration of 3D games and 3D renderings, analyze the user's operation behavior through the user's gestures, and package it into a unified UI interface. The application can use these interfaces to achieve different UI styles and give users a completely different user experience. When the present invention is actually applied to mobile phones, its applications can be said to be innumerable. E.g:

1.3D切水果:通过手在空间中做真实的切水果的动作,来完成游戏。1.3D fruit cutting: complete the game by doing real fruit cutting movements with your hands in space.

2.虚拟魔方:用户可以通过手势在手机屏幕前直接做各种旋转操作,直接来控制屏幕中的魔方转到,实现3D展示效果。2. Virtual Rubik's Cube: Users can directly perform various rotation operations in front of the mobile phone screen through gestures, and directly control the rotation of the Rubik's Cube on the screen to achieve a 3D display effect.

3.TOM猫:用户可以在手机屏幕前做一个拉的动作,去拉TOM的毛耳朵或者毛,给TOM搭配上更多的表情动作。3. TOM cat: Users can make a pulling action in front of the mobile phone screen to pull TOM's furry ears or fur, and match TOM with more expressions.

4.3D桌面:打破原有的2D桌面效果,桌面上的图标可以是做成立体效果,桌面空间也可以变成3D空间,通过手势将图标拉进,放远,放大,缩小等操作。4.3D desktop: break the original 2D desktop effect, the icons on the desktop can be made into a three-dimensional effect, and the desktop space can also be turned into a 3D space, and the icons can be pulled in, farther away, zoomed in, zoomed out and other operations through gestures.

3D操作的例子不胜枚举,本发明的优点在于利用空间的手势去判断用户的行为,完全不同于现有触摸屏技术,用户无需触碰屏幕就可完成操作。并且解放了手指,将触碰操作变为了空间运动行为分析。用户可以有更多的选择,更多的手势去控制手机。同时也为UI层提供了丰富的操作类,对3D效果的完美支持,让UI变地得更加唯美,更加感性。Examples of 3D operations are too numerous to enumerate. The present invention has the advantage of using spatial gestures to judge the user's behavior, which is completely different from the existing touch screen technology, and the user can complete the operation without touching the screen. And it liberates the fingers, turning the touch operation into a spatial motion behavior analysis. Users can have more choices and more gestures to control the phone. At the same time, it also provides rich operation classes for the UI layer and perfect support for 3D effects, making the UI more beautiful and emotional.

Claims (11)

1. a kind of three-dimensional manipulating method, it is characterised in that include step:
Based on the range information measured by detection part and angle information analysis and the operation behavior for judging user;
Decision type, quantization operation parameter according to the judged result of the operation behavior of user;
According to action type and operating parameter, three-dimensional manipulating is performed in user interface,
Wherein, the step of analyzing based on the range information measured by detection part and angle information and judge the operation behavior of user Including:Coordinate is calculated according to range information and angle information, coordinate projected on coordinate surface, according on coordinate surface Projection is judged movement locus, quantitative analysis is carried out according to the respective characteristics of motion of movement locus, to determine the behaviour of user Make behavior,
Wherein, the operation behavior of the user includes rotating, and the operation behavior of the user of the rotation is carried out as follows It is determined that:Tentatively judge whether the gesture of user rotates;Judge finger number;By analyzing within the same period each The variation tendency of sampling point vector on finger analyzes the rotating tendency of user gesture;Analyze the fortune of user's finger in space Dynamic rail mark;The operation behavior of user is packaged into class.
2. three-dimensional manipulating method according to claim 1, it is characterised in that the detection part includes three Distance-sensings Device and angular transducer.
3. three-dimensional manipulating method according to claim 2, it is characterised in that the detection part further comprise one away from Check sensor is used as from sensor and angular transducer.
4. three-dimensional manipulating method according to claim 1, it is characterised in that the detection part includes radar.
5. three-dimensional manipulating method according to claim 1, it is characterised in that the operating parameter includes movement velocity, fortune Dynamic direction and with screen angulation.
6. three-dimensional manipulating method according to claim 1, it is characterised in that the operation behavior of the user also includes unidirectional Motion and Multidirectional motion.
7. three-dimensional manipulating method according to claim 6, it is characterised in that the operation behavior of the user of the one-way movement It is determined as follows:
Judge whether the variation tendency of the coordinate of each point on finger at a time is identical;
Judge whether the coordinate projection at least one plane is linear in three coordinate planes;
The operation behavior of user is packaged into class.
8. three-dimensional manipulating method according to claim 6, it is characterised in that the operation behavior of the user of the Multidirectional motion It is determined as follows:
Judge finger number;
Simulate trail change of the finger in space;
The operation behavior of user is packaged into class.
9. a kind of three-dimensional manipulating device, it is characterised in that including three dimensional display, controller, memory and detection part,
Wherein, detection part measurement distance information and angle information,
Controller performs following operation:Coordinate is calculated according to range information and angle information;Coordinate is carried out on coordinate surface Projection;Movement locus is judged according to the projection on coordinate surface;Quantified according to the respective characteristics of motion of movement locus Analysis, to determine the operation behavior of user;Decision type, quantization operation according to the judged result of the operation behavior of user Parameter;According to action type and operating parameter, three-dimensional manipulating is performed in user interface,
Three dimensional display shows three-dimensional picture according to the action type of judgement,
Wherein, the operation behavior of the user includes rotating, and the operation behavior of the user of the rotation by operating progress as follows It is determined that:Tentatively judge whether the gesture of user rotates;Judge finger number;By analyzing within the same period each The variation tendency of sampling point vector on finger analyzes the rotating tendency of user gesture;Analyze the fortune of user's finger in space Dynamic rail mark;The operation behavior of user is packaged into class.
10. three-dimensional manipulating device according to claim 9, it is characterised in that multiple distances and angular transducer are arranged on The periphery of the three dimensional display.
11. three-dimensional manipulating device according to claim 9, it is characterised in that the controller is according to multiple away from walk-off angle Spend the information analysis of sensor and judge the operation behavior of user, and the operation behavior of the user judged is referred to a kind of behaviour Make type.
CN201310075672.4A 2013-03-08 2013-03-08 Three-dimensional manipulating method and three-dimensional manipulating device Expired - Fee Related CN104035552B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201310075672.4A CN104035552B (en) 2013-03-08 2013-03-08 Three-dimensional manipulating method and three-dimensional manipulating device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201310075672.4A CN104035552B (en) 2013-03-08 2013-03-08 Three-dimensional manipulating method and three-dimensional manipulating device

Publications (2)

Publication Number Publication Date
CN104035552A CN104035552A (en) 2014-09-10
CN104035552B true CN104035552B (en) 2017-08-25

Family

ID=51466354

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201310075672.4A Expired - Fee Related CN104035552B (en) 2013-03-08 2013-03-08 Three-dimensional manipulating method and three-dimensional manipulating device

Country Status (1)

Country Link
CN (1) CN104035552B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11592909B2 (en) 2015-10-06 2023-02-28 Google Llc Fine-motion virtual-reality or augmented-reality control using radar

Families Citing this family (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9575560B2 (en) 2014-06-03 2017-02-21 Google Inc. Radar-based gesture-recognition through a wearable device
US9811164B2 (en) 2014-08-07 2017-11-07 Google Inc. Radar-based gesture sensing and data transmission
US11169988B2 (en) 2014-08-22 2021-11-09 Google Llc Radar recognition-aided search
US9778749B2 (en) 2014-08-22 2017-10-03 Google Inc. Occluded gesture recognition
US9600080B2 (en) 2014-10-02 2017-03-21 Google Inc. Non-line-of-sight radar-based gesture recognition
CN105487692A (en) * 2014-12-22 2016-04-13 哈尔滨安天科技股份有限公司 Controller switching method and system based on three-dimensional display space
KR102327044B1 (en) 2015-04-30 2021-11-15 구글 엘엘씨 Type-agnostic rf signal representations
KR102002112B1 (en) 2015-04-30 2019-07-19 구글 엘엘씨 RF-based micro-motion tracking for gesture tracking and recognition
US10088908B1 (en) 2015-05-27 2018-10-02 Google Llc Gesture detection and interactions
US10492302B2 (en) 2016-05-03 2019-11-26 Google Llc Connecting an electronic component to an interactive textile
CN111198644B (en) * 2018-11-16 2023-10-27 西安易朴通讯技术有限公司 Screen operation identification method and system of intelligent terminal

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102236414A (en) * 2011-05-24 2011-11-09 北京新岸线网络技术有限公司 Picture operation method and system in three-dimensional display space
CN102804786A (en) * 2009-06-16 2012-11-28 Lg电子株式会社 Viewing range notification method and television receiver for implementing the method

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP3795647B2 (en) * 1997-10-29 2006-07-12 株式会社竹中工務店 Hand pointing device
CN102810015B (en) * 2011-05-31 2016-08-03 中兴通讯股份有限公司 Input method based on space motion and terminal

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102804786A (en) * 2009-06-16 2012-11-28 Lg电子株式会社 Viewing range notification method and television receiver for implementing the method
CN102236414A (en) * 2011-05-24 2011-11-09 北京新岸线网络技术有限公司 Picture operation method and system in three-dimensional display space

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11592909B2 (en) 2015-10-06 2023-02-28 Google Llc Fine-motion virtual-reality or augmented-reality control using radar

Also Published As

Publication number Publication date
CN104035552A (en) 2014-09-10

Similar Documents

Publication Publication Date Title
CN104035552B (en) Three-dimensional manipulating method and three-dimensional manipulating device
TWI827633B (en) System and method of pervasive 3d graphical user interface and corresponding readable medium
US8610714B2 (en) Systems, methods, and computer-readable media for manipulating graphical objects
US8576253B2 (en) Grasp simulation of a virtual object
US9632677B2 (en) System and method for navigating a 3-D environment using a multi-input interface
JP5752715B2 (en) Projector and depth camera for deviceless augmented reality and interaction
US8502795B2 (en) Physics simulation-based interaction for surface computing
CN101819462B (en) Image texture haptic representation system based on force/haptic interaction equipment
KR101318244B1 (en) System and Method for Implemeting 3-Dimensional User Interface
Kim et al. Encountered‐type haptic display for large VR environment using per‐plane reachability maps
Andersen et al. Immersion or diversion: Does virtual reality make data visualisation more effective?
WO2016188318A1 (en) 3d human face reconstruction method, apparatus and server
CN103235545B (en) Simulation test method and device for photoelectric tracking system
WO2013010027A1 (en) Drawing aid system for multi-touch devices
CN111665933A (en) Method and device for operating object in virtual or augmented reality
CN103942053A (en) Three-dimensional model gesture touch browsing interaction method based on mobile terminal
US20210311844A1 (en) Multiple xr extended reality application validation process and testing
US11809701B2 (en) Handwriting forming method and apparatus, and electronic device
US20170177077A1 (en) Three-dimension interactive system and method for virtual reality
Katzakis et al. INSPECT: extending plane-casting for 6-DOF control
CN103914242A (en) Pointer control method on touch screen
JP3722994B2 (en) Object contact feeling simulation device
US20130249807A1 (en) Method and apparatus for three-dimensional image rotation on a touch screen
CN107145706A (en) Method and device for evaluating performance parameters of virtual reality VR equipment fusion algorithm
CN104050721B (en) The method and device of smooth control three dimensional object

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
CF01 Termination of patent right due to non-payment of annual fee
CF01 Termination of patent right due to non-payment of annual fee

Granted publication date: 20170825

Termination date: 20210308