CN115904201A - Virtual reality equipment, its control method, device, and computer storage medium - Google Patents

Abstract

The application discloses a virtual reality device, its control method, device and computer storage medium, belonging to the technical field of virtual reality. The method includes: acquiring an environment image of the external environment of the device body through an image acquisition component; performing plane detection in the environment image; displaying at least one detected plane in the virtual environment of the virtual reality device; responding to recognizing the keyboard in the environment image , generating a virtual keyboard at the corresponding position of the keyboard in the virtual environment of the virtual reality device; in response to the virtual keyboard being placed on the target plane in at least one plane, establishing a virtual desktop at the corresponding position of the target plane in the virtual environment; obtaining the input of the mouse Signal; based on the input signal of the mouse, a selection ray corresponding to the mouse is generated in the virtual environment. In this way, the virtual environment can be controlled through an external mouse and keyboard, reducing the possibility of false touches.

Description

Virtual reality equipment, its control method, device, and computer storage medium

technical field

The present application relates to the technical field of virtual reality, and in particular to a virtual reality device, its control method, device and computer storage medium.

Background technique

A virtual reality device is a device that can provide a user with a virtual reality experience. Based on different application scenarios, virtual reality devices can provide different functions, such as games, film and television, and office work.

A control method of a virtual reality device, which is used to provide an office scene to a user. In the method, a virtual desktop is first generated in a virtual environment, a virtual keyboard is generated on the virtual desktop, and an image of the user's hand is captured by a camera. to generate a model of the hand in a virtual environment. When the user's hand model touches a corresponding key of the virtual keyboard, information corresponding to the key can be obtained.

However, in the above method, the possibility of the user touching by mistake is relatively high, resulting in low accuracy of the above method.

Contents of the invention

Embodiments of the present application provide a virtual reality device and its control method, device, and computer storage medium. Described technical scheme is as follows:

According to an aspect of the embodiment of the present application, a method for controlling a virtual reality device is provided, which is used for a virtual reality device. The virtual reality device includes a device body, a keyboard, and a mouse, and the device body has an image acquisition component. The Methods include:

Acquiring an environmental image of the external environment of the device body through the image acquisition component;

performing plane detection in the environment image;

displaying the detected at least one plane in a virtual environment of the virtual reality device;

in response to identifying the keyboard in the environment image, generating a virtual keyboard at a corresponding location of the keyboard in the virtual environment of the virtual reality device;

Responsive to the virtual keyboard being placed on a target plane in the at least one plane, establishing a virtual desktop at a corresponding location of the target plane in the virtual environment;

obtaining an input signal of the mouse;

Based on the input signal of the mouse, a selection ray corresponding to the mouse is generated in the virtual environment.

Optionally, the generating a selection ray corresponding to the mouse in the virtual environment based on the input signal of the mouse includes:

acquiring the plane coordinates of the mouse based on the input signal of the mouse;

transforming the plane coordinates into world coordinates in the virtual environment;

The selection ray is generated based on the world coordinates and the orientation of the virtual camera in the virtual environment, the selection ray passes through the world coordinates, and the direction is parallel to the orientation of the virtual camera.

Optionally, after generating the selection ray based on the world coordinates and the orientation of the virtual camera in the virtual environment, the method further includes:

performing collision detection on the selected ray and objects in the virtual environment;

The first object collided by the selection ray is determined as the object currently selected by the mouse.

Optionally, before acquiring the input signal of the mouse, the method further includes:

In response to identifying the mouse in the environment image, a virtual mouse is generated at a corresponding location of the mouse in the virtual environment of the virtual reality device.

Optionally, the performing plane detection in the environment image includes:

The plane detection is performed in the environment image by a random sampling consensus algorithm, a Hough transform algorithm or a region growing clustering algorithm.

Optionally, the generating a virtual keyboard at a corresponding position of the keyboard in the virtual environment of the virtual reality device includes:

Detecting whether the keyboard has any one of a plurality of preset characteristics;

When the keyboard has any of the features, obtain a keyboard model corresponding to any of the features;

The keyboard model is loaded at the corresponding position of the keyboard in the virtual environment of the virtual reality device.

Optionally, the feature includes at least one of a preset key arrangement, a preset optical mark, and a preset shape.

Optionally, loading the keyboard model at the corresponding position of the keyboard in the virtual environment of the virtual reality device includes:

determining the position and posture of the keyboard in the virtual environment through the environment image;

and loading the keyboard model in the virtual environment based on the position and posture.

Optionally, the method also includes:

When the keyboard does not have the preset feature, generating a first model of the keyboard based on the image of the keyboard;

The first model is loaded at a corresponding position of the keyboard in the virtual environment of the virtual reality device.

According to another aspect of the embodiment of the present application, a control device for a virtual reality device is provided. The virtual reality device includes a device body, a keyboard, and a mouse. The device body has an image acquisition component. The control device for the virtual reality device Devices include:

An image acquisition module, configured to acquire an environmental image of the external environment of the device main body through the image acquisition component;

a plane detection module, configured to perform plane detection in the environment image;

A plane display module, configured to display at least one detected plane in the virtual environment of the virtual reality device;

a keyboard generation module, configured to generate a virtual keyboard at a corresponding position of the keyboard in the virtual environment of the virtual reality device in response to the recognition of the keyboard in the environment image;

a desktop creation module, configured to create a virtual desktop at a corresponding position of the target plane in the virtual environment in response to the virtual keyboard being placed on the target plane in the at least one plane;

an input acquisition module, configured to acquire an input signal of the mouse;

The ray creation module is configured to generate a selection ray corresponding to the mouse in the virtual environment based on the input signal of the mouse.

Optionally, the ray building module is used for:

acquiring the plane coordinates of the mouse based on the input signal of the mouse;

transforming the plane coordinates into world coordinates in the virtual environment;

The selection ray is generated based on the world coordinates and the orientation of the virtual camera in the virtual environment, the selection ray passes through the world coordinates, and the direction is parallel to the orientation of the virtual camera.

According to another aspect of the embodiments of the present application, a virtual reality device is provided, the virtual reality device includes a processor and a memory, and at least one instruction, at least one program, a code set or an instruction set are stored in the memory, the At least one instruction, the at least one program, the code set or the instruction set are loaded and executed by the processor to implement the above-mentioned virtual reality device control method.

According to another aspect of the embodiments of the present application, a computer storage medium is provided, wherein at least one instruction, at least one section of program, code set or instruction set is stored in the computer storage medium, and the at least one instruction, the at least one section of program . The code set or instruction set is loaded and executed by the processor to implement the control method of the virtual reality device as described above.

According to another aspect of the embodiments of the present application, a computer program product or computer program is provided, the computer program product or computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the methods provided in the various optional implementation manners above.

The beneficial effects brought by the technical solutions provided by the embodiments of the present application at least include:

By displaying at least one plane in the detected external environment in the virtual environment of the virtual reality device, and generating a virtual keyboard corresponding to the keyboard in the virtual environment, and then establishing a virtual desktop at the position of the plane where the virtual keyboard is placed, and then placing the mouse The input signal of the input signal is converted into a selection ray in the virtual environment, so that the virtual environment can be controlled through an external mouse and keyboard, reducing the possibility of false touch, and achieving the effect of improving the accuracy of the control method of the virtual reality device .

In addition, in the method provided by the embodiment of the present application, the input signal of the mouse is converted to the corresponding selection ray in the virtual environment, so that the user can directly use the mouse commonly used in office to perform corresponding control in the virtual environment without using a virtual reality device The dedicated handle saves costs, reduces user learning costs, and improves user experience.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings that need to be used in the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application. For those skilled in the art, other drawings can also be obtained based on these drawings without creative effort.

FIG. 1 is a schematic diagram of an implementation environment in an embodiment of the present application;

Fig. 2 is a method flowchart of a method for controlling a virtual reality device provided by an embodiment of the present application;

Fig. 3 is a method flow chart of another control method of a virtual reality device provided by an embodiment of the present application;

Fig. 4 is a schematic diagram of a virtual environment in the method provided by the embodiment of the present application;

Fig. 5 is a block diagram of a control device for a virtual reality device provided by an embodiment of the present application.

By means of the above drawings, specific embodiments of the present application have been shown, which will be described in more detail hereinafter. These drawings and text descriptions are not intended to limit the scope of the concept of the application in any way, but to illustrate the concept of the application for those skilled in the art by referring to specific embodiments.

Detailed ways

In order to make the purpose, technical solution and advantages of the present application clearer, the implementation manners of the present application will be further described in detail below in conjunction with the accompanying drawings.

The control method of the virtual reality device provided by the embodiment of the present application can be applied to the virtual reality device, as shown in Figure 1, which is a schematic diagram of an implementation environment in the embodiment of the application, and the implementation environment can include a virtual reality It is assumed that the virtual reality device may include a device main body 11 , a keyboard 12 and a mouse 13 , and an image acquisition component 111 is provided on the device main body 11 .

Wherein, the device main body 11 may be various head-mounted virtual reality devices such as virtual reality (Virtual Reality, VR) glasses, virtual reality helmets, and the like. The device main body 11 can be a six degrees of freedom (6DOF) virtual reality device, so that users can have a better experience. The device main body 11 has an image acquisition component 111 , and the image acquisition component 111 can face directly in front of the user wearing the device main body 11 , so as to acquire images of the environment directly in front of the user. The image acquisition component 111 may include a visible light camera, a depth camera, and the like. The device main body 11 may have a display screen, and the user can obtain a virtual reality experience by viewing the display screen.

The keyboard 12 can establish a wireless connection or a wired connection with the device main body 11 , for example, the keyboard 12 can establish a Bluetooth connection with the device main body 11 . The keyboard 12 may be a keyboard of a preset model, and a pre-established model of the keyboard of the preset model may be stored in the device main body 11 .

The mouse 13 can establish a wireless connection or a wired connection with the device main body 11 , for example, the mouse 13 can establish a Bluetooth connection with the device main body 11 . The mouse 13 may also be a mouse of a preset model, and a pre-established model of the mouse of the preset model may be stored in the device main body 11 .

A keyboard 12 and a mouse 13 may be located on the desktop D. When the keyboard 12 and the mouse 13 are wirelessly connected to the device main body 11, the delay can be within 20 milliseconds to ensure user experience.

In addition, the virtual reality device 10 can also include a virtual reality host, which can serve as a signal source for the device main body 11 and provide control information and processed image data for the device main body 11 . The virtual reality host can be partly or fully set in the device main body 11 , or can be located outside the device main body 11 . The virtual reality host may have a control component for controlling the device main body 11 .

The method for controlling a virtual reality device provided in the embodiment of the present application can provide a user with an application scene combining virtual reality, and the user can use the scene for various activities such as office work and entertainment.

Fig. 2 is a method flowchart of a method for controlling a virtual reality device provided by an embodiment of the present application, the method can be used for the virtual reality device shown in Fig. 1, and the method may include the following steps:

Step 201, acquire an environmental image of the external environment of the main body of the device through an image acquisition component.

Step 202, perform plane detection in the environment image.

Step 203, displaying the detected at least one plane in the virtual environment of the virtual reality device.

Step 204: In response to recognizing the keyboard in the environment image, generate a virtual keyboard at a corresponding position of the keyboard in the virtual environment of the virtual reality device.

Step 205: In response to the virtual keyboard being placed on the target plane in at least one plane, create a virtual desktop at a corresponding position of the target plane in the virtual environment.

Step 206, acquiring an input signal of the mouse.

Step 207, based on the input signal of the mouse, generate a selection ray corresponding to the mouse in the virtual environment.

To sum up, the control method of the virtual reality device provided by the embodiment of the present application displays at least one plane in the detected external environment in the virtual environment of the virtual reality device, and generates a virtual plane corresponding to the keyboard in the virtual environment. Keyboard, and then establish a virtual desktop at the position of the plane where the virtual keyboard is placed, and then convert the input signal of the mouse into a selection ray in the virtual environment, so that the virtual environment can be controlled through the external mouse and keyboard, reducing false touches possibility, the effect of improving the accuracy of the control method of the virtual reality device is achieved.

In addition, in the method provided by the embodiment of the present application, the input signal of the mouse is converted to the corresponding selection ray in the virtual environment, so that the user can directly use the mouse commonly used in office to perform corresponding control in the virtual environment without using a virtual reality device The dedicated handle saves costs, reduces user learning costs, and improves user experience.

Fig. 3 is a method flow chart of another method for controlling a virtual reality device provided by an embodiment of the present application. This method can be used for the virtual reality device shown in Fig. 1, and the method may include the following steps:

Step 301, acquire an environmental image of the external environment of the main body of the device through an image acquisition component.

The image acquisition component can continuously acquire the environment image of the external environment of the device body. The environment image is an image of the environment around the user wearing the device body, which can facilitate the device body to display the virtual environment of the virtual reality device based on the image. Exemplarily, when the image acquisition component faces the front of the device body (the front of the device body may be in the same direction as the front of the user wearing the device body), the environment image may reflect the front of the user wearing the device body. environment, and display corresponding content in front of the user in the virtual environment based on the environment image.

The method provided by the embodiment of the present application can be applied to a control component of a virtual reality device, and the control component can be located in the device main body, or can also be located outside the device main body. When the control component is located outside the device main body, the control component establishes a wireless connection or a wired connection with the device main body.

Step 302, perform plane detection in the environment image.

The control component can perform plane detection in the environment image acquired by the image acquisition component, so as to detect the plane existing in the external environment of the subject. Since the external environment of the detection subject is a real environment, it is usually relatively complex, and there will be many planes, such as desktops, floors, walls, and ceilings, etc. After performing plane detection, multiple planes can be detected.

In an exemplary embodiment, the control component may perform plane detection in the environment image through a random sampling consensus algorithm (RANdom SAmple Consensus, RANSAC), a Hough transform algorithm, or a region growing clustering algorithm.

The random sampling consensus algorithm is an algorithm that calculates the mathematical model parameters of the data based on a set of sample data sets containing abnormal data, thereby obtaining effective sample data. Random sampling consensus algorithms are widely used in computer vision. When applying the random sampling consensus algorithm for plane detection, the random sampling consensus algorithm can be used cyclically to perform plane detection for environmental patterns. When the loop termination condition is reached (such as the loop to the plane area is the set minimum threshold), the loop random sampling is terminated. consensus algorithm.

The Hough transform algorithm is a feature extraction algorithm. The Hough transform algorithm is widely used in image analysis, computer vision and digital image processing. The Hough transform algorithm can be used to identify various features in an image. In the embodiment of the present application, a Hough transform algorithm may be applied to detect planes in the environment image.

The region growing clustering algorithm is a network segmentation algorithm based on region growing, which belongs to a kind of local greedy algorithm. In the embodiment of the present application, a region growing clustering algorithm may be applied to detect planes in the environment image.

For the specific process of plane detection through the random sampling consensus algorithm, the Hough transform algorithm, or the region growing clustering algorithm, reference may be made to related technologies, and the embodiments of the present application will not repeat them here.

Step 303: Display the detected at least one plane in the virtual environment of the virtual reality device.

The control component can display the detected at least one plane in the virtual environment of the virtual reality device, so that the user can see the detected at least one plane in the virtual environment. The control component can display the detected at least one plane at a corresponding position in the virtual environment, and then the at least one plane can reflect the positions of various planes and objects in the real environment, so that it is convenient for the user to At least one plane is used to know the positions of various planes in the real environment. On the one hand, it can facilitate the user to determine the plane used as the desktop during office work based on the displayed plane, and on the other hand, it can prevent the user from colliding with these planes.

In the virtual environment, planes can be displayed in various ways, for example, planes can be marked by grid graphics, or planes can be marked by models, planes can be marked by dot matrix, or planes can be marked by A plurality of cross-shaped patterns arranged in an array are used to mark planes and the like, which is not limited in this embodiment of the present application.

Step 304: In response to recognizing the keyboard in the environment image, detect whether the keyboard has any one of a plurality of preset characteristics.

The control component can perform keyboard detection in the environment image. Since the keyboard has multiple keys arranged in an array, and the keys have letters or symbols, the overall characteristics are more obvious. When the keyboard appears within the field of view of the image acquisition component, The control component can recognize the keyboard based on the environment image collected by the image acquisition component, and track the keyboard in real time.

The control component can detect whether the keyboard has any one of the preset characteristics when the keyboard is detected in the environment image. Wherein, the feature includes at least one of a preset key arrangement, a preset optical mark, and a preset shape. The preset feature may be one or more predetermined features of the keyboard.

Step 305, when the keyboard has any feature, obtain a keyboard model corresponding to any feature.

The keyboard model corresponding to each preset feature can be pre-stored in the device body, and when the control component recognizes that the keyboard has any feature, it can obtain the keyboard model corresponding to any feature from the local storage of the device body.

Exemplarily, the above feature is a preset optical mark, and the optical mark can specifically be a mark of a keyboard model. When the control component detects that the keyboard has a mark of one of the keyboard models, the keyboard model can be obtained from the local storage The keyboard model corresponding to the tag of .

In an exemplary embodiment, the fine model of the one or more keyboards may be pre-stored in the main body of the device. The fine model has a high degree of fineness and may have corresponding special effects, such as lighting effects and animations. effects, etc., which is conducive to improving user experience.

In addition, the control component may generate a first model of the keyboard based on the recognized image of the keyboard when the keyboard does not have preset features, and load the first model at the corresponding position of the keyboard in the virtual environment of the virtual reality device. The fineness of the first model may be lower than that of the keyboard model pre-stored in the main body of the device, so as to reduce the computing pressure of the processing component.

Step 306, loading the keyboard model at the corresponding position of the keyboard in the virtual environment of the virtual reality device.

After the control component acquires the keyboard model, it can load the keyboard model at the corresponding position of the keyboard in the virtual environment of the virtual reality device. When the image acquisition component continues to acquire the environment image, the control component can load the keyboard model in real time at the corresponding position in the virtual environment of the virtual reality device (the corresponding position can refer to the corresponding position of the keyboard in the real environment in the virtual environment), In this case, the effect of projecting the keyboard in the real environment into the virtual environment is realized. When the user picks up the keyboard in the real environment and moves the keyboard, the keyboard in the virtual environment will also move accordingly.

In an exemplary embodiment, the control component can determine the position and posture of the keyboard in the virtual environment through the environment image, and load the keyboard model in the virtual environment based on the position and posture, so as to realize more accurate movement of the keyboard model, Improve user experience.

Step 307: In response to the virtual keyboard being placed on the target plane in at least one plane, create a virtual desktop at a corresponding position of the target plane in the virtual environment.

When the control component detects that the virtual keyboard is placed on the target plane in at least one plane, indicating that the target plane is the plane selected by the user as the office desktop, the control component can establish the corresponding position of the target plane in the virtual environment virtual desktop. The virtual desktop may have a desk-like appearance, or the control component may pre-store multiple virtual desktop models (different virtual desktop models have different shapes and materials), and the user may select one of them.

At the end of step 307, the method provided by the embodiment of the present application provides the user with a virtual desktop and a virtual keyboard located on the virtual desktop. The virtual keyboard can be a projection of a physical keyboard in the display environment, and the user operates the physical keyboard in the display environment , the virtual keyboard can give corresponding feedback, and the control component can obtain the information input by the user, and perform corresponding control based on the information. Moreover, when the user operates the physical keyboard, the possibility of false touch is low, so that the accuracy of the method can be improved.

Step 308: In response to recognizing the mouse in the environment image, generate a virtual mouse at a corresponding position of the mouse in the virtual environment of the virtual reality device.

The control component can recognize the mouse in the environment image, and when the mouse is recognized, generate a virtual mouse at the corresponding position of the mouse in the virtual environment of the virtual reality device. When the image acquisition component continues to acquire environmental images, the control component can load the mouse model in real time at the corresponding position in the virtual environment of the virtual reality device (the corresponding position can refer to the corresponding position of the mouse in the real environment in the virtual environment), In this case, the effect of projecting the mouse in the real environment into the virtual environment is realized. When the user holds the mouse in the real environment and moves and operates the mouse, the mouse in the virtual environment will also move accordingly. In this way, the effect that the user operates the mouse synchronously in the real environment and the virtual environment is realized, and the user experience is improved.

Step 309, acquiring the input signal of the mouse.

The control component can obtain the input signal of the mouse through the connection with the mouse. The input signal of the mouse may include a position signal of the mouse. In addition, the input signal may also include a key signal.

Step 310, acquiring the plane coordinates of the mouse based on the input signal of the mouse.

The control component can obtain the plane coordinates of the mouse based on the input signal of the mouse. The plane coordinates can record the coordinates of the mouse in the desktop plane coordinate system, and the plane coordinate system can be the pixel coordinate system on the display screen of the device main body, and the range of the coordinate system can be (0,0) to (Sw, Sh ), Sw is the pixel width of the display screen, and Sh is the pixel height of the display screen.

Step 311 , transform the plane coordinates into world coordinates in the virtual environment.

Control components can convert plane coordinates to world coordinates in the virtual environment. Exemplarily, the control component can implement the conversion process through a rendering engine.

Step 312, generating a selection ray based on the world coordinates and the orientation of the virtual camera in the virtual environment.

After the control component acquires the world coordinates corresponding to the mouse position, it can generate a selection ray based on the world coordinates and the orientation of the virtual camera in the virtual environment. Among them, the orientation of the virtual camera can be determined by the orientation of the device body, the selection ray passes through the world coordinates, and the direction is parallel to the orientation of the virtual camera, that is, the selection ray can be determined by the position of the mouse and the orientation of the virtual camera .

Step 313, performing collision detection on the selected ray and objects in the virtual environment.

After the control component generates the selection ray, it can perform collision detection on the selection ray and objects in the virtual environment. Wherein, the objects in the virtual environment may be various objects displayed during office work, such as one or more screens and three-dimensional models.

Step 314, determining the first object collided by the selection ray as the object currently selected by the mouse.

The control component can determine the first object collided by the selection ray as the object currently selected by the mouse. In an exemplary embodiment, the object currently selected by the mouse can be highlighted in the virtual environment first, and when a certain instruction from the mouse or keyboard is received (the instruction can be clicked on a button of the mouse, or a specified button on the keyboard When it is clicked), further operations can be performed on the selected object, such as opening operations, etc.

Please refer to FIG. 4. FIG. 4 is a schematic diagram of a virtual environment in the method provided by the embodiment of the present application, wherein the virtual environment includes a virtual desktop d, a virtual keyboard k and a virtual mouse m located on the virtual desktop d, the virtual desktop d. Virtual keyboard k and virtual mouse m are the projections of the desktop, keyboard and mouse in the real environment. The virtual environment also includes virtual display screens s1, s2 and s3. These three virtual display screens can be used to display office-related In addition, the virtual environment also displays a selection ray L corresponding to the mouse, and the selection ray L can be used to select the content displayed on the display screens s1, s2, and s3, or other objects in the virtual environment.

In this way, the operation of the mouse on the plane is realized to realize the effect of controlling the three-dimensional virtual environment, and the user does not need to use a dedicated virtual reality handle, which saves equipment costs and user learning costs.

To sum up, the control method of the virtual reality device provided by the embodiment of the present application displays at least one plane in the detected external environment in the virtual environment of the virtual reality device, and generates a virtual plane corresponding to the keyboard in the virtual environment. Keyboard, and then establish a virtual desktop at the position of the plane where the virtual keyboard is placed, and then convert the input signal of the mouse into a selection ray in the virtual environment, so that the virtual environment can be controlled through the external mouse and keyboard, reducing false touches possibility, the effect of improving the accuracy of the control method of the virtual reality device is achieved.

In addition, in the method provided by the embodiment of the present application, the input signal of the mouse is converted to the corresponding selection ray in the virtual environment, so that the user can directly use the mouse commonly used in office to perform corresponding control in the virtual environment without using a virtual reality device The dedicated handle saves costs, reduces user learning costs, and improves user experience.

The following are device embodiments of the present disclosure, which can be used to implement the method embodiments of the present disclosure. For details not disclosed in the disclosed device embodiments, please refer to the disclosed method embodiments.

Fig. 5 is a block diagram of a control device for a virtual reality device provided by an embodiment of the present application. The device can be implemented as part or all of the control component of the virtual reality device through software, hardware or a combination of the two. The control device 500 of the virtual reality device includes:

An image acquisition module 510, configured to acquire an environmental image of the external environment of the device main body through an image acquisition component;

A plane detection module 520, configured to perform plane detection in the environment image;

A plane display module 530, configured to display at least one detected plane in the virtual environment of the virtual reality device;

A keyboard generation module 540, configured to generate a virtual keyboard at a corresponding position of the keyboard in the virtual environment of the virtual reality device in response to recognizing the keyboard in the environment image;

A desktop creation module 550, configured to create a virtual desktop at a corresponding position of the target plane in the virtual environment in response to the virtual keyboard being placed on the target plane in at least one plane;

Input obtaining module 560, is used for obtaining the input signal of mouse;

The ray creation module 570 is configured to generate a selection ray corresponding to the mouse in the virtual environment based on the input signal of the mouse.

To sum up, the control device of the virtual reality device provided by the embodiment of the present application displays at least one plane in the detected external environment in the virtual environment of the virtual reality device, and generates a virtual plane corresponding to the keyboard in the virtual environment. Keyboard, and then establish a virtual desktop at the position of the plane where the virtual keyboard is placed, and then convert the input signal of the mouse into a selection ray in the virtual environment, so that the virtual environment can be controlled through the external mouse and keyboard, reducing false touches possibility, the effect of improving the accuracy of the control method of the virtual reality device is achieved.

In addition, the control device for virtual reality equipment provided by the embodiment of the present application converts the input signal of the mouse to the corresponding selection ray in the virtual environment, so that the user can directly use the mouse commonly used in office to perform corresponding control in the virtual environment without the need for Using a special handle for virtual reality equipment saves costs, reduces user learning costs, and improves user experience.

Optionally, ray builds modules for:

Obtain the plane coordinates of the mouse based on the input signal of the mouse;

Convert plane coordinates to world coordinates in the virtual environment;

The selected ray is determined based on the world coordinates and the orientation of the virtual camera in the virtual environment, the selected ray passes through the world coordinates, and the direction is parallel to the orientation of the virtual camera.

Optionally, the control device of the virtual reality device also includes:

A collision detection module, used for collision detection between the selected ray and objects in the virtual environment;

The selection module is used to determine the first object collided by the selection ray as the object currently selected by the mouse.

Optionally, the control device of the virtual reality device also includes:

The virtual mouse generating module is configured to generate a virtual mouse at a corresponding position of the mouse in the virtual environment of the virtual reality device in response to recognizing the mouse in the environment image.

Optionally, a plane detection module for:

Plane detection in environment images by random sampling consensus algorithm, Hough transform algorithm or region growing clustering algorithm.

Optionally, a keyboard generation module for:

Detecting whether the keyboard has any one of a plurality of preset characteristics;

When the keyboard has any feature, obtain the keyboard model corresponding to any feature;

The keyboard model is loaded at the corresponding position of the keyboard in the virtual environment of the virtual reality device.

Optionally, the feature includes at least one of a preset key arrangement, a preset feature, and a preset shape.

Optionally, a keyboard generation module for:

Determine the position and posture of the keyboard in the virtual environment through the environment image;

Load the keyboard model in the virtual environment based on position and pose.

Optionally, the control device of the virtual reality device also includes:

A model generation module, configured to generate a first model of the keyboard based on the image of the keyboard when the keyboard does not have preset features;

The model loading module is configured to load the first model at the corresponding position of the keyboard in the virtual environment of the virtual reality device.

To sum up, the control device of the virtual reality device provided by the embodiment of the present application displays at least one plane in the detected external environment in the virtual environment of the virtual reality device, and generates a virtual plane corresponding to the keyboard in the virtual environment. Keyboard, and then establish a virtual desktop at the position of the plane where the virtual keyboard is placed, and then convert the input signal of the mouse into a selection ray in the virtual environment, so that the virtual environment can be controlled through the external mouse and keyboard, reducing false touches possibility, the effect of improving the accuracy of the control method of the virtual reality device is achieved.

In addition, the control device for virtual reality equipment provided by the embodiment of the present application converts the input signal of the mouse to the corresponding selection ray in the virtual environment, so that the user can directly use the mouse commonly used in office to perform corresponding control in the virtual environment without the need for Using a special handle for virtual reality equipment saves costs, reduces user learning costs, and improves user experience.

In addition, the embodiment of the present application also provides a virtual reality device, the virtual reality device includes a processor and a memory, at least one instruction, at least one program, code set or instruction set are stored in the memory, at least one instruction, at least one program, code The set or instruction set is loaded and executed by the processor to implement the control method of the above-mentioned virtual reality device.

The embodiment of the present application also provides a computer storage medium, at least one instruction, at least one program, code set or instruction set is stored in the computer storage medium, at least one instruction, at least one program, code set or instruction set is loaded by the processor and Execute to realize the above-mentioned virtual reality device control method.

The embodiment of the present application also provides a computer program product or computer program, where the computer program product or computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the methods provided in the various optional implementation manners above.

The term "at least one of A and B" in this application is just an association relationship describing associated objects, which means that there may be three relationships. For example, at least one of A and B can mean: A exists alone, and at the same time There are three cases of A and B, and B alone. Similarly, "at least one of A, B, and C" means that seven relationships can exist, which can mean: A exists alone, B exists alone, C exists alone, A and B exist simultaneously, A and C exist simultaneously, and C and B, there are seven situations of A, B and C at the same time. Similarly, "at least one of A, B, C, and D" means that there can be fifteen relationships, which can mean: A exists alone, B exists alone, C exists alone, D exists alone, A and B exist simultaneously, and A and C, A and D, C and B, D and B, C and D, A, B, and C, A, B, and D, A, C and D, B, C, and D exist at the same time, and A, B, C, and D exist at the same time, these fifteen situations.

In the present application, the term "first" is used for descriptive purpose only, and cannot be understood as indicating or implying relative importance. The term "plurality" means two or more, unless otherwise clearly defined.

In the several embodiments provided in this application, it should be understood that the disclosed devices and methods may be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or May be integrated into another system, or some features may be ignored, or not implemented. In another point, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

Those of ordinary skill in the art can understand that all or part of the steps for implementing the above embodiments can be completed by hardware, and can also be completed by instructing related hardware through a program. The program can be stored in a computer-readable storage medium. The above-mentioned The storage medium mentioned may be a read-only memory, a magnetic disk or an optical disk, and the like.

The above are only optional embodiments of the application, and are not intended to limit the application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the application shall be included in the protection of the application. within range.

Claims (13)

1. A control method of a virtual reality device is used for the virtual reality device, the virtual reality device comprises a device body, a keyboard and a mouse, an image acquisition assembly is arranged on the device body, and the method comprises the following steps:

acquiring an environment image of an environment outside the device main body through the image acquisition assembly;

performing plane detection in the environment image;

presenting the detected at least one plane in a virtual environment of the virtual reality device;

in response to identifying the keyboard in the environmental image, generating a virtual keyboard at a corresponding location of the keyboard in a virtual environment of the virtual reality device;

responsive to the virtual keyboard being placed on a target plane of the at least one plane, establishing a virtual desktop at a corresponding location of the target plane in the virtual environment;

acquiring an input signal of the mouse;

and generating a selection ray corresponding to the mouse in the virtual environment based on the input signal of the mouse.

2. The method of claim 1, wherein generating the selection ray corresponding to the mouse in the virtual environment based on the input signal of the mouse comprises:

acquiring a plane coordinate of the mouse based on the input signal of the mouse;

converting the planar coordinates to world coordinates in the virtual environment;

generating the selection ray based on the world coordinates and an orientation of a virtual camera in the virtual environment, the selection ray passing through the world coordinates and oriented parallel to the orientation of the virtual camera.

3. The method of claim 2, wherein after generating the selected ray based on the world coordinates and an orientation of a virtual camera in the virtual environment, the method further comprises:

performing collision detection on the selected ray and an object in the virtual environment;

and determining the first object collided by the selection ray as the object currently selected by the mouse.

4. The method of claim 1, wherein before the obtaining the input signal of the mouse, the method further comprises:

in response to identifying the mouse in the environmental image, generating a virtual mouse at a corresponding location of the mouse in a virtual environment of the virtual reality device.

5. The method of claim 1, wherein the performing plane detection in the environment image comprises:

and carrying out plane detection in the environment image through a random sampling consistency algorithm, a Hough transform algorithm or a region growing clustering algorithm.

6. The method of claim 1, wherein generating a virtual keyboard at a corresponding location of the keyboard in a virtual environment of the virtual reality device comprises:

detecting whether the keyboard has any one of a plurality of preset characteristics;

when the keyboard has any one feature, acquiring a keyboard model corresponding to the any one feature;

and loading the keyboard model at the corresponding position of the keyboard in the virtual environment of the virtual reality equipment.

7. The method of claim 6, wherein the characteristic comprises at least one of a predetermined key arrangement, a predetermined characteristic, and a predetermined shape.

8. The method of claim 6, wherein the loading the keyboard model at a corresponding location of the keyboard in the virtual environment of the virtual reality device comprises:

determining the position and the posture of the keyboard in the virtual environment through the environment image;

loading the keyboard model in the virtual environment based on the position and the pose.

9. The method of claim 6, further comprising:

generating a first model of the keyboard based on an image of the keyboard when the keyboard does not have the preset features;

loading the first model at a corresponding location of the keyboard in a virtual environment of the virtual reality device.

10. The utility model provides a controlling means of virtual reality equipment, its characterized in that, virtual reality equipment includes equipment principal, keyboard and mouse, the equipment principal is last to have the image acquisition subassembly, the controlling means of virtual reality equipment includes:

the image acquisition module is used for acquiring an environment image of the external environment of the equipment main body through the image acquisition assembly;

the plane detection module is used for carrying out plane detection in the environment image;

a plane presentation module for presenting the detected at least one plane in a virtual environment of the virtual reality device;

a keyboard generation module for generating a virtual keyboard at a corresponding position of the keyboard in a virtual environment of the virtual reality device in response to identifying the keyboard in the environment image;

a desktop establishing module, configured to, in response to the virtual keyboard being placed on a target plane of the at least one plane, establish a virtual desktop at a corresponding location of the target plane in the virtual environment;

the input acquisition module is used for acquiring an input signal of the mouse;

and the ray establishing module is used for generating a selection ray corresponding to the mouse in the virtual environment based on the input signal of the mouse.

11. The control apparatus of a virtual reality device according to claim 10, wherein the ray establishing module is configured to:

acquiring a plane coordinate of the mouse based on the input signal of the mouse;

converting the planar coordinates to world coordinates in the virtual environment;

generating the selection ray based on the world coordinates and an orientation of a virtual camera in the virtual environment, the selection ray passing through the world coordinates and oriented parallel to the orientation of the virtual camera.

12. A virtual reality device comprising a processor and a memory, the memory having stored therein at least one instruction, at least one program, a set of codes or a set of instructions, the at least one instruction, the at least one program, the set of codes or the set of instructions being loaded and executed by the processor to implement the method of controlling a virtual reality device according to any one of claims 1 to 9.

13. A computer storage medium having stored therein at least one instruction, at least one program, a set of codes, or a set of instructions, which is loaded and executed by a processor to implement a method of controlling a virtual reality device according to any one of claims 1 to 9.

Images