CN116126147A - Virtual scene control method and device - Google Patents

Abstract

The application provides a virtual scene control method and device, wherein in the application, the real hand operation of a real user is detected, and the real hand operation is mapped into the virtual hand operation of a virtual user in a virtual scene image; if the hand gesture of the real hand operation belongs to the set object holding gesture for maintaining the object pose, acquiring voice information input by a real user; identifying a target object described in the voice information; adding a virtual target object to a virtual hand of the virtual user that makes the virtual hand operation.

Description

Virtual scene control method and device

technical field

The present application relates to the technical field of image processing, in particular to a virtual scene control method and device.

Background technique

In a virtual scene based on virtual technologies such as virtual reality or augmented reality, an input operation of a real user in a physical scene may be converted into an input operation of a virtual user in a virtual scene.

However, in the process of mapping real users to virtual users, the actions of real users can only be converted into actions related to virtual users in the virtual scene, but the objects held or expected to be displayed by real users cannot be presented on virtual users. As a result, the current virtual scene cannot more accurately present the interactive scene expected by the user, and the display flexibility is poor.

Contents of the invention

On the one hand, the application provides a virtual scene control method, including:

Detecting the real hand operation of the real user, mapping the real hand operation to the virtual hand operation of the virtual user in the virtual scene image;

If the hand gesture of the real hand operation belongs to the set object holding posture for maintaining the object pose, obtain the voice information input by the real user;

identifying the target object described in the voice information;

The virtual target object is added to the virtual hand of the virtual user performing the virtual hand operation.

In a possible implementation manner, if the hand gesture of the real hand operation belongs to the set object holding gesture for maintaining the pose of the object, obtaining the voice information input by the real user includes:

If the hand gesture of the real hand operation belongs to the set object-holding posture for maintaining the object posture, and the time length of the real hand operation hand posture maintained in the object-holding posture exceeds the set duration, the obtained Voice information input by real users.

In yet another possible implementation manner, the identifying the target object described in the voice information includes:

Recognizing the object category and object characteristics of the target object described in the voice information;

Before adding the virtual target object to the virtual hand of the virtual user performing the virtual hand operation, it also includes:

A virtual target object is constructed based on the object category and object features of the target object.

In yet another possible implementation manner, the mapping the real hand operation to the virtual hand operation of the virtual user in the virtual scene image includes:

determining the input gesture type of the real hand operation;

Based on the type of input gesture, determine the input sensitivity mapped from the real hand operation to the virtual input operation in the virtual scene image;

According to the input sensitivity, the real hand operation is mapped to the virtual hand operation of the virtual user in the virtual scene image.

In yet another possible implementation manner, the mapping the real hand operation to the virtual hand operation of the virtual user in the virtual scene image according to the input sensitivity includes:

According to the input sensitivity, determine the distance mapping relationship between the moving distance of the real hand operation and the virtual moving distance of the virtual hand in the virtual scene image;

According to the distance mapping relationship and the movement track of the real hand operation, the real hand operation is mapped to the virtual hand operation of the virtual user in the virtual scene image.

In yet another possible implementation manner, the determining the input gesture type of the real hand operation includes:

determining a finger gesture for indicating a gesture type in said real hand manipulation;

Based on the gesture of the finger, an input gesture type of the real hand operation is determined.

In yet another possible implementation manner, the determining the finger gesture used to indicate the gesture type in the real hand operation includes:

If the real hand operation belongs to a hand movement operation, determining a gesture of a finger used to indicate a gesture type in the real hand operation.

In yet another possible implementation manner, before adding the virtual target object to the virtual hand of the virtual user performing the virtual hand operation, the method further includes:

determining object characteristics of the target object based on the real hand operation of the real user;

A virtual target object is constructed based on the object characteristics of the target object.

In yet another aspect, the present application also provides a virtual scene control device, including:

An operation mapping unit, configured to detect a real hand operation of a real user, and map the real hand operation to a virtual hand operation of a virtual user in the virtual scene image;

A voice obtaining unit, configured to obtain the voice information input by the real user if the hand gesture of the real hand operation belongs to the set object holding posture for maintaining the object pose;

a speech recognition unit, configured to recognize the target object described in the speech information;

An object adding unit, configured to add the virtual target object to the virtual hand of the virtual user performing the virtual hand operation.

In a possible implementation manner, the operation mapping unit includes:

A type determining unit, configured to detect a real hand operation of a real user, and determine the input gesture type of the real hand operation;

A sensitivity determining unit, configured to determine the input sensitivity mapped from the real hand operation to the virtual input operation in the virtual scene image based on the input gesture type;

A mapping processing unit, configured to map the real hand operation to the virtual hand operation of the virtual user in the virtual scene image according to the input sensitivity.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the accompanying drawings that need to be used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings in the following description are only the embodiments of the present application. For Those of ordinary skill in the art can also obtain other drawings based on the provided drawings without making creative efforts.

Fig. 1 shows a schematic flow chart of a virtual scene control method provided by an embodiment of the present application;

Fig. 2 shows another schematic flow chart of the virtual scene control method provided by the embodiment of the present application;

Fig. 3 shows another schematic flow chart of the virtual scene control method provided by the embodiment of the present application;

Fig. 4 shows another schematic flow chart of the virtual scene control method provided by the embodiment of the present application;

Fig. 5 shows a schematic diagram of the composition and structure of the virtual scene control device provided by the embodiment of the present application;

FIG. 6 shows a schematic diagram of a composition structure of an electronic device provided by an embodiment of the present application.

Detailed ways

The solution of this application can be applied to any electronic device or a system composed of multiple electronic devices that needs to present virtual scenes with the help of virtual reality, augmented display or other types of virtual technologies, so as to more flexibly control the presentation of virtual scenes and improve the performance of virtual scenes. interactive flexibility.

The following will clearly and completely describe the technical solutions in the embodiments of the application with reference to the drawings in the embodiments of the application. Apparently, the described embodiments are only some of the embodiments of the application, not all of them. Based on the embodiments in this application, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the scope of protection of this application.

As shown in Fig. 1, it shows a schematic flow chart of the virtual scene control method provided by the embodiment of the present application. The method of this embodiment can be applied to any electronic device or virtual scene control system for virtual scene control.

The method of this embodiment may include:

S101. Detect a real hand operation of a real user, and map the real hand operation to a virtual hand operation of a virtual user in a virtual scene image.

Wherein, the real user refers to a real user in a physical scene. For ease of distinction, the hand operations of a real user in the physical world are called real hand operations, and the hand operations of a virtual user in a virtual scene image are called virtual hand operations.

It can be understood that, during the presentation of the virtual scene, the presentation of the virtual image in the virtual scene can be controlled or adjusted through the input operation of the real user. In this application, the virtual scene image at least includes the virtual user associated with the real user. On this basis, by capturing the action behavior of the real user, the action behavior of the real user can be converted into the response action behavior of the virtual user in the virtual scene image.

For example, in various scenarios such as digital communication scenarios or virtual e-commerce anchors, it may involve constructing virtual users in virtual scene images, and drive virtual users through real users. Of course, there may be other possibilities in the application scenario of this case, and this application does not limit it.

Wherein, there are many possible specific implementation manners for detecting the real hand operation of the real user. For example, a user image of a real user may be collected through a camera, and a real hand operation of the real user may be determined through the user image. For another example, a sensor worn on at least one part such as the hand of the real user may be used in combination with motion capture technology to obtain the real hand operation of the real user.

Of course, here are only two cases as examples for a brief description, and there are other possibilities to detect or capture real hand operations of real users in practical applications. In practical applications, multiple different technologies may be combined to detect real hand operations of real users, without limitation.

It can be understood that mapping real user's real hand operations to virtual user's virtual hand operations refers to transforming real user's hand operations into virtual user's virtual hand operations, so that the virtual user's virtual hand operations are consistent with the real The user's real hand operations are kept consistent, or a specific conversion rule is satisfied between the virtual hand operations and the real hand operations, and there is no limitation on this.

For example, the real hand operation is the operation of spreading the fingers of the hand and moving upwards, then by continuously mapping the real hand operation to the virtual hand operation at each moment, the virtual hand can also make the gesture of spreading the virtual hand. The operation of moving the finger up.

It should be noted that the real hand operation is a continuous operation action. Therefore, in practical applications, the real hand operation can be obtained irregularly or according to the sampling period, and the real hand operation obtained at different times can be continuously mapped It is a virtual hand operation, so that step S101 can be continuously performed multiple times.

S102, if the hand gesture of the real hand operation belongs to the set holding gesture for maintaining the pose of the object, obtain voice information input by the real user.

Wherein, the object-holding posture refers to a hand posture capable of maintaining an object in at least one state such as a certain position and posture.

For example, the holding posture can be the holding posture of the hand holding the object or other objects; The lifting posture of the object; of course, the holding posture can also be a posture for maintaining the object such as holding the object with the hands or holding it, and there is no limitation on this.

It is understandable that when the real user's hand gesture at a certain moment is the holding gesture, it means that the real user's hand is currently holding an object or other object, or the real user hopes to express the hand posture through the hand gesture. The effect of holding objects such as objects. Based on this, in order to accurately determine the object that the real user expects the virtual user's hand to hold, the application can further obtain the voice information of the real user, so that in the subsequent analysis, when the real hand has an object-holding posture, the real user wants to present Object.

S103. Identify the target object described in the voice information.

Wherein, the target object may be an object or a small animal that can be displayed by hands.

It can be understood that, in the case that the real hand gesture of the real user is an object-holding gesture, the object held by the real user's hand can be analyzed based on the voice information of the real user.

It can be understood that there may be many ways to recognize voice information, and this application does not limit the specific implementation of voice information recognition.

S104. Add a virtual target object to the virtual hand of the virtual user performing the virtual hand operation.

It can be understood that by adding a virtual target object to the virtual hand of the virtual user, the virtual user can appear to hold the target object, so that the virtual user in the virtual scene can present more realistically or flexibly Create the object-holding effect that real users want to achieve.

For ease of understanding, an application scenario is taken as an example:

Take scenarios such as capturing the actions of real users and driving virtual users through motion capture technology, and live broadcasting through virtual users as examples. In this scene, the expressions and actions of the real user can be vividly presented on the virtual user. However, in the case of items held by the real user, in order to accurately capture the actions Behavior, other backgrounds other than real users will be removed, so items held by real users cannot be captured.

However, in this application, when it is recognized that the real hand gesture of the real user is in the holding posture, by identifying the target object described in the voice information input by the real user, it can be determined that the user wants to be in the virtual user's hand position. For the item presented by the virtual hand, by adding the corresponding virtual item to the virtual user's virtual hand, it can show the effect that the virtual user also holds the item.

For example, if the real user's real hand is holding an apple, and the real user says "look at this apple", then it can be determined that the real user's hand is holding an apple. Correspondingly, when the virtual user makes a lift gesture Add an apple to the virtual hand.

Of course, here is an example of a real user holding an item. If the real user’s hand makes a holding gesture, but the real user does not actually hold the corresponding item, the real user wants the virtual user to appear to be holding the object. The effect of this item, as long as the real user describes the object that the hand wants to hold through language, the virtual user's hand can also achieve the effect of holding the corresponding item.

Wherein, there may be many specific ways of adding the virtual target object to the virtual hand of the virtual user.

In an optional manner, in order to present the virtual user's holding effect more realistically, a target object may be added to the virtual hand in combination with the category of the holding gesture corresponding to the virtual hand operation of the virtual hand.

For example, the virtual hand operation of the virtual hand corresponds to the holding posture of holding an object, then a virtual target object can be added to the center of the virtual hand to present the effect of the virtual hand holding the target object.

As another example, the virtual hand operation of the virtual hand corresponds to the holding posture of the pinching object, and a virtual target object can be added to the fingers of the virtual hand that make the pinching gesture, so as to show that the virtual user Improve the effect of the virtual target object.

As can be seen from the above, in the embodiment of the present application, while the real hand operation of the real user is mapped to the virtual hand operation of the virtual user in the virtual scene image, if the hand gesture of the real hand operation of the real user belongs to the set Based on the target object described in the voice information of the real user, the virtual target object can be added to the corresponding virtual hand of the virtual user, so that the virtual user in the virtual scene image can not only reflect the real user The action behavior of the real user can also be presented, so that the virtual scene image can more accurately present the scene expected by the real user, and the display flexibility of the virtual scene is improved.

It can be understood that, in order to be able to add a virtual target object to the virtual hand of the virtual user, the virtual target object can also be constructed first, wherein there are many possibilities for constructing the virtual target object, for example, Use artificial intelligence to automatically generate content (AI generated content, AIGC) technology to construct a virtual target object. Of course, there may be other ways of constructing the virtual target object, and there is no limitation on this.

It is understandable that the types of objects are diverse, and even the same object can be divided into different subcategories, and the objects of different subcategories also have their own characteristics, and even the same object has the same appearance characteristics. will vary. For example, apples can be divided into green apples and red apples, and different green apples have differences in size and color depth.

Based on this, in order to present the target object actually held or described by the real user more realistically, in this application, the object characteristics of the target object may be determined before constructing the virtual object.

The solution of the present application will be described below in conjunction with a manner of determining object characteristics of a target object. As shown in Figure 2, it shows a schematic flow chart of another embodiment of the virtual scene control method provided by the embodiment of the present application. The method of this embodiment may include:

S201. Detect a real hand operation of a real user, and map the real hand operation to a virtual hand operation of a virtual user in a virtual scene image.

S202. If the hand gesture of the real hand operation belongs to the set object holding gesture for maintaining the pose of the object, obtain voice information input by the real user.

For the above steps S201 and S202, reference may be made to the relevant introduction of the previous embodiments, and details are not repeated here.

S203. Identify the object category and object features of the target object described in the voice information.

Wherein, the object category of the target object may be the category to which the target object belongs. The object category of the target object can represent what kind of object the target object is, and may also represent the name of the object.

For example, if the object category of the target object is an apple among fruits, it may be determined that the target object is an apple. Of course, if the object category also includes apple category subdivision information, the type or name of the apple can be further determined. For example, if the object category includes the apple category "Huang Yuanshuai", then it can be determined that the apple is an apple of the type Huang Yuanshuai.

The object feature may be an attribute or characteristic of the object itself. For example, the object is an apple, and the characteristics of the apple may include the color of the peel, the texture of the peel, and the size of the apple.

Among them, there may also be various specific implementations for identifying object categories and object features from voice information, which is not limited in this application.

S204. Construct a virtual target object based on the object category and object features of the target object.

Similar to the above, in this embodiment, there may be various specific implementation manners for constructing the virtual target object, which is not limited.

For example, taking the target object as an apple, assuming that the real user is holding a big red apple in his hand, and the real user describes in words "Look at this apple, it is really big, round and red", then the target can be identified The object is a big, red and round apple, so that a big, round red apple can be constructed, so that the virtual user can also show the effect of holding a big red apple in his hand.

Or, when the real user makes the gesture of lifting the object, the hand does not hold the apple, but the real user hopes to include in the displayed virtual scene image: the hand holding the big red apple A virtual user, so that other users can more intuitively and truly understand the apple described by the real user by watching the virtual scene image. Then, it is actually used to describe in words "Look at this apple, it is really big, round and red" while making a gesture of lifting an object, then the solution based on this application will also construct a virtual big and round apple. A red round apple is placed on the corresponding hand of the virtual user to present the virtual scene image that the user wishes to present.

S205. Add the virtual target object to the virtual hand of the virtual user performing the virtual hand operation.

It can be understood that in this embodiment, when constructing a virtual target object, the object category and object characteristics of the target object are considered, so that the constructed virtual target object can be more suitable for the target object described by the real user, so that the real The objects actually held by the user or expected to be presented are accurately presented in the hands of the virtual user.

It can be understood that FIG. 2 is an example of a way of constructing a virtual target object. In practical applications, while or after the target object is recognized based on the voice information input by the real user, the object characteristics of the target object may also be determined based on the real hand operation of the real user. On this basis, a virtual target object is constructed based on the object characteristics of the target object.

It can be understood that when the real hand operation is an object-holding posture, the weight, size, and volume of the object held by the hand can also be reflected according to the category of the object-holding posture, and the gestures of the hands and fingers of the real hand operation And features such as softness, based on this, combined with the hand gesture features of real hand operations, the object features of the target object can also be determined.

For example, if the target object is an apple, and the real hand operation of the real user is the posture of holding the object, then according to the holding posture of the real hand operation, the size of the apple held can be analyzed, and the size of the apple can be obtained. a feature.

For another example, if the real hand operation is an embracing gesture of embracing an object, then the general posture and size of the object supported by the hand can also be analyzed according to the embracing gesture.

Of course, here are just a few simple examples. In practical applications, in addition to combining the gestures of the hands, arms, and fingers of real hand operations, the real user's body posture and other information can also be considered to comprehensively determine the target. Object characteristics of the object.

It is understandable that in this application, considering that the real hand operations of real users will be different at different times, it is very likely that the real user does not want to make an object-holding gesture at a certain moment, but is recognized due to a certain operation For holding posture. Based on this, in order to reduce misidentification, the present application can also make the hand gesture of the user's real hand operation belong to the set holding posture for maintaining the object pose, and the hand gesture of the real hand operation is maintained Only when the time in the holding posture exceeds the set time can the voice information input by the real user be obtained and the target object described in the voice information be recognized.

Wherein, the set duration can be set according to needs, for example, the set duration can be 3 seconds or 5 seconds.

For example, when a real user holds an item and wants to present the corresponding item through the virtual user, the real user will inevitably continue to hold the item for a period of time. On this basis, the item described by the voice information of the real user , the item can be added to the hand of the virtual user making a holding motion, so that the virtual user presents the effect of holding the item.

If the real user's hand shows a holding posture during other actions, but the holding posture does not last for a set period of time, it means that the user is not really holding the object, and there is no need to recognize the real user's voice information at this time The described items naturally do not need to be added to the hands of the virtual user.

It can be understood that there may be many possibilities in the specific realization of mapping the operation behavior of the real user to the virtual operation behavior of the virtual user, which is not limited in this application.

However, it is found through research that: under the premise of controlling the posture and action content of the virtual user's virtual operation behavior to be consistent with the real user's operation behavior, generally according to the set mapping ratio, the real user's operation behavior is mapped to the virtual user's same operation behavior. Type corresponding virtual operation behavior. For example, the proportional mapping method can be used to map the real operation behavior of the real user to the virtual operation behavior of the virtual user, so that the virtual operation behavior of the virtual user is consistent with the operation behavior of the real user in terms of action content, moving distance and action range. unanimous.

However, when the operating range of the virtual scene is large or precise continuous input is required, the intelligence maps the real action behavior to the virtual action action based on the set mapping ratio, resulting in poor controllable flexibility of the virtual scene and easy There is a case where input efficiency is low due to reasons such as input operation inconvenience.

For example, a scenario where a slider in an interactive interface in a virtual scene image needs to be dragged through an operation of a real user is taken as an example for illustration. Due to the large virtual space of the virtual scene image, the display area of the interactive interface is relatively large. Therefore, if the proportional mapping method is used, the real user needs to use the hand to make a sliding operation of dragging the slider for a long distance. It can be mapped to the virtual user in the virtual scene image to complete the operation of dragging the slider, so that the sliding distance of the sliding operation made by the real user is long and time-consuming, and the input efficiency is low.

Based on this, in the embodiment of the present application, various input sensitivities mapped from real hand operations to virtual hand operations in the virtual scene image may be set. Among them, the input sensitivity is different, and the mapping ratio from the real hand operation to the virtual hand operation is different. For example, in the case of the same real hand operation, the input sensitivity is different, and the range of motion changes mapped to the virtual hand operation is also different.

The solution of the present application will be described below by taking an implementation manner of determining the input sensitivity as an example. As shown in FIG. 3 , it shows a schematic flowchart of another embodiment of the virtual scene control method provided by the embodiment of the present application.

The method of this embodiment may include:

S301. Detecting a real hand operation of a real user, and determining an input gesture type of the real hand operation.

Wherein, the input gesture type is a gesture type corresponding to a gesture part that can reflect input sensitivity in a real hand operation. Correspondingly, the input sensitivity required by the real hand operation can be reflected by the input gesture type.

In this application, the type of input gestures is different from the type of control operations performed by real hand operations on the virtual scene. Therefore, according to the different requirements for input sensitivity, when the real user performs the same type of control operation on the virtual scene, what the real user does The types of input gestures corresponding to real hand operations of this type of control operations may also be different.

Among them, there are many possible specific implementation manners for determining the input gesture type, which will be described below in conjunction with several possible situations.

In a possible situation, after the real hand operation is obtained, the gesture of the finger used to indicate the gesture type in the real hand operation may be determined. Correspondingly, based on the gesture of the finger, the input gesture type of the real hand operation can be determined.

Wherein, the finger gesture indicating the gesture type is also the finger gesture indicating the input finger type. The gesture of the finger can be represented by multiple dimensions such as the gesture of the finger and the number of fingers, without limitation. In practical applications, a corresponding number of finger gestures can be set in combination with the number of types of input gestures.

for example:

Take the number of finger pinches to represent different input gesture types as an example.

Assuming that the input sensitivity is divided into three sensitivity levels: high, medium and low, correspondingly, the three levels of input sensitivity correspond to three different types of input gestures. On this basis, you can set a single finger as the first type of input gesture type corresponding to the high sensitivity level, pinch two fingers as the second type of input gesture type corresponding to the medium sensitivity level, and pinch three fingers as the corresponding low sensitivity level The third category of input gesture types.

On this basis, if two fingers are pinched together in the real hand operation by the real user, it can be determined that the input gesture type is the second type of input gesture type.

In yet another implementation, the real user can also set the input gesture type corresponding to the real hand operation to be input before making the real hand operation. For example, before inputting the real hand operation, the real The user pre-selects the desired input gesture type from a variety of gesture type options.

In practical applications, while obtaining the real hand operation, it is also possible to assist in determining the input gesture type corresponding to the real hand operation by means of body postures other than the hand. For example, the input gesture type corresponding to the real hand operation is determined by means of the number of times the head shakes or the number of consecutive eye blinks, etc. Of course, there may be other ways to determine the type of the input gesture, which is not limited.

S302. Based on the input gesture type, determine the input sensitivity mapped from the real hand operation to a virtual input operation in the virtual scene image.

Wherein, the input sensitivity may refer to the change degree of the virtual input operation generated by the virtual object (virtual user or other virtual controlled object) in the virtual scene image that can be triggered by the input operation of the real user. The higher the input sensitivity, the greater the operation change generated by the virtual object in the virtual scene image triggered by the input operation of the real user.

Based on this, the input sensitivity can affect the operation frequency and the operation range of the virtual input operation mapped from the real hand operation to the virtual scene image.

For example, in a possible situation, the input sensitivity may reflect the mapping relationship between the operation frequency of the real hand operation and the operation frequency of the virtual hand operation. For example, when the real hand operation is a click operation, the input sensitivity is different, and the number of clicks of the real hand operation mapped to the number of clicks in the virtual scene image will also be different.

In yet another possible situation, the input sensitivity is a distance mapping relationship between the moving distance of the real hand operation and the virtual moving distance of the virtual hand in the virtual scene image. Wherein, the distance mapping relationship is actually a kind of mapping relationship of the operating range, and the distance mapping relationship reflects the proportional relationship between the moving distance operated by the real hand and the moving distance of the virtual hand.

For example, if the distance mapping relationship is a one-to-one mapping relationship, then the moving distance of the real hand operation is the same as the virtual moving distance of the virtual hand. If the distance mapping relationship is two to one, then the moving distance of the real hand operation is twice the virtual moving distance of the virtual hand.

Wherein, if the input sensitivity is different, the distance mapping relationship will also be different. Generally, the higher the input sensitivity, the lower the ratio of the moving distance of the real hand operation to the virtual moving distance of the virtual hand, so that the real hand operation of the real user only needs a small movement to trigger the virtual hand to produce a large distance movement.

S303, according to the input sensitivity, map the real hand operation to the virtual hand operation of the virtual user in the virtual scene image.

From the previous introduction, it can be seen that the input sensitivity can affect the change degree of the virtual hand operation triggered by the real hand operation. Therefore, the input sensitivity is different, and the real hand operation can trigger the operation frequency of the virtual hand operation and the change range. One or more will also vary.

For example, when the input sensitivity is the distance mapping relationship between the moving distance of the real hand operation and the virtual moving distance of the virtual hand in the virtual scene image, the distance mapping relationship and the moving track of the real hand operation can be , to map the real hand manipulation to the virtual hand manipulation of the virtual user in the virtual scene image.

Of course, the input sensitivity can also be reflected in multiple dimensions such as operation frequency and operation range. In this case, this application can also consider the mapping from the operation frequency of real hand operations such as clicks to the operation frequency of virtual hand operations. to comprehensively determine the mapped virtual hand operation, which will not be repeated here.

It should be noted that in the operations of the above steps S301 to S303, it does not care whether the hand posture of the real hand operation is an object-holding posture, as long as the real hand operation is obtained, the operation of the real hand can be performed according to the above steps. The hand operation is mapped to the virtual hand operation of the virtual user.

S304, if the hand gesture of the real hand operation belongs to the set object-holding gesture for maintaining the pose of the object, obtain voice information input by the real user.

S305. Identify the target object described in the voice information.

S306. Add the virtual target object to the virtual hand of the virtual user performing the virtual hand operation.

For the above steps S304 to S306, reference may be made to the relevant introduction of the previous embodiments, and details are not repeated here.

In the embodiment of this application, after obtaining the real hand operation, the input sensitivity can be determined based on the input operation type of the real hand operation, so that the real user can reasonably control the input sensitivity according to the needs, and then realize the reasonable control of the real hand operation on the virtual The mapping relationship of hand operation improves the flexibility of virtual scene control. Moreover, the input sensitivity can be reasonably adjusted through the input operation type of the real hand operation, so as to reasonably adjust the mapping relationship between the real hand operation and the virtual hand operation, which can help reduce the input that exists in the process of finely controlling the virtual scene Errors can also reduce the situation that real users need to move their hands a long distance to complete the virtual input operation, which leads to input efficiency.

In order to facilitate the understanding of the specific implementation of determining the input sensitivity and mapping the real hand operation to the virtual hand operation based on the input sensitivity in this application, an implementation manner is taken as an example for description below. Considering that there is a movement operation of the real hand, combined with the input sensitivity, it is possible to more flexibly control the amount of movement change between the real hand mapped to the virtual hand. Therefore, there is a hand movement in combination with the real hand operation Be explained.

As shown in Figure 4, it shows another schematic flow chart of the virtual scene control method provided by the embodiment of the present application. The method of this embodiment may include:

S401. Detecting a real hand operation of a real user, and determining a finger gesture used to indicate a gesture type in the real hand operation if the real hand operation includes a hand movement operation.

Wherein, the real hand operation including the hand movement operation means that the real hand of the real user moves during the real user's real hand operation. For example, when the real user uses the hand to hold or lift an object, while the real user's hand moves upward, the real hand operation is an operation of presenting a lifting gesture and moving upward.

Since there is a hand movement operation in the real hand operation, the virtual hand must also move accordingly. Therefore, in order to determine the moving distance mapping relationship between the real hand and the virtual hand, it is necessary to determine the expected input sensitivity.

Among them, the gestures of the fingers in the real hand operation can be referred to the previous introduction. For example, in this application, three input sensitivities can be set. Correspondingly, the finger gestures corresponding to the three input finger types can be set. For example, the first The finger posture of the first input gesture type is a single finger extended, and the finger posture of the second input gesture type corresponding to the medium sensitivity is thumb and index finger pinching, and the finger posture of the third input gesture type corresponding to the low sensitivity is thumb, index finger and The middle fingers pinch each other.

S402. Based on the gesture of the finger, determine an input gesture type of a real hand operation.

S403. Based on the input gesture type, determine the input sensitivity mapped from the real hand operation to the virtual input operation in the virtual scene image.

It can be seen from the foregoing introduction that, in the case that the finger gesture belongs to the finger gesture corresponding to the set input finger type, the input gesture type to which the real hand operation belongs can be determined. corresponding. The input sensitivity can be determined according to the corresponding relationship between the input gesture type and the input sensitivity.

S404. According to the input sensitivity, determine a distance mapping relationship between the moving distance of the real hand operation and the virtual moving distance of the virtual hand in the virtual scene image.

S405. Map the real hand operation to the virtual hand operation of the virtual user in the virtual scene image according to the distance mapping relationship and the movement trajectory of the real hand operation.

As an example, assume that the input sensitivity is divided into low sensitivity, medium sensitivity, and high sensitivity.

Wherein, the distance mapping relationship corresponding to the intermediate sensitivity may be a basic distance ratio, for example, the basic distance ratio may be set as required. Assuming that the basic distance ratio is 1, then when the input sensitivity is intermediate sensitivity, the distance mapping relationship is that the ratio of the moving distance of the real hand operation to the virtual moving distance of the virtual hand is 1, correspondingly, the movement of the virtual hand operation The distance is the same as the movement distance of a real hand operation.

The distance ratio corresponding to the distance mapping relationship in the low-level sensitivity may be the base distance ratio multiplied by a set coefficient, and the set coefficient may be a value greater than 0 and less than or equal to 1. The distance ratio corresponding to the distance mapping relationship in the advanced sensitivity can be multiplied by (1/setting coefficient) as the basic distance ratio. After the distance ratio corresponding to the low-level sensitivity and high-level sensitivity is determined, if the input sensitivity is low-level sensitivity or high-level sensitivity, the process of mapping the moving distance of the real hand operation to the moving distance of the virtual hand operation is similar and will not be described again.

S406, if the hand gesture of the real hand operation belongs to the set object-holding gesture for maintaining the pose of the object, obtain voice information input by the real user.

S407. Identify the target object described in the voice information.

S408. Add the virtual target object to the virtual hand of the virtual user performing the virtual hand operation.

For the above steps S406 to S408, reference may be made to the related introduction of the previous embodiments, and details are not repeated here.

Corresponding to a virtual scene control method of the present application, the present application also provides a virtual scene control device.

As shown in Figure 5, it shows a schematic diagram of the composition and structure of the virtual scene control device provided by the embodiment of the present application. The device of this embodiment may include:

An operation mapping unit 501, configured to detect a real hand operation of a real user, and map the real hand operation to a virtual hand operation of a virtual user in a virtual scene image;

A voice obtaining unit 502, configured to obtain the voice information input by the real user if the hand gesture of the real hand operation belongs to the set object holding posture for maintaining the object pose;

a speech recognition unit 503, configured to recognize the target object described in the speech information;

The object adding unit 504 is configured to add the virtual target object to the virtual hand of the virtual user performing the virtual hand operation.

In a possible implementation manner, the voice obtaining unit includes:

The speech acquisition subunit is used for if the hand gesture of the real hand operation belongs to the set holding posture for maintaining the object posture, and the hand posture of the real hand operation is maintained in the holding posture for more than The duration is set, and the voice information input by the real user is obtained.

In yet another possible implementation, the speech recognition unit includes:

an object recognition unit, configured to recognize the object category and object characteristics of the target object described in the voice information;

In this application, the device also includes:

The first object construction unit is configured to add the virtual target object based on the object category and object characteristics of the target object before the object adding unit adds the virtual target object to the virtual hand of the virtual user performing the virtual hand operation, Build a virtual target object.

In another possible implementation manner, the device also includes:

A feature determining unit, configured to determine the target object based on the real hand operation of the real user before the object adding unit adds the virtual target object to the virtual hand of the virtual user performing the virtual hand operation. Describe the object characteristics of the target object;

The second object construction unit is configured to construct the virtual target object based on the object characteristics of the target object.

In yet another possible implementation manner, the operation mapping unit includes:

A type determining unit, configured to detect a real hand operation of a real user, and determine the input gesture type of the real hand operation;

A sensitivity determining unit, configured to determine the input sensitivity mapped from the real hand operation to the virtual input operation in the virtual scene image based on the input gesture type;

A mapping processing unit, configured to map the real hand operation to the virtual hand operation of the virtual user in the virtual scene image according to the input sensitivity.

In yet another possible implementation manner, the mapping processing unit includes:

A relationship determining subunit, configured to determine the distance mapping relationship between the moving distance of the real hand operation and the virtual moving distance of the virtual hand in the virtual scene image according to the input sensitivity;

The mapping processing subunit is configured to map the real hand operation to a virtual hand operation of a virtual user in the virtual scene image according to the distance mapping relationship and the movement trajectory of the real hand operation.

In yet another possible implementation manner, the type determination unit includes:

a gesture determining subunit, configured to determine a gesture of a finger used to indicate a gesture type in the real hand operation;

The type determination subunit is configured to determine the input gesture type of the real hand operation based on the gesture of the finger.

In yet another possible implementation manner, the posture determining subunit is specifically configured to, if the real hand operation belongs to a hand movement operation, determine a finger posture used to indicate a gesture type in the real hand operation .

In yet another aspect, the present application also provides an electronic device, as shown in FIG. 6 , which shows a schematic structural diagram of the electronic device. The electronic device may be any type of electronic device, and the electronic device includes at least Processor 601 and memory 602;

Wherein, the processor 601 is configured to execute the virtual scene control method in any one of the above embodiments.

The memory 602 is used to store programs necessary for the processor to perform operations.

It can be understood that the electronic device may further include a display unit 603 and an input unit 604 .

Of course, the electronic device may also have more or fewer components than those shown in FIG. 6 , without limitation.

On the other hand, the present application also provides a computer-readable storage medium, at least one instruction, at least one program, code set or instruction set is stored in the computer-readable storage medium, the at least one instruction, the at least A program, the code set or instruction set is loaded and executed by the processor to implement the virtual scene control method described in any one of the above embodiments.

The present application also proposes a computer program, the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. When the computer program runs on the electronic device, it is used to execute the virtual scene control method in any one of the above embodiments.

It should be understood that in this application, the terms "first", "second", "third", "fourth", etc. (if any) in the description and claims and the above drawings are used to distinguish similar parts, and are not necessarily used to describe a particular order or sequence. It is to be understood that the data so used are interchangeable under appropriate circumstances such that the embodiments of the application described herein can be practiced in sequences other than those illustrated herein.

It should be noted that each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. For the same and similar parts in each embodiment, refer to each other, that is, Can. At the same time, the features recorded in the various embodiments in this specification can be replaced or combined with each other, so that those skilled in the art can realize or use the present application. As for the device-type embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and for related parts, please refer to part of the description of the method embodiments.

Finally, it should also be noted that in this text, relational terms such as first and second etc. are only used to distinguish one entity or operation from another, and do not necessarily require or imply that these entities or operations, any such actual relationship or order exists. Furthermore, the term "comprises", "comprises" or any other variation thereof is intended to cover a non-exclusive inclusion such that a process, method, article, or apparatus comprising a set of elements includes not only those elements, but also includes elements not expressly listed. other elements of or also include elements inherent in such a process, method, article, or device. Without further limitations, an element defined by the phrase "comprising a ..." does not preclude the presence of additional identical elements in the process, method, article, or apparatus that includes the element.

The above description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the application. Therefore, the present application will not be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

The above are only the preferred embodiments of the present application. It should be pointed out that for those of ordinary skill in the art, some improvements and modifications can be made without departing from the principle of the application, and these improvements and modifications should also be considered as For the scope of protection of this application.

Claims (10)

1. A virtual scene control method, comprising:

detecting the real hand operation of a real user, and mapping the real hand operation into the virtual hand operation of a virtual user in a virtual scene image;

if the hand gesture of the real hand operation belongs to a set object holding gesture for maintaining the object pose, acquiring voice information input by the real user;

Identifying a target object described in the voice information;

adding the virtual target object to a virtual hand of the virtual user making the virtual hand operation.

2. The method of claim 1, wherein if the hand gesture of the real hand operation belongs to a set object holding gesture for maintaining the object pose, obtaining the voice information input by the real user comprises:

and if the hand gesture of the real hand operation belongs to the set object holding gesture for maintaining the object pose, and the duration of maintaining the hand gesture of the real hand operation in the object holding gesture exceeds the set duration, acquiring the voice information input by the real user.

3. The method of claim 1, the identifying a target object described in the speech information, comprising:

identifying object categories and object features of a target object described in the voice information;

before adding the virtual target object to the virtual hand of the virtual user making the virtual hand operation, the method further comprises:

and constructing a virtual target object based on the object category and the object characteristics of the target object.

4. The method of claim 1, the mapping the real hand operation to a virtual hand operation of a virtual user in a virtual scene image, comprising:

determining the input gesture type of the real hand operation;

determining an input sensitivity mapped by the real hand operation to a virtual input operation in a virtual scene image based on the input gesture type;

and mapping the real hand operation into a virtual hand operation of a virtual user in the virtual scene image according to the input sensitivity.

5. The method of claim 4, wherein mapping the real hand operation to a virtual hand operation of a virtual user in a virtual scene image according to the input sensitivity comprises:

determining a distance mapping relation between the moving distance of the real hand operation and the virtual moving distance of the virtual hand in the virtual scene image according to the input sensitivity;

and mapping the real hand operation into a virtual hand operation of a virtual user in the virtual scene image according to the distance mapping relation and the moving track of the real hand operation.

6. The method of claim 4 or 5, the determining the input gesture type of the real hand operation comprising:

Determining a finger gesture used for indicating a gesture type in the real hand operation;

based on the finger gesture, an input gesture type of the real hand operation is determined.

7. The method of claim 6, the determining a finger gesture in the real hand operation for indicating a gesture type, comprising:

and if the real hand operation comprises a hand movement operation, determining the finger gesture used for indicating the gesture type in the real hand operation.

8. The method of claim 1, further comprising, prior to adding the virtual target object to a virtual hand of the virtual user that made the virtual hand operation:

determining object features of the target object based on a real hand operation of the real user;

and constructing a virtual target object based on the object characteristics of the target object.

9. A virtual scene control apparatus comprising:

an operation mapping unit, configured to detect a real hand operation of a real user, and map the real hand operation into a virtual hand operation of a virtual user in a virtual scene image;

a voice obtaining unit configured to obtain voice information input by the real user if the hand gesture of the real hand operation belongs to a set object holding gesture for maintaining the object pose;

A voice recognition unit for recognizing a target object described in the voice information;

and the object adding unit is used for adding the virtual target object to the virtual hand of the virtual user which makes the virtual hand operation.

10. The apparatus of claim 9, the operation mapping unit comprising:

the type determining unit is used for detecting the real hand operation of a real user and determining the input gesture type of the real hand operation;

a sensitivity determination unit configured to determine an input sensitivity mapped by the real hand operation to a virtual input operation in a virtual scene image based on the input gesture type;

and the mapping processing unit is used for mapping the real hand operation into a virtual hand operation of a virtual user in the virtual scene image according to the input sensitivity.

Images