CN117784918A - Information interaction methods, devices, electronic equipment and storage media - Google Patents

Abstract

The embodiment of the disclosure provides an information interaction method, an information interaction device, electronic equipment and a storage medium, wherein a terminal interface of external terminal equipment is displayed through a first target area in an extended reality space; and responding to the first user operation aiming at the augmented reality space, and displaying a calling result of a target application in a terminal interface, wherein the target application is an application program running on external terminal equipment. The terminal interface of the external terminal equipment is displayed in the extended reality space, and the target application running on the external terminal equipment is controlled by aiming at the operation of the extended reality space, so that the purpose of displaying and controlling the external terminal equipment in the extended reality space is realized, the efficient interaction between the virtual reality equipment and the external terminal equipment is realized, the user can conveniently control the external terminal equipment in the process of using the virtual reality equipment, and the use convenience is improved.

Description

Information interaction methods, devices, electronic equipment and storage media

Technical field

The embodiments of the present disclosure relate to the field of virtual reality technology, and in particular, to an information interaction method, device, electronic device, and storage medium.

Background technique

Virtual Reality (VR) is a technology that uses computing devices and display devices to simulate objects in the real world, enabling users to have an immersive viewing experience. With the continuous improvement of image processing technology and computing power, virtual reality technology brings an increasingly strong sense of reality and immersion to users.

In the prior art, virtual reality applications are usually used in conjunction with corresponding virtual reality devices, such as virtual reality helmets, virtual reality controllers, and headphones, etc. However, the virtual reality devices in the prior art have the problem of being unable to display and control external terminal devices when working, causing inconvenience to users.

Summary of the invention

Embodiments of the present disclosure provide an information interaction method, device, electronic device, and storage medium to overcome the problem that virtual reality devices cannot display and operate external terminal devices.

In a first aspect, embodiments of the present disclosure provide an information interaction method, including:

In the first target area within the extended reality space, display the terminal interface of the external terminal device; in response to the first user operation on the extended reality space, display the calling result of the target application in the terminal interface, wherein, The target application is an application program running on the external terminal device.

In a second aspect, an embodiment of the present disclosure provides an information interaction device, including:

The display module is used to display the terminal interface of the external terminal device in the first target area within the extended reality space;

An interaction module is used to display a call result of a target application in the terminal interface in response to a first user operation on the extended real space, wherein the target application is an application program running on the external terminal device.

In a third aspect, an embodiment of the present disclosure provides an electronic device, including:

A processor, and a memory communicatively connected to the processor;

The memory stores computer-executable instructions;

The processor executes computer execution instructions stored in the memory to implement the information interaction method described in the first aspect and various possible designs of the first aspect.

In a fourth aspect, embodiments of the present disclosure provide a computer-readable storage medium. Computer-executable instructions are stored in the computer-readable storage medium. When the processor executes the computer-executable instructions, the above first aspect and the first aspect are implemented. various possible designs of the described information interaction methods.

In a fifth aspect, embodiments of the present disclosure provide a computer program product, including a computer program that, when executed by a processor, implements the information interaction method described in the first aspect and various possible designs of the first aspect.

The information interaction method, device, electronic device and storage medium provided by this embodiment display the terminal interface of the external terminal device through the first target area in the extended reality space; in response to the first user operation for the extended reality space , display the calling result of the target application in the terminal interface, wherein the target application is an application program running on the external terminal device. Since the purpose of displaying and controlling the external terminal device in the extended reality space is achieved by displaying the terminal interface of the external terminal device in the extended reality space, and controlling the target application running on the external terminal device through operations directed to the extended reality space, Efficient interaction between virtual reality equipment and external terminal equipment is realized, allowing users to conveniently control external terminal equipment during the use of virtual reality equipment, improving convenience of use.

Description of drawings

In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, a brief introduction will be made below to the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the drawings in the following description These are some embodiments of the present disclosure. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without exerting any creative effort.

Figure 1 is an application scenario diagram of the information interaction method provided by an embodiment of the present disclosure;

Figure 2 is a schematic flow chart 1 of an information interaction method provided by an embodiment of the present disclosure;

Figure 3 is a schematic diagram of an extended real space provided by an embodiment of the present disclosure;

Figure 4 is a flow chart of an implementation manner of step S101 in the embodiment shown in Figure 2;

Figure 5 is a schematic diagram of a process for displaying a terminal interface of an external terminal device in an extended reality space according to an embodiment of the present disclosure;

Figure 6 is a flow chart of another implementation manner of step S101 in the embodiment shown in Figure 2;

Figure 7A is a schematic diagram of a process of displaying a functional interface provided by an embodiment of the present disclosure;

Figure 7B is a schematic diagram of another process of displaying a functional interface provided by an embodiment of the present disclosure;

Figure 8 is a schematic diagram of a process for displaying a target terminal interface based on a trigger operation provided by an embodiment of the present disclosure;

Figure 9 is a flow chart of an implementation manner of step S102 in the embodiment shown in Figure 2;

Figure 10 is a schematic flowchart 2 of the information interaction method provided by an embodiment of the present disclosure;

Figure 11 is a schematic diagram of a notification interface provided by an embodiment of the present disclosure;

FIG12 is a schematic diagram of a process of displaying a notification interface provided by an embodiment of the present disclosure;

Figure 13 is a schematic diagram of a dynamic first target area provided by an embodiment of the present disclosure;

Figure 14 is a structural block diagram of an information interaction device provided by an embodiment of the present disclosure;

Figure 15 is a schematic structural diagram of an electronic device provided by an embodiment of the present disclosure;

Figure 16 is a schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present disclosure.

Detailed ways

In order to make the purpose, technical solutions and advantages of the embodiments of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments These are some embodiments of the present disclosure, but not all embodiments. Based on the embodiments in this disclosure, all other embodiments obtained by those of ordinary skill in the art without making creative efforts fall within the scope of protection of this disclosure.

The application scenarios of the embodiments of the present disclosure are explained below:

Figure 1 is an application scenario diagram of the information interaction method provided by the embodiment of the disclosure. The information interaction method provided by the embodiment of the disclosure can be applied to virtual reality (VR) applications and mixed reality (Mix Reality, MR) applications. in the scene. Specifically, as shown in Figure 1, the method provided by the embodiment of the present disclosure can be applied to a virtual reality device. The virtual reality device is, for example, the virtual reality helmet shown in the figure. The virtual reality helmet has a device for running a virtual reality application ( For example, computing units for virtual reality games and virtual reality movies. On the one hand, users can immersively watch images displayed by virtual reality applications by wearing a virtual reality helmet. On the other hand, the virtual reality device can communicate and connect with an external terminal device (smartphone shown in the figure), and interact with the external terminal device for information. Among them, the virtual reality device and the external terminal device can communicate directly through Wife, Bluetooth, etc., or indirectly through network devices such as routers and servers, without specific restrictions.

Further, for example, the virtual reality device can also be virtual reality glasses, a virtual reality handle, a virtual reality host, etc. The method provided in this embodiment can be run in a virtual reality helmet or glasses, a virtual reality handle, and a virtual reality host. in any one or more devices. The method provided by the embodiment of the present disclosure can also be applied to a cloud server that communicates with the above-mentioned virtual reality device. The cloud server communicates through the network, receives user instructions obtained through the virtual reality device, and sends image data to the virtual reality device for display. The execution subject of the information interaction method provided in this embodiment can be set according to specific needs, and will not be described in detail here.

In the existing technology, virtual reality applications usually need to be used with corresponding virtual reality devices. After users wear virtual reality helmets, headsets and other devices, due to their highly immersive information display methods, the users are in the same external real environment. The isolation state causes the user to be unable to receive messages from other external terminal devices (such as mobile phones). In some related technologies, by communicating and connecting external terminal devices such as smartphones with virtual reality devices, relevant notification information of the external terminal device can be displayed in the extended reality space. When the user sees the notification information of the external terminal device, he or she needs to When further operating the external terminal device, such as replying to a text message through a mobile phone, it is still necessary to operate the external terminal device, causing the problem of repeatedly wearing the virtual reality device and causing inconvenience to the user. Embodiments of the present disclosure provide an information interaction method to solve the above problems.

Referring to Figure 2, Figure 2 is a schematic flowchart 1 of an information interaction method provided by an embodiment of the present disclosure. The method of this embodiment can be applied in a virtual reality device or a cloud server. In this embodiment, the virtual reality device is used as the execution subject for illustration. The information interaction method includes:

Step S101: Display the terminal interface of the external terminal device in the first target area in the extended reality space.

Illustratively, a virtual reality device (VR device) is a device provided with a computing unit for running a virtual reality application (Application), and includes, for example, a virtual reality helmet (i.e., all-in-one machine), a virtual reality host (i.e., a split machine), etc. , Further, the virtual reality host can be a dedicated host device or a general-purpose computer device. A virtual reality operating system runs in the virtual reality device, and virtual reality applications are run through the virtual reality operating system. Among them, the virtual reality device itself has a display unit (all-in-one machine), or is communicatively connected to the display unit (separate machine). The display unit is, for example, a virtual reality helmet, virtual reality glasses, etc. The display unit has a display screen, and displays an extended real space (that is, a virtual reality space or a virtual display interface) through the display screen to realize image display of the virtual space constructed by the virtual reality device.

Figure 3 is a schematic diagram of an extended reality space provided by an embodiment of the present disclosure. As shown in Figure 3, exemplarily, the virtual reality device is, for example, a virtual reality helmet. After the virtual reality helmet is started and initialized, through the virtual reality helmet The display screen displays the extended reality space. In one possible case, the system interface of the virtual operating system is displayed in the extended reality space, including application icons, function icons and other common icons. Clicking such common icons will run the corresponding virtual reality Applications and functions will not be described in detail here. On the other hand, in the first target area of the extended reality space, a terminal interface of an external terminal device is displayed. As shown in Figure 3, the external terminal device is, for example, a smartphone, and the corresponding terminal interface is a mobile phone interface. Displayed in the mobile phone interface There are running terminal applications, such as communication applications. After displaying the smartphone interface in the extended reality space, the user can simultaneously observe the terminal application running in the mobile interface while viewing the extended reality space, thereby realizing virtual reality. Information communication between space and terminal equipment in the external environment.

Furthermore, in a possible implementation, the terminal interface of the external terminal device can always be displayed in the extended reality space. For example, after the operating system of the virtual reality device is initialized, the corresponding data is obtained through communication with the external terminal device. , the terminal interface of the external terminal device is fixedly displayed in the first target area of the extended reality space, such as the lower right corner of the extended reality space, thereby realizing the normalized display of the terminal interface of the external terminal device in the extended reality space, so that the user can simultaneously View and process information from two terminal devices (virtual reality device and external terminal device).

In another possible implementation manner, after receiving the trigger instruction, the virtual reality device displays the terminal interface of the external terminal device in the extended reality space. For example, the triggering instruction may be generated based on push information from an external terminal device. In a possible implementation, as shown in Figure 4, the specific implementation steps of step S101 include:

Step S1011: Obtain the push information sent by the external terminal device. The push information represents the application message generated by the target application running on the external terminal device.

Step S1012: Based on the push information, render the corresponding target terminal interface in the first target area in the extended reality space. The target terminal interface is used to display application messages generated by the target application.

The application messages generated by the target application are in-application messages generated by the target application running on the external terminal device in response to a trigger event during the running process. For example, the instant messaging application (target application) receives user messages, videos, etc. The message generated during the call. Figure 5 is a schematic diagram of a process for displaying a terminal interface of an external terminal device in an extended reality space according to an embodiment of the present disclosure. As shown in Figure 5, the external terminal device is, for example, a smartphone, and there is a gap between the external terminal device and the virtual reality device. Connected through wireless communication methods such as wifi, before the instant messaging application (target application) running in the external terminal device receives the user message (the first moment), only content related to the virtual reality application is displayed in the extended reality space; on the external terminal device After the instant messaging application running in the mobile phone receives the user message (the second moment), the external terminal device sends push information to the virtual reality device. After receiving the push information, the virtual reality device generates a trigger instruction, and based on the trigger instruction, in the extended reality space Display the target terminal interface containing the user message, so that the user message received by the instant messaging application running in the external terminal device can be synchronously displayed in the extended reality space.

Among them, after the virtual reality device receives the push information, it can generate the target terminal interface in the following optional ways. In one possible way, for example, after the virtual reality device receives the push information, it establishes an interface projection data channel with the external terminal device. , after that, the external terminal device projects the application interface of the target application (used to display application messages) into the first target area of the extended reality space for rendering, thereby generating a target terminal interface displayed in the extended reality space.

In another possible way, for example, the push message includes the application message of the target application. After receiving the push information, the virtual reality device uses the push message to determine the corresponding target application, and then based on the preset interface description information of the target application, The interface layout of the target application is reconstructed and rendered in the first target area, and data is filled in combination with the application message of the target application carried in the push message, thereby generating a target terminal interface displayed in the extended reality space. Exemplarily, the target application displays the application message in a first interface of the external terminal device; the first interface and the target terminal interface have the same interface layout. Through the interface description information, we can achieve accurate simulation of the first interface of the target application, improve the consistency between the terminal interface in the extended reality space and the first interface of the external terminal device running the target application, and improve visual immersion.

In this embodiment, the push information is used to trigger the display of the terminal interface in the extended reality space, thereby realizing real-time push of messages between the external terminal device and the virtual reality device, while avoiding the impact of long-term display of the terminal interface in the extended reality space. Visual immersion issues to improve user experience.

In another possible implementation, as shown in Figure 6, the specific implementation steps of step S101 include:

Step S1013: In response to the fourth user operation, a function interface is displayed in the second target area of the extended reality space, where the function interface is used to display an identifier of at least one alternative application, where the alternative application is an application program run by an external terminal device.

FIG. 7A is a schematic diagram of a process for displaying a functional interface provided by an embodiment of the present disclosure. FIG. 7B is a schematic diagram of a process of displaying a functional interface provided by an embodiment of the present disclosure. The above process will be described in detail below based on FIG. 7A and FIG. 7B introduce. For example, in the extended reality space, the first target area is set on the right side of the interface, and the second target area is set on the left side of the interface. First, after the fourth user operation is detected, in the second target area of the extended reality space , display the function interface. In one possible implementation, as shown in Figure 7A, the fourth user operation may be a triggering operation for a certain functional component in the extended reality space, for example, by operating the right virtual reality handle to click on a functional component in the extended reality space. A. Display function interface. In another possible implementation, as shown in Figure 7B, the fourth user operation can also be an operating gesture for the virtual reality handle, such as a lift gesture of lifting the left virtual reality handle to display the functional interface and simultaneously display The gesture identifier corresponding to the raise gesture.

Further, the identification of at least one alternative application is displayed in the functional interface, and the alternative application is an application program run by an external terminal device, for example, a phone application, a text message application, an instant messaging application, etc. executed on the side of the external terminal device. After that, by clicking on the logo of the alternative application displayed in the function interface, the application program executed on the side of the external terminal device can be called. The specific implementation will be introduced in detail in subsequent steps.

Step S1014: In response to the triggering operation for the identification of the candidate application, determine the target application, and render the corresponding target terminal interface in the first target area in the extended reality space, where the target terminal interface is used to display the target application running on the external terminal device.

For example, the alternative applications displayed in the functional interface can be called through the corresponding set functional components. Specifically, the corresponding functional components are triggered by clicking on the logo of the alternative applications. After the functional components are triggered, the remote execution Call the method to control the external terminal to execute the corresponding alternative application. In this embodiment, first, based on the identification of the candidate application targeted by the trigger operation, the candidate application selected by the trigger operation is determined as the target application, and the corresponding functional component is triggered. After that, the remote calling method is executed to control the external terminal. The target application is run, and a target terminal interface for displaying the target application is rendered in the first target area in the extended reality space.

Figure 8 is a schematic diagram of a process for displaying a target terminal interface based on a trigger operation provided by an embodiment of the present disclosure. As shown in Figure 8, the functional interface includes application identifiers corresponding to multiple alternative applications set in a ring, and each application identifier corresponds to A component that displays the corresponding target terminal interface in the first target area when the user selects and clicks on different application logos through the virtual reality controller, thereby realizing the application running in the external terminal from the functional interface of the extended reality space. call. Referring to Figure 8, when application logo A is clicked, the terminal interface View1 corresponding to application A is displayed in the first target area; when application logo B is clicked, the terminal interface View2 corresponding to application B is displayed in the first target area.

There are many ways to render the corresponding target terminal interface in the first target area in the extended reality space. In one possible way, for example, after the virtual reality device detects the trigger operation and determines the target application, it communicates with the external terminal device Establish an interface projection data channel. After that, the external terminal device projects the application interface of the target application (used to display application messages) into the first target area of the extended reality space for rendering, thereby generating the target terminal interface displayed in the extended reality space. .

In another possible way, for example, after the virtual reality device detects the trigger operation and determines the target application, it reconstructs the interface layout of the target application in the first target area and renders it based on the preset interface description information of the target application. , then or at the same time, send a request for the target application to the external terminal device, obtain the application message sent by the target application through the messaging system (messaging system), and use the application message of the target application to fill in data, thereby generating a message displayed in the extended reality space. Target terminal interface. Exemplarily, the target application displays application messages in the first interface of the external terminal device; the first interface and the target terminal interface have the same interface layout, and the interface description information is used to achieve an accurate simulation of the first interface of the target application, improving Expand the consistency between the terminal interface in the real space and the first interface in the target application running on the external terminal device to improve visual immersion.

In this embodiment, by displaying a function interface in response to a first user operation, rendering of the target terminal interface is achieved based on a trigger operation on the function interface, thereby realizing external function calls based on user needs in the extended reality space, and improving the operational flexibility and convenience when calling applications of external terminal devices in the extended reality space.

S102: In response to the first user operation on the extended reality space, display the calling result of the target application in the terminal interface, where the target application is an application running on an external terminal device.

For example, the terminal interface is displayed in the extended reality space, and the user can further operate the terminal interface based on the content displayed on the terminal interface, thereby realizing interaction with the external terminal device.

In a possible implementation process, as shown in FIG9 , the specific implementation steps of step S102 include:

Step S1021: In response to the trigger operation for the target function identifier in the terminal interface, send a call request to the external terminal device, where the call request includes input parameters for executing the target function of the target application.

Step S1022: Receive the call result corresponding to the call request returned by the external terminal device, and display the call result in the first target area.

Specifically, the terminal interface is used to display the application interface corresponding to the target application. There is at least one function identifier in the application interface. Each function identifier corresponds to a functional component. For example, the "send" identifier in the instant messaging application corresponds to sending a message. The functional components; the "emoticon" logo corresponds to the functional component for displaying the emoticon page; and each "letter logo" in the input method area corresponds to the functional component for inputting letters in the input method.

Further, the user triggers a button in the terminal interface in the extended reality space, for example, through a virtual reality handle. In response to the triggering operation, the virtual reality device sends a corresponding call request to the external terminal device. The call request includes executing Input parameters of the target function of the target application, such as the component identifier of the triggered functional component, trigger action type, etc. After receiving the call request, the external terminal device responds to it and executes the target function of the target application. For example, corresponding to the above example, sending messages, displaying emoticon pages, inputting subtitles, etc., the execution result corresponding to the target function is, that is, Call results, such as notification of successful message sending, emoticon page, etc. The external terminal device returns the call result to the virtual reality device, and the virtual reality device obtains the call result and displays it in the terminal interface to display the result of the external terminal device responding to the trigger operation. In terms of the visual perception of the extended reality space, The external terminal device is projected into the extended reality space, and the user can control the external terminal device based on operations on the terminal interface in the extended reality space, thus realizing the interaction between the virtual reality device and the external terminal device.

Among them, based on different implementation methods of the terminal interface, the call result displayed in the terminal interface can be the content actually displayed in the display interface of the external terminal device running the target application. This method can be achieved by running the target application on the external terminal device. The display interface of the application is projected; it can also be simulated content generated based on the content displayed on the display interface of the target application running on the side of the external terminal device. This method can be based on the application layout of the target application and combined with the content sent by the external terminal device. The application message of the target application is implemented by reconstructing the display interface of the target application. For the specific implementation process, please refer to the detailed introduction of the steps of rendering the target terminal interface in the steps of the above embodiment, and will not be described again here.

In this embodiment, the terminal interface of the external terminal device is displayed through the first target area in the extended reality space; in response to the first user operation on the extended reality space, the calling result of the target application is displayed in the terminal interface, where , the target application is an application running on an external terminal device. By displaying the terminal interface of the external terminal device in the extended reality space and controlling the target application running on the external terminal device through operations in the extended reality space, the purpose of displaying and controlling the external terminal device in the extended reality space is achieved. The efficient interaction between virtual reality equipment and external terminal equipment allows users to conveniently control external terminal equipment during the use of virtual reality equipment, improving convenience of use.

Referring to FIG10 , FIG10 is a flow chart of the information interaction method provided by the embodiment of the present disclosure. Based on the embodiment shown in FIG2 , this embodiment further refines step S101 and adds a step of determining the first target area. The information interaction method includes:

Step S201: acquiring push information sent by an external terminal device, where the push information represents an application message generated by a target application running on the external terminal device.

Step S202: Display a notification interface in the second target area of the extended reality space. The notification interface is used to display abbreviated messages and/or reception identifiers corresponding to the push information.

Figure 11 is a schematic diagram of a notification interface provided by an embodiment of the present disclosure. As shown in Figure 11, the second target area is located on the left side of the extended reality space. After receiving the push information sent by the external terminal device, the virtual reality device based on the The content in the push message displays an interface for displaying the abbreviated message and/or the receiving identification corresponding to the push information in the second target area, that is, the notification interface, where the area of the second target area is smaller than that of the first target area. area. More specifically, in one implementation, the notification interface displays an abbreviated message corresponding to the push information. The abbreviated message is a part of the application message represented by the push information. Exemplarily, the application message includes the Chinese character string "I will go there this afternoon." I’m looking for you to get what’s mine.” The corresponding abbreviated message is the string "I'm going to find you this afternoon...".

In another implementation, the notification interface displays a receiving identifier corresponding to the push information. The receiving identifier represents a notification of receiving a push message sent by an external terminal device. For example, the receiving identifier is a combination of an application identifier and a specific identifier of the target application. , for example, receive a pattern consisting of a square icon identified as the target application and a circle located in the upper right corner of the square icon.

In this embodiment, the notification interface is displayed in the second target area of the extended reality space to prompt the user that the target application of the external terminal device has sent an application message to prevent the user from missing the application message on the side of the external terminal device. At the same time, through area update Use a small second target area to display notification messages to avoid directly displaying the terminal interface, which may visually affect immersion, and improve the visual effect.

In a possible implementation, a specific implementation of step S202 includes:

Step S2021: Display a first notification interface in the second target area of the extended reality space, where the notification interface is used to display a receiving identifier corresponding to the push information.

Step S2021: In response to a third user operation, synchronously display a second notification interface in a third target area of the extended reality space.

Illustratively, the first notification interface and the second notification interface are two ways to implement the notification interface. The area of the first notification interface is smaller than the area of the second notification interface. The first notification interface is used to display the receiving identification, and the second notification interface is used to display the receiving identification. The notification interface is used to display thumbnail messages. The display of the second notification interface needs to be triggered based on a third user operation. For example, after detecting a raise gesture for the virtual reality handle, the notification interface is displayed, and at the same time, a gesture mark corresponding to the raise gesture is displayed in the second target area. Furthermore, the gesture logo is displayed near a second target area, and the second target area is located on a side close to the raising gesture in the extended reality space.

Figure 12 is a schematic diagram of a process for displaying a notification interface provided by an embodiment of the present disclosure. As shown in Figure 12, first, after the virtual reality device receives the push information sent by the external terminal device, it first displays the first message in the second target area. In the notification interface, the first notification interface displays a receiving identifier. The receiving identifier is the name of the target application "app_1", indicating that app_1 has sent a push message. Afterwards, when the virtual reality device detects the raising gesture for the virtual reality handle, the second notification interface is displayed in the third target area. The area of the second notification interface is larger than the area of the first notification interface, and the displayed information is also More, for example, as shown in the figure, the second notification interface displays the abbreviated message "I will go find you this afternoon..." corresponding to the push message sent by the target application app_1. Optionally, when the second notification interface is displayed, a gesture identifier may be displayed near the second notification interface to prompt the user that the raise gesture is currently applied.

Further, the gesture logo can be, for example, a wrist pattern for wearing a watch, and the second notification interface is a matching watch interface. By displaying the matching gesture logo and the second notification interface, the user's operation complexity is reduced and the user's comfort level is improved. Ease of use.

In the steps of this embodiment, after the virtual reality device receives the push information, it first displays a first notification interface with a smaller area, and then displays a second notification interface based on the user's operation, thereby realizing the push information sent by the external terminal device. Hierarchical display, in the context of virtual reality application, further reduces the impact of external information access on the visual immersion in the virtual space interface, and improves the flexibility and convenience of external information display.

In another possible implementation, the specific implementation of step S201 can directly display the second notification interface without displaying the first notification interface, that is, the specific implementation of step S201 includes: responding to the third user Operation, the notification interface is synchronously displayed in the second target area of the extended reality space, and the gesture logo corresponding to the raise gesture is displayed, wherein the third user operation is a raise gesture for the virtual reality handle, and the second target area is located in the extended reality space Center closer to the side of the raise gesture. The specific implementation method is the same as the process type of displaying the second notification interface in the above steps, and will not be described again here. Further, in this implementation, the notification interface may be the same interface as the functional interface introduced in the previous embodiment, that is, the notification interface is used to display the identification of at least one alternative application, and the alternative application is run by an external terminal device s application. For specific implementation methods, please refer to the introduction of the notification interface in the above embodiment, and will not be described again here.

Optionally, after step S202, it also includes:

Step S203: Obtain the target image currently displayed in the extended reality space, and the target image includes at least one display object.

Step S204: Determine the first target area based on the position of at least one display object in the target image.

Exemplarily, based on the operating system or application program currently running on the virtual reality device, a corresponding target image is displayed in the extended reality space. The target image includes at least one display object. Based on the content specifically displayed in the target image, the display object can, for example, They are virtual objects, virtual characters, and also system controls, application icons, etc. displayed on the operating system interface. By parsing and identifying the target image, the position of each display object can be obtained. Specifically, the position of the display object is, for example, an array consisting of the four corner coordinates of the quadrilateral frame used to position the display object in a two-dimensional plane. ; Or, an array composed of the center point coordinates of the display object and the area of the display object. After obtaining the position of each display object in the target image, based on different purposes, an area for displaying the terminal interface can be determined, that is, the first target area.

In a possible implementation, the implementation steps of step S204 include:

Step S2041: Obtain the separation distance between the display objects, and determine at least one candidate area based on the separation distance between the display objects. The candidate area is an area that can accommodate the terminal interface.

Step S2042: Determine a first target area from at least one candidate area.

For example, in order to prevent the terminal interface displayed on the upper layer from blocking the display objects in the target image, the separation distance between the display objects is determined based on the position of each display object. The greater the separation distance, the smaller the distance between the display objects. The larger the non-occlusion area is, if the separation distance is greater than the preset separation threshold, it is considered that the non-occlusion area between the two display objects can accommodate the terminal interface, and the non-occlusion area is determined as a candidate area. Further, after at least one candidate area is determined, the first target area is selected from the at least one candidate area according to a preset selection rule. Among them, the preset selection rules are, for example, dividing the target image into multiple image areas, giving priority to the candidate areas located in the edge areas of the target image. If there are no alternative areas in the edge areas, then selecting the central area of the target image. alternative area, so that the first target area is as close as possible to the edge of the extended reality space, thereby improving the display effect of the extended reality space.

In the steps of this embodiment, the separation distance between each display object is obtained, and based on the separation distance between each display object, candidate areas that will not cause occlusion are determined, and then the edge areas of the extended reality space are retrieved sequentially. , from which the first target area is determined. Avoid the terminal interface from blocking the effective information in the target image and improve the visual display effect.

In another possible implementation, the implementation steps of step S204 include:

Step S2043: Perform target recognition on each display object to obtain the target recognition object;

Step S2044: Determine a first target area based on the outline of the target recognition object.

Illustratively, in this embodiment, image recognition is first performed on each display object in the target image, and the objects belonging to the target object category are determined, that is, the target recognition objects, such as mobile phones, tablets, tables, photo frames, etc. After that, the outline of the target recognition object is determined as the first target area to realize the mapping between the target recognition object and the terminal interface. The outline of the target recognition object may be the outline of the target recognition object, the outline of the positioning frame, or the target recognition object. The outline of the display screen will not be given one by one. For example, the application message sent by the target application on the external terminal device side is displayed on the TV (target recognition object) in the target image displayed by the virtual reality device, thereby realizing the transmission of external information into the virtual reality space. In the steps of this embodiment, by setting the first target area carrying the terminal interface on the target recognition object, smooth interaction between the virtual reality space and the external real space is achieved, and visual immersion and visual experience are improved.

Further, the object category of the target recognition object is determined by the external terminal device. More specifically, the object category of the target recognition object is consistent with the external terminal device. The push information sent by the external terminal device received by the virtual reality device includes The device identification of the external terminal device, the virtual reality device determines the target object category based on the device identification, and then determines the target recognition object based on the target object category. For example, if based on the device identification, the external terminal device is represented as a smartphone, then the corresponding target object category is determined to be P class, and P class represents cell phones and tablet computers. When it is recognized that the target image includes smartphones and tablet computers, then the The first target area is selected to be set within the outline of the smartphone or tablet unit; if based on the device identification, the external terminal device is represented as a smart TV, then the corresponding target object category is determined to be the T category, and the T category represents the TV. When the target is recognized When a TV is included in the image, the first target area is set within the outline of the TV. Figure 13 is a schematic diagram of a dynamic first target area provided by an embodiment of the present disclosure. As shown in Figure 13, exemplarily, the virtual reality device is communicatively connected to the smart TV and the smart phone respectively. Wherein, when the virtual reality device receives After the smart TV sends the push information, it identifies the currently displayed target image, determines the outline of the "TV" in the target image as the first target area Zoom1, and displays the terminal interface of the smart TV after the trigger conditions are met. The first target area Zoom1 of the extended real space; when the virtual reality device receives the push information sent by the smartphone, it identifies the currently displayed target image and determines the outline of the "mobile phone" in the target image as the first target area Zoom2. And after the trigger conditions are met, the terminal interface of the smart TV is displayed in the first target area of the extended reality space, Zoom2.

In this embodiment, by identifying the target identification object corresponding to the type of the external terminal device in the target object, the purpose of matching and displaying the external terminal device and the virtual device in the virtual space is achieved, and the matching between the external device and the virtual device in the virtual space is realized. Seamless interaction further enhances visual immersion.

Furthermore, the method provided in this embodiment can also be applied to application scenarios with mixed reality (MR). Mixed reality technology is based on virtual reality technology and combined with augmented reality (AR) to combine the real environment with The images in the virtual reality space are integrated into the virtual reality space. The specific implementation principle is based on existing technology and will not be described again here. Regarding the steps in this embodiment, the target image displayed in the extended reality space includes real image objects and virtual objects, that is, the display objects include virtual reality objects and real display objects. In a possible implementation, based on the position of at least one display object in the target image, the specific implementation steps of determining the first target area include: performing target recognition on each real display object, obtaining the target real display object, and the target real display The object is a real image of the external terminal device; the first target area is determined based on the outline of the target real display object.

In this embodiment, for the mixed reality application scenario, in the process of identifying the target image, image recognition is only performed on the real display object, so as to obtain the target real display object corresponding to the external terminal device, so that the target image can be carried The first target area of the terminal page is displayed on the outline of the target real display object, realizing further interactive integration of external terminal equipment and virtual reality equipment in mixed reality application scenarios, further improving the visual performance of the extended real space in mixed reality.

Step S205: In response to the second user operation on the notification interface, rendering a corresponding target terminal interface in the first target area in the extended reality space.

Exemplarily, after the first target area is determined, in response to the second user operation on the notification interface, for example, when the user selects and clicks the thumbnail message in the notification interface through the virtual reality handle and receives the identification, the first target area is determined. In the target area, the corresponding target terminal interface is displayed to display the abbreviated message and the detailed information corresponding to the receiving identification, that is, the application interface of the target application. Thus, the application program running in the external terminal can be called from the functional interface of the extended reality space.

There are many ways to render the corresponding target terminal interface in the first target area in the extended reality space. In one possible way, for example, after the virtual reality device detects the second user operation, the corresponding target application is determined, And establish an interface projection data channel with the external terminal device. After that, the external terminal device projects the application interface of the target application (used to display application messages) to the first target area of the extended reality space for rendering, thereby generating a display in the extended reality space. The target terminal interface within.

In another possible way, for example, after detecting the second user operation, the virtual reality device determines the corresponding target application, and reconstructs the target application in the first target area based on the preset interface description information of the target application. The interface is laid out and rendered, and then or at the same time, a request for the target application is sent to the external terminal device, the application message sent by the target application is obtained through the message system, and the application message of the target application is used for data filling, thereby generating a display in the extended reality space The target terminal interface within. Exemplarily, the target application displays application messages in the first interface of the external terminal device; the first interface and the target terminal interface have the same interface layout, and the interface description information is used to achieve an accurate simulation of the first interface of the target application, improving Expand the consistency between the terminal interface in the real space and the first interface in the target application running on the external terminal device to improve visual immersion.

Step S206: In response to the first user operation on the extended real space, the call result of the target application is displayed in the terminal interface, wherein the target application is an application program running on the external terminal device.

In this embodiment, the specific implementation of step S201 and step S206 has been introduced in detail in the corresponding steps of the embodiment shown in FIG. 2 , and will not be repeated here.

Corresponding to the information interaction method of the above embodiment, FIG. 14 is a structural block diagram of the information interaction device 3 provided by the embodiment of the present disclosure. For convenience of explanation, only parts related to the embodiments of the present disclosure are shown. Referring to Figure 14, the information interaction device 3 includes:

The display module 31 is used to display the terminal interface of the external terminal device in the first target area within the extended reality space;

The interaction module 32 is configured to display the calling result of the target application in the terminal interface in response to the first user operation on the extended reality space, where the target application is an application running on an external terminal device.

In one embodiment of the present disclosure, the display module 31 is specifically used to: obtain push information sent by an external terminal device, where the push information represents an application message generated by a target application running on the external terminal device; based on the push information, in the extended reality space The first target area renders the corresponding target terminal interface, and the target terminal interface is used to display application messages generated by the target application.

In one embodiment of the present disclosure, when rendering the corresponding target terminal interface in the first target area in the extended reality space based on the push information, the display module 31 is specifically used to: display in the second target area of the extended reality space. Notification interface, the notification interface is used to display abbreviated messages and/or reception identifiers corresponding to push information, and the area of the second target area is smaller than the area of the first target area; in response to the second user operation on the notification interface, in the extended reality space The first target area within renders the corresponding target terminal interface.

In one embodiment of the present disclosure, when displaying the notification interface in the second target area of the extended reality space, the display module 31 is specifically configured to: in response to a third user operation, synchronously display the notification in the second target area of the extended reality space. interface, wherein the third user operation is a raise gesture for the virtual reality handle, and the second target area is located on the side close to the raise gesture in the extended reality space; the display module 31 also includes: displaying a gesture logo corresponding to the raise gesture.

In one embodiment of the present disclosure, the target application displays the application message in the first interface of the external terminal device; the first interface and the target terminal interface have the same interface layout.

In one embodiment of the present disclosure, the display module 31 is specifically configured to: in response to the fourth user operation, display a functional interface in the second target area of the extended reality space, and the functional interface is used to display at least one alternative application. Identifies that the candidate application is an application program run by an external terminal device; in response to the triggering operation for the identification of the candidate application, the target application is determined, and the corresponding target terminal interface is rendered in the first target area within the extended reality space, and the target The terminal interface is used to display the target application running on the external terminal device.

In one embodiment of the present disclosure, the interaction module 32 is specifically used to: in response to a trigger operation for a target function identifier in a terminal interface, send a call request to an external terminal device, the call request including input parameters for executing the target function of the target application; receive a call result corresponding to the call request returned by the external terminal device, and display the call result in a first target area.

In one embodiment of the present disclosure, before displaying the terminal interface of the external terminal device in the first target area in the extended reality space, the display module 31 is also used to: obtain the target image currently displayed in the extended reality space, the target image including at least one display object; and determining the first target area based on the position of the at least one display object in the target image.

In one embodiment of the present disclosure, when the display module 31 determines the first target area based on the position of at least one display object in the target image, it is specifically used to: obtain the spacing distance between each display object; based on the spacing distance between each display object, determine at least one alternative area, the alternative area is an area that can accommodate the terminal interface; and determine the first target area from at least one alternative area.

In one embodiment of the present disclosure, when the display module 31 determines the first target area based on the position of at least one display object in the target image, it is specifically used to: perform target recognition on each display object to obtain the target recognition object, the target The object category of the recognition object is determined based on the terminal category of the external terminal device; the first target area is determined based on the outline of the target recognition object.

In one embodiment of the present disclosure, the display objects include virtual reality objects and real display objects. When determining the first target area based on the position of at least one display object in the target image, the display module 31 is specifically configured to: The display object performs target recognition to obtain the real display object of the target, which is the real image of the external terminal device; based on the outline of the real display object of the target, the first target area is determined.

The device provided in this embodiment can be used to execute the technical solutions of the above method embodiments. Its implementation principles and technical effects are similar, and will not be described again in this embodiment.

Figure 15 is a schematic structural diagram of an electronic device provided by an embodiment of the present disclosure. As shown in Figure 15, the electronic device 4 includes:

Processor 41, and memory 42 communicatively connected to processor 41;

Memory 42 stores computer execution instructions;

The processor 41 executes the computer execution instructions stored in the memory 42 to implement the information interaction method in the embodiment shown in FIGS. 2-13.

Wherein, optionally, the processor 41 and the memory 42 are connected through the bus 43 .

Relevant descriptions can be understood by referring to the relevant descriptions and effects corresponding to the steps in the embodiments corresponding to Figures 2 to 13, and will not be described in detail here.

Referring to FIG. 16 , a schematic structural diagram of an electronic device 900 suitable for implementing an embodiment of the present disclosure is shown. The electronic device 900 may be a terminal device or a server. The terminal devices may include, but are not limited to, mobile phones, notebook computers, digital broadcast receivers, personal digital assistants (Personal Digital Assistant, PDA for short), tablet computers (Portable Android Device, PAD for short), portable multimedia players (Portable Mobile terminals such as Media Player (PMP for short), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), and fixed terminals such as digital TVs, desktop computers, etc. The electronic device shown in FIG. 16 is only an example and should not impose any limitations on the functions and scope of use of the embodiments of the present disclosure.

As shown in Figure 16, the electronic device 900 may include a processing device (such as a central processing unit, a graphics processor, etc.) 901, which may process data according to a program stored in a read-only memory (Read Only Memory, ROM for short) 902 or from a storage device. 908 loads the program in the random access memory (Random Access Memory, RAM for short) 903 to perform various appropriate actions and processes. In the RAM 903, various programs and data required for the operation of the electronic device 900 are also stored. The processing device 901, the ROM 902 and the RAM 903 are connected to each other via a bus 904. An input/output (I/O) interface 905 is also connected to bus 904.

Generally, the following devices can be connected to the I/O interface 905: input devices 906 including, for example, a touch screen, touch pad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; including, for example, a liquid crystal display (Liquid Crystal Display, LCD for short) , an output device 907 such as a speaker, a vibrator, etc.; a storage device 908 including a magnetic tape, a hard disk, etc.; and a communication device 909. The communication device 909 may allow the electronic device 900 to communicate wirelessly or wiredly with other devices to exchange data. Although FIG. 16 illustrates electronic device 900 with various means, it should be understood that implementation or availability of all illustrated means is not required. More or fewer means may alternatively be implemented or provided.

In particular, according to embodiments of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product including a computer program carried on a computer-readable medium, the computer program containing program code for performing the method illustrated in the flowchart. In such embodiments, the computer program may be downloaded and installed from the network via communication device 909, or from storage device 908, or from ROM 902. When the computer program is executed by the processing device 901, the above-mentioned functions defined in the method of the embodiment of the present disclosure are performed.

It should be noted that the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the above two. The computer-readable storage medium may be, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any combination thereof. More specific examples of computer readable storage media may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard drive, random access memory (RAM), read only memory (ROM), removable Programmed read-only memory (EPROM or flash memory), fiber optics, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above. In this disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code therein. Such propagated data signals may take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device . Program code embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to: wire, optical cable, RF (radio frequency), etc., or any suitable combination of the above.

The above-mentioned computer-readable medium may be included in the above-mentioned electronic device; it may also exist independently without being assembled into the electronic device.

The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device performs the method shown in the above embodiment.

Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, including object-oriented programming languages such as Java, Smalltalk, C++, and conventional Procedural programming language—such as "C" or a similar programming language. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or it can be connected to an external computer (e.g. using an Internet service provider to connect via the Internet).

The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operations of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, segment, or portion of code that contains one or more logic functions that implement the specified executable instructions. It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown one after another may actually execute substantially in parallel, or they may sometimes execute in the reverse order, depending on the functionality involved. It will also be noted that each block of the block diagram and/or flowchart illustration, and combinations of blocks in the block diagram and/or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or operations. , or can be implemented using a combination of specialized hardware and computer instructions.

The units involved in the embodiments of the present disclosure can be implemented in software or hardware. The name of the unit does not constitute a limitation on the unit itself under certain circumstances. For example, the first acquisition unit can also be described as "the unit that acquires at least two Internet Protocol addresses."

The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), Systems on Chips (SOCs), Complex Programmable Logical device (CPLD) and so on.

In the context of this disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media may include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media would include one or more wire-based electrical connections, laptop disks, hard drives, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above.

In a first aspect, according to one or more embodiments of the present disclosure, an information interaction method is provided, including:

In the first target area within the extended reality space, display the terminal interface of the external terminal device; in response to the first user operation on the extended reality space, display the calling result of the target application in the terminal interface, wherein, The target application is an application program running on the external terminal device.

According to one or more embodiments of the present disclosure, displaying a terminal interface of an external terminal device in the first target area in the extended reality space includes: obtaining push information sent by the external terminal device, where the push information represents An application message generated by a target application run by the external terminal device; based on the push information, a corresponding target terminal interface is rendered in the first target area in the extended reality space, and the target terminal interface is used to display the target Application messages generated by the application.

According to one or more embodiments of the present disclosure, rendering the corresponding target terminal interface in the first target area in the extended reality space based on the push information includes: a second target in the extended reality space area, displaying a notification interface, the notification interface being used to display the abbreviated message and/or receiving identification corresponding to the push information, the area of the second target area being smaller than the area of the first target area; in response to the According to the second user operation of the notification interface, the corresponding target terminal interface is rendered in the first target area in the extended reality space.

According to one or more embodiments of the present disclosure, displaying a notification interface in a second target area of the extended reality space includes: in response to a third user operation, synchronously displaying the notification interface in the second target area of the extended reality space, wherein the third user operation is a lift gesture for a virtual reality handle, and the second target area is located on a side of the extended reality space close to the lift gesture; the method also includes: displaying a gesture identifier corresponding to the lift gesture.

According to one or more embodiments of the present disclosure, the target application displays the application message in a first interface of the external terminal device; the first interface has the same interface layout as the target terminal interface.

According to one or more embodiments of the present disclosure, the first target area in the extended reality space displays a terminal interface of an external terminal device, including: in response to a fourth user operation, in the second target area of the extended reality space. The target area displays a functional interface, the functional interface is used to display the identification of at least one alternative application, and the alternative application is an application program run by the external terminal device; in response to the identification of the alternative application The triggering operation determines the target application, and renders the corresponding target terminal interface in the first target area in the extended reality space. The target terminal interface is used to display the target application run by the external terminal device.

According to one or more embodiments of the present disclosure, in response to the first user operation on the extended reality space, displaying the calling result of the target application in the terminal interface includes: in response to the first user operation on the extended reality space in the terminal interface. trigger operation of the target function identifier, send a call request to the external terminal device, the call request includes input parameters for executing the target function of the target application; receive the call request returned by the external terminal device The corresponding call result is displayed in the first target area.

According to one or more embodiments of the present disclosure, before displaying the terminal interface of the external terminal device in the first target area within the extended reality space, it also includes: acquiring the target image currently displayed in the extended reality space, the target image including at least one display object; determining the first target area based on the position of at least one of the display objects in the target image.

According to one or more embodiments of the present disclosure, determining the first target area based on the position of at least one of the display objects in the target image includes: obtaining a separation distance between each of the display objects. ; Based on the separation distance between each of the display objects, determine at least one alternative area, the alternative area is an area that can accommodate the terminal interface; determine the first candidate area from the at least one alternative area target area.

According to one or more embodiments of the present disclosure, determining the first target area based on the position of at least one display object in the target image includes: performing target recognition on each of the display objects to obtain A target recognition object, the object category of the target recognition object is determined based on the terminal category of the external terminal device; the first target area is determined based on the outline of the target recognition object.

According to one or more embodiments of the present disclosure, the display object includes a virtual reality object and a real display object, and the first target area is determined based on the position of at least one of the display objects in the target image, The method includes: performing target recognition on each of the real display objects to obtain the target real display object, wherein the target real display object is a real image of the external terminal device; and determining the target real display object based on the outline of the target real display object. First target area.

In a second aspect, according to one or more embodiments of the present disclosure, an information interaction device is provided, including:

The display module is used to display the terminal interface of the external terminal device in the first target area within the extended reality space;

An interaction module, configured to respond to a first user operation on the extended reality space and display a call result of a target application in the terminal interface, where the target application is an application running on the external terminal device.

According to one or more embodiments of the present disclosure, the display module is specifically configured to: obtain push information sent by the external terminal device, where the push information represents an application message generated by a target application run by the external terminal device; Based on the push information, a corresponding target terminal interface is rendered in the first target area in the extended reality space, and the target terminal interface is used to display application messages generated by the target application.

According to one or more embodiments of the present disclosure, when the display module renders the corresponding target terminal interface in the first target area in the extended reality space based on the push information, the display module is specifically configured to: A second target area in the real space displays a notification interface. The notification interface is used to display abbreviated messages and/or reception identifiers corresponding to the push information. The area of the second target area is smaller than that of the first target area. area; in response to the second user operation on the notification interface, rendering the corresponding target terminal interface in the first target area within the extended reality space.

According to one or more embodiments of the present disclosure, when the display module displays a notification interface in the second target area of the extended reality space, it is specifically configured to: in response to a third user operation, display the notification interface in the second target area of the extended reality space. The notification interface is synchronously displayed in a second target area, wherein the third user operation is a raise gesture for the virtual reality handle, and the second target area is located in a region in the extended reality space close to the raise gesture. side; the display module further includes: displaying a gesture identifier corresponding to the raising gesture.

According to one or more embodiments of the present disclosure, the target application displays the application message in a first interface of the external terminal device; the first interface and the target terminal interface have the same interface layout.

According to one or more embodiments of the present disclosure, the display module is specifically configured to: in response to a fourth user operation, display a functional interface in the second target area of the extended reality space, and the functional interface is used to display Identification of at least one alternative application, the alternative application being an application program run by the external terminal device; in response to a triggering operation for the identification of the alternative application, determining the target application, and in the The first target area in the extended reality space renders a corresponding target terminal interface, and the target terminal interface is used to display the target application run by the external terminal device.

According to one or more embodiments of the present disclosure, the interaction module is specifically configured to: in response to a triggering operation for a target function identifier in the terminal interface, send a calling request to the external terminal device, the calling request including input parameters for executing the target function of the target application; receiving the call result corresponding to the call request returned by the external terminal device, and displaying the call result in the first target area.

According to one or more embodiments of the present disclosure, before displaying the terminal interface of the external terminal device in the first target area within the extended reality space, the display module is also used to: obtain the target image currently displayed in the extended reality space, the target image including at least one display object; and determine the first target area based on the position of at least one display object in the target image.

According to one or more embodiments of the present disclosure, when determining the first target area based on the position of at least one display object in the target image, the display module is specifically configured to: obtain each of the displayed The separation distance between objects; based on the separation distance between each of the display objects, determine at least one alternative area, the alternative area is an area that can accommodate the terminal interface; from the at least one alternative area , determine the first target area.

According to one or more embodiments of the present disclosure, when determining the first target area based on the position of at least one of the display objects in the target image, the display module is specifically configured to: The object is subjected to target recognition to obtain a target recognition object, the object category of the target recognition object is determined based on the terminal category of the external terminal device; the first target area is determined based on the outline of the target recognition object.

According to one or more embodiments of the present disclosure, the display object includes a virtual reality object and a real display object, and the display module determines the first display object based on the position of at least one of the display objects in the target image. When the target area is used, it is specifically used to: perform target recognition on each of the real display objects to obtain the target real display object, and the target real display object is the real image of the external terminal device; based on the outline of the target real display object , determine the first target area.

In a third aspect, according to one or more embodiments of the present disclosure, an electronic device is provided, including: a processor, and a memory communicatively connected to the processor;

The memory stores computer execution instructions;

The processor executes computer execution instructions stored in the memory to implement the information interaction method described in the first aspect and various possible designs of the first aspect.

In a fourth aspect, according to one or more embodiments of the present disclosure, a computer-readable storage medium is provided. Computer-executable instructions are stored in the computer-readable storage medium. When a processor executes the computer-executed instructions, Implement the information interaction method described in the first aspect and various possible designs of the first aspect.

In a fifth aspect, an embodiment of the present disclosure provides a computer program product, including a computer program, which, when executed by a processor, implements the information interaction method described in the first aspect and various possible designs of the first aspect.

The above description is only a description of the preferred embodiments of the present disclosure and the technical principles applied. Those skilled in the art should understand that the disclosure scope involved in the present disclosure is not limited to technical solutions composed of specific combinations of the above technical features, but should also cover solutions composed of the above technical features or without departing from the above disclosed concept. Other technical solutions formed by any combination of equivalent features. For example, a technical solution is formed by replacing the above features with technical features with similar functions disclosed in this disclosure (but not limited to).

Furthermore, although operations are depicted in a specific order, this should not be understood as requiring that these operations be performed in the specific order shown or performed in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Likewise, although several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of the present disclosure. Certain features that are described in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination.

Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are merely example forms of implementing the claims.

Claims (15)

1. An information interaction method, characterized by including: Display the terminal interface of the external terminal device in the first target area within the extended reality space; In response to the first user operation on the extended reality space, a call result of a target application is displayed in the terminal interface, where the target application is an application running on the external terminal device. 2. The method according to claim 1, characterized in that the first target area in the extended reality space displays a terminal interface of an external terminal device, including: Obtain push information sent by the external terminal device, where the push information represents an application message generated by a target application run by the external terminal device; Based on the push information, a corresponding target terminal interface is rendered in the first target area in the extended reality space, and the target terminal interface is used to display application messages generated by the target application. 3. The method according to claim 2, wherein the step of rendering a corresponding target terminal interface in the first target area in the extended reality space based on the push information comprises: In the second target area of the extended reality space, a notification interface is displayed. The notification interface is used to display abbreviated messages and/or reception identifiers corresponding to the push information. The area of the second target area is smaller than that of the third target area. The area of a target area; In response to the second user operation on the notification interface, the corresponding target terminal interface is rendered in the first target area within the extended reality space. 4. The method according to claim 3, characterized in that, in the second target area of the extended reality space, displaying a notification interface includes: In response to a third user operation, the notification interface is synchronously displayed in a second target area of the extended reality space, wherein the third user operation is a raising gesture for the virtual reality handle, and the second target area is located The side in the extended reality space close to the raise gesture; The method also includes: Display the gesture identifier corresponding to the raise gesture. 5. The method of claim 2, wherein the target application displays the application message in a first interface of the external terminal device; the first interface and the target terminal interface have the same Interface layout. 6. The method according to claim 1, characterized in that the first target area in the extended reality space displays a terminal interface of an external terminal device, including: In response to the fourth user operation, a functional interface is displayed in the second target area of the extended reality space. The functional interface is used to display an identification of at least one alternative application, and the alternative application is provided by the external terminal. applications running on the device; In response to a triggering operation for the identification of the candidate application, the target application is determined, and a corresponding target terminal interface is rendered in a first target area within the extended reality space, wherein the target terminal interface is used to display the target application running on the external terminal device. 7. The method according to claim 1, characterized in that, in response to the first user operation on the extended reality space, displaying the calling result of the target application in the terminal interface includes: In response to a triggering operation on a target function identifier in the terminal interface, sending a call request to the external terminal device, wherein the call request includes input parameters for executing the target function of the target application; Receive the call result corresponding to the call request returned by the external terminal device, and display the call result in the first target area. 8. The method according to any one of claims 1 to 6, characterized in that, before displaying the terminal interface of the external terminal device in the first target area in the extended reality space, it further includes: Obtaining a target image currently displayed in the extended reality space, where the target image includes at least one display object; The first target area is determined based on the position of at least one of the display objects in the target image. 9. The method according to claim 8, characterized in that the determining the first target area based on the position of at least one of the display objects in the target image comprises: Obtain the separation distance between each of the displayed objects; Determine at least one alternative area based on the separation distance between the display objects, and the alternative area is an area capable of accommodating the terminal interface; From the at least one candidate area, the first target area is determined. 10. The method of claim 8, wherein determining the first target area based on the position of at least one display object in the target image includes: Performing target recognition on each of the display objects to obtain a target recognition object, wherein the object category of the target recognition object is determined based on the terminal category of the external terminal device; The first target area is determined based on the outline of the target recognition object. 11. The method of claim 8, wherein the display object includes a virtual reality object and a real display object, and the first determination is based on the position of at least one of the display objects in the target image. A target area includes: Perform target recognition on each of the real display objects to obtain the target real display object, where the target real display object is a real image of the external terminal device; The first target area is determined based on the outline of the target real display object. 12. An information interaction device, characterized in that it includes: The display module is used to display the terminal interface of the external terminal device in the first target area within the extended reality space; An interaction module, configured to respond to a first user operation on the extended reality space and display a call result of a target application in the terminal interface, where the target application is an application running on the external terminal device. 13. An electronic device, comprising: a processor, and a memory communicatively connected to the processor; The memory stores computer execution instructions; The processor executes computer execution instructions stored in the memory to implement the information interaction method according to any one of claims 1 to 11. 14. A computer-readable storage medium, characterized in that computer-executable instructions are stored in the computer-readable storage medium. When the processor executes the computer-executable instructions, the implementation of any one of claims 1 to 11 is implemented. The information interaction method described above. 15. A computer program product, characterized by comprising a computer program that implements the information interaction method according to any one of claims 1 to 11 when executed by a processor.