TWI877952B - System, method and computer program product for synchronizing virtual and real world scene content - Google Patents

Abstract

A system, a method, and a computer program product for synchronizing virtual and real world scene content are provided. A virtual scene is rendered, and a virtual representative of an AR device and a virtual avatar of a VR device are rendered to be placed in the virtual scene. Parameters and their timestamps from the AR device and the VR device are received to be applied to the virtual representative and the virtual avatar, thereby controlling a viewpoint picture of respective virtual camera of the virtual representative and the virtual avatar. Afterwards, the viewpoint picture of the virtual camera of the virtual representative and the viewpoint picture of the virtual camera of the virtual avatar are transmitted to the AR device and the VR device, respectively, allowing users of both the AR device and the VR device to interact with each other on the same timeline.

Description

Systems, methods and computer program products for synchronizing virtual and real-world scene content

This case is about virtual reality and augmented reality technology, and more specifically, about systems, methods and computer program products for synchronizing virtual and real world scene content.

Holding exhibitions and performances in the virtual world has become a new form of display. Combined with remote rendering technology, high-quality visual effects can be achieved, allowing the audience to enjoy realistic and detailed virtual world exhibitions and performances. This method provides users with new viewing options.

For example, when providing online games, the server can stream virtual scene images to the user's augmented reality (AR) or virtual reality (VR) device through remote rendering technology for an immersive experience. In addition, AR devices or VR devices can also interact with virtual scenes. Furthermore, the system can capture the activity images held in the real world, build them in the virtual world, and allow the audience to participate as virtual characters. However, the above-mentioned previous technologies only explain how to interact with virtual scenes or virtual objects, but have not yet solved how to use remote rendering technology to allow people in the real world to interact with people in the virtual world.

To solve the above problems and other problems, this case discloses a system, method, computer program product, and computer-readable recording medium for synchronizing virtual and real-world scene content.

The present invention discloses a system for synchronizing virtual and real world scene contents, comprising: a virtual avatar rendering module for rendering a virtual avatar of a virtual reality device; a virtual representative rendering module for rendering a virtual representative of an augmented reality device; a virtual scene rendering module for rendering a virtual scene; A virtual scene is formed to place the virtual avatar and the virtual representative into the virtual scene; a parameter receiving and applying module receives parameters and timestamps from the virtual reality device to apply to the virtual avatar, so that the virtual camera of the virtual avatar is updated according to the parameters and timestamps. The parameter receiving and applying module receives the parameters and timestamp from the augmented reality device to apply to the virtual representation, so that the virtual camera of the virtual representation updates the perspective of the virtual camera of the virtual representation according to the parameters and timestamp; and the streaming transmission module transmits the The view angle of the virtual camera of the virtual avatar is transmitted to the virtual reality device, and the streaming module transmits the view angle of the virtual camera of the virtual representation to the augmented reality device, so that the virtual reality device and the user of the augmented reality device can interact on the same time axis.

A method for synchronizing virtual and real world scene contents disclosed in the present case includes: rendering a virtual scene; rendering a virtual avatar of a virtual reality device to place the virtual avatar in the virtual scene; rendering a virtual representation of an augmented reality device to place the virtual representation in the virtual scene; receiving parameters and a timestamp from the virtual reality device to apply to the avatar, so that a virtual camera of the avatar updates the view angle of the virtual camera of the avatar according to the parameters and the timestamp. receiving parameters and timestamps from the augmented reality device to apply to the virtual representative so that the virtual camera of the virtual representative updates the view of the virtual camera of the virtual representative according to the parameters and timestamps; and transmitting the view of the virtual camera of the virtual avatar to the virtual reality device and transmitting the view of the virtual camera of the virtual representative to the augmented reality device, so that the virtual reality device and the user of the augmented reality device can interact on the same timeline.

A computer program product disclosed in this case is used to execute the method disclosed in this case for synchronizing virtual and real world scene contents by loading the program into a computer.

The computer-readable recording medium disclosed in this case stores instructions, and can be used to execute the computer-readable recording medium through a computing device or a computer through a processor and/or a memory, so as to execute the method disclosed in this case for synchronizing virtual and real world scene contents when executing the computer-readable recording medium.

In one embodiment, an image and a timestamp of a user wearing the augmented reality device taken by an image capture and transmission device at the scene can be received, and the position and movement posture of the user can be analyzed, so as to generate the virtual representative according to the analyzed movement posture, and put the virtual representative into the virtual scene according to the analyzed position.

In one embodiment, when transmitting the view angle picture of the virtual camera of the virtual representative, the background of the virtual avatar in the view angle picture of the virtual camera of the virtual representative is first removed to form a virtual avatar background-removed image, and then the virtual avatar background-removed image is streamed to the augmented reality device, so that the augmented reality device can overlay the virtual avatar background-removed image on the real scene image taken by the augmented reality device.

In one embodiment, an image and a time stamp of a spectator captured by an image capture and transmission device at the scene may be received, and the position and movement posture of the spectator may be analyzed to generate a virtual character according to the analyzed movement posture, and the virtual character may be placed in the virtual scene according to the analyzed position, and the virtual representative of the virtual character may be transmitted in the virtual camera. When the virtual character in the virtual camera's perspective image is viewed, the background of the virtual character in the virtual camera's perspective image represented by the virtual representation is first removed to form a virtual character background-removed image, and then the virtual character background-removed image is streamed to the augmented reality device so that the augmented reality device can overlay the virtual character background-removed image on the real scene image shot by the augmented reality device.

In one embodiment, the parameter of the virtual reality device is the degree of freedom data of the virtual reality device itself, and the parameter of the augmented reality device is the degree of freedom data of the augmented reality device itself, and wherein the degree of freedom data of the virtual reality device is used to control the view angle of the virtual camera of the virtual avatar, and the degree of freedom data of the augmented reality device is used to control the view angle of the virtual camera of the virtual representation.

In one embodiment, the parameter of the virtual reality device is the controller data of the virtual reality device, and the parameter of the augmented reality device is the controller data of the augmented reality device, and wherein the controller data of the virtual reality device is used to control the action of the avatar, and the controller data of the augmented reality device is used to control the action of the virtual representation.

In other words, in the system, method, computer program product, and computer-readable recording medium for synchronizing virtual and real world scene contents in this case, the remote rendering server (i.e., the system described in this case) receives images and timestamps of live scenes captured by several cameras in a specific area of a real-world activity. The remote rendering server is responsible for rendering the virtual scene and the virtual avatar of the VR device user, and analyzes the audience body and its posture based on the content in the camera image, and generates virtual characters (non-AR device users) and virtual representatives (AR device users) based on this and places them in the virtual scene. In addition, AR/VR devices can apply their DoF data, controller data, and their timestamps to the virtual cameras of virtual representatives and avatars in real time, so the virtual camera can update the view of the virtual camera as the user moves the AR/VR device. Furthermore, the remote rendering server synchronously streams the view or interactive images captured by the virtual cameras of the virtual representatives and avatars back to the user's AR/VR device to ensure that the images seen by users of both devices are on the same timeline.

10: Image capture and transmission device

20: System

21: Virtual scene rendering module

22: Avatar rendering module

23: Virtual representative rendering module

24: Image reception and analysis module

25: DoF parameter receiving application module

26: Action parameter receiving application module

27: Streaming transmission module

30:AR device

31: DoF parameter transmission module

32: Action parameter transmission module

33: Image stream receiving module

34: Virtual and real image synchronization superposition module

35: Camera module

40: VR device

41: DoF parameter transmission module

42: Action parameter transmission module

43: Image stream receiving module

S201~S214: Steps

1: Camera

2: Remote rendering server

3:AR device

4: VR device

FIG1 is a block diagram of an embodiment of the system for synchronizing virtual and real world scene contents in the present case.

Figure 2 is a schematic diagram of an implementation example of the method for synchronizing virtual and real world scene contents in this case.

FIG3 is a schematic diagram of a specific implementation example of the system and method for synchronizing virtual and real world scene contents in this case.

The following specific implementation examples are used to illustrate the implementation of this case. People familiar with this technology can easily understand other advantages and effects of this case from the content disclosed in this article. The structures, ratios, sizes, etc. shown in the attached figures of this manual are only used to match the content disclosed in the manual for people familiar with this technology to understand and read, and are not used to limit the implementation conditions of this case. Therefore, any modification, change or adjustment should still fall within the scope of the technical content disclosed in this case without affecting the effects and purposes that can be achieved by this case.

As used herein, the terms "include", "comprising", "having", "containing" or any other variations thereof are intended to cover a non-exclusive inclusion. Unless otherwise indicated, singular forms such as "a", "an", "the" may also be used in the plural, and "or", "and/or" and the like may be used interchangeably.

Please refer to FIG. 1, which is a schematic diagram of the architecture of the system for synchronizing virtual and real world scene contents in this case. The system 20 includes a virtual scene rendering module 21, a virtual avatar rendering module 22, a virtual representative rendering module 23, an image receiving and analyzing module 24, a DoF parameter receiving and applying module 25, an action parameter receiving and applying module 26, and a streaming transmission module 27.

System 20 may be a server (e.g., a remote rendering server), wherein each module or unit may be software, hardware, or firmware; if it is hardware, it may be a processing unit, processor, or computer host with data processing and computing capabilities; if it is software or firmware, it may include instructions executable by a processing unit, processor, computer, or computer host, and may be installed on the same hardware device or distributed on multiple different hardware devices.

The virtual scene rendering module 21 is used to render a virtual scene. In one embodiment, the virtual scene can be a three-dimensional space consistent with the real world (such as the metaverse).

The avatar rendering module 22 is used to render the avatar, and the virtual scene rendering module 21 puts the avatar into the virtual scene. In one embodiment, the avatar corresponds to the user wearing the VR device 40.

The virtual representative rendering module 23 is used to render a virtual representative, and the virtual scene rendering module 21 places the virtual representative into the virtual scene. In one embodiment, the virtual representative corresponds to a user wearing the AR device 30.

The image receiving and analyzing module 24 is used to receive the live image and time stamp from the image capture and transmission device 10 (such as a live camera), thereby analyzing the position and action posture of the live audience from the live image and time stamp, so that the system 20 can generate a virtual representative or a virtual character according to the analyzed action posture, and put the virtual representative or virtual character into the virtual scene according to the analyzed position.

In one embodiment, for the on-site audience wearing the AR device 30, the image receiving and analyzing module 24 receives the image and timestamp of the user wearing the AR device 30 from the image capturing and transmitting device 10, and analyzes the position and action posture of the user. The virtual representative rendering module 23 generates the virtual representative according to the action posture analyzed by the image receiving and analyzing module 24, so that the virtual scene rendering module 21 puts the virtual avatar into the virtual scene according to the position analyzed by the image receiving and analyzing module 24.

In another embodiment, for on-site spectators who do not wear the AR device 30, the image receiving and analyzing module 24 receives the image and timestamp of the on-site spectators from the image capture and transmission device 10, and analyzes the position and action posture of the on-site spectators. The system 20 generates the virtual character according to the action posture analyzed by the image receiving and analyzing module 24, so that the virtual scene rendering module 21 puts the virtual character into the virtual scene according to the position analyzed by the image receiving and analyzing module 24.

In other words, on the system 20, the received camera images are used to identify the coordinate position and action posture of the audience body through image recognition, and the analyzed action posture (such as raising a hand) is applied to the virtual character, wherein the coordinate information can be calculated from multiple camera images at the same time point at multiple angles, and the position of the audience body from the camera and the relative position in the scene are calculated, and finally the virtual character is placed in the relative position of the virtual three-dimensional space (corresponding to the real world scene). Similarly, the virtual avatar is also rendered in a similar way at the specified coordinate position in the virtual three-dimensional space.

The parameter receiving and applying module includes the DoF parameter receiving and applying module 25 and the motion parameter receiving and applying module 26.

The DoF parameter receiving and applying module 25 is used to receive the degrees of freedom (DoF) data parameters and timestamp from the DoF parameter transmitting module 41 of the VR device 40, and apply them to the avatar, so that the virtual camera of the avatar updates the viewing angle of the virtual camera of the avatar according to the parameters and timestamp, that is, the DoF parameter receiving and applying module 25 uses the DoF data transmitted from the VR device 40 to control the viewing angle of the virtual camera of the avatar. In addition, the DoF parameter receiving and applying module 25 receives the parameters and timestamps from the DoF parameter transmitting module 31 of the AR device 30 to apply to the virtual representation, so that the virtual camera of the virtual representation updates the view angle of the virtual camera of the virtual representation according to the parameters and timestamps, that is, the DoF parameter receiving and applying module 25 uses the DoF data transmitted from the AR device 30 to control the view angle of the virtual camera of the virtual representation.

The action parameter receiving and applying module 26 is used to receive the controller data parameters and timestamp from the action parameter transmitting module 42 of the VR device 40 to apply to the virtual avatar, so that the virtual avatar updates the action of the virtual avatar according to the action data and timestamp, that is, the action parameter receiving and applying module 26 uses the controller data transmitted from the VR device 40 to control the action of the virtual avatar. In addition, the action parameter receiving and applying module 26 is used to receive the controller data parameters and timestamps from the action parameter transmitting module 32 of the AR device 30, and apply them to the virtual representative, so that the virtual representative updates the action of the virtual representative according to the action data and timestamp, that is, the action parameter receiving and applying module 26 uses the controller data transmitted from the AR device 30 to control the action of the virtual representative.

In this way, no matter the user of the AR device 30 or the VR device 40, since their corresponding virtual representatives and virtual avatars are in the same virtual three-dimensional space, the virtual cameras on each other can capture each other's movements and the scene perspective.

The streaming module 27 transmits the view of the virtual camera of the virtual avatar to the VR device 40, so that the image streaming receiving module 43 of the VR device 40 can receive and play it. In addition, the streaming module 27 transmits the view of the virtual camera of the virtual representation to the AR device 30, so that the image streaming receiving module 33 of the AR device 30 can receive and play it. In this way, the users of the VR device 40 and the AR device 30 can interact on the same time axis to achieve interaction between the virtual world and the real world.

In one embodiment, when transmitting the view angle picture of the virtual camera represented by the virtual device, the streaming transmission module 27 removes the background of the virtual avatar in the view angle picture of the virtual camera represented by the virtual device to form a virtual avatar background-removed image, and then streams the virtual avatar background-removed image to the AR device 30, so that the virtual image synchronization superposition module 34 of the AR device 30 can superimpose the virtual avatar background-removed image on the real scene image captured by the camera module 35 of the AR device 30.

In another embodiment, the streaming module 27 removes the background of the virtual character in the view angle of the virtual camera represented by the virtual character to form a virtual character background-removed image, and then streams the virtual character background-removed image to the AR device 30, so that the virtual image synchronization overlay module 34 of the AR device 30 overlays the virtual character background-removed image on the real scene image captured by the camera module 35 of the AR device 30.

In one embodiment, the AR device 30 and VR device 40 (such as an AR/VR head-mounted device or an MR helmet) shown in FIG. 1 may be, for example, Apple Vision Pro (AR as the main function, VR as the auxiliary function) or Meta Oculus Quest3 (VR as the main function, AR as the auxiliary function). The AR device 30 and VR device 40 have a built-in or loaded Metaverse software application. Through the Metaverse software application, the DoF data of the AR device 30 and VR device 40 are transmitted to the DoF parameter receiving and applying module 25 in real time through the DoF parameter transmission modules 31 and 41 to be applied to the virtual camera on the virtual representative and the virtual avatar to update the perspective of the virtual camera. In addition, the Metaverse software application also transmits the DoF data of the AR device 30 and VR device 40 to the DoF parameter receiving and applying module 25 in real time. The action data (such as gestures) detected by the controllers on the AR device 30 and the VR device 40 are transmitted to the action parameter receiving and applying module 26 in real time through the action parameter transmitting modules 32 and 42 to be applied to the virtual representative and the avatar, thereby ensuring that the perspective images or interactive images presented on the AR device 30 and the VR device 40 are on the same timeline.

In addition, the image capture and transmission device 10 can capture live images including people and objects and can capture live sounds and transmit them to the system 20 so that the system 20 can render a virtual scene including images and sounds, and then stream them to the AR device 30 and the VR device 40.

According to one or more embodiments of the system for synchronizing virtual and real world scenes disclosed in the above case, the present case uses remote rendering and synchronization technology to allow users in the real world and the virtual world to see each other and interact with each other through AR devices and VR devices. That is, when a user logs into the metaverse through a VR device , can become a virtual avatar, and can interact with other virtual avatars, virtual scenes, virtual objects, virtual representatives, virtual characters, etc., and users in the real world can interact with virtual avatars, virtual scenes, virtual objects, virtual representatives, virtual characters, etc. in the virtual world through virtual representatives even without logging into the virtual world.

Please refer to Figure 2, which is a schematic diagram of the steps of the method for synchronizing virtual and real world scene contents in this case. It should be noted that the method shown in Figure 2 can be executed by the system 20 or server shown in Figure 1.

Step S201, rendering a virtual scene. In one embodiment, the virtual scene may be a virtual three-dimensional space created according to a real scene or another virtual world, and the rendered virtual scene may include sound or other multimedia information.

Steps S202~S207 are about the virtual representation of the AR device. Steps S208~S213 are about the virtual avatar of the VR device.

Steps S202 and S203 render a virtual representation of the AR device and place the virtual representation in the virtual scene; steps S208 and S209 render a virtual avatar of the VR device and place the virtual avatar in the virtual scene.

In one embodiment, the virtual representative can be generated based on the action posture analyzed from the image of the live audience wearing the AR device, which can be captured by the live camera for analysis, and then the virtual representative is placed at the relative position of the virtual scene according to the position analyzed from the image of the live audience wearing the AR device.

In another embodiment, a virtual character corresponding to the on-site audience or a virtual object corresponding to the on-site object can be rendered, and the virtual character can be generated based on the action posture analyzed from the on-site audience image, and similarly, the rendered virtual character or object can be placed in the virtual scene based on the analyzed position.

In another embodiment, the virtual avatar can be determined by the user of the VR device.

Steps S204 and S205 receive parameters and timestamps from the AR device, and apply the parameters and timestamps of the AR device to the virtual representation; Steps S210 and S211 receive parameters and timestamps from the VR device, and apply the parameters and timestamps of the VR device to the virtual avatar.

In one embodiment, the parameter and timestamp refer to the DoF data and timestamp of the AR device or VR device itself, which can be received to control the perspective of the virtual representative or avatar. In another embodiment, the parameter and timestamp refer to the controller data and timestamp of the AR device or VR device, which can be received to control the action of the virtual representative or avatar.

In step S206, the virtual camera of the virtual representative updates the view angle of the virtual camera of the virtual representative according to the parameter and the timestamp; in step S212, the virtual camera of the avatar updates the view angle of the virtual camera of the avatar according to the parameter and the timestamp. That is, when the user of the AR device or VR device moves his head, the view angle of the virtual camera of the corresponding virtual representative will also change synchronously. At this time, regardless of whether they are users of AR or VR devices, since their corresponding virtual representatives and virtual avatars are in the same virtual three-dimensional space, the virtual cameras on each other can capture each other's movements and the scene perspective.

Step S207, transmitting the view angle of the virtual camera of the virtual representative to the AR device; Step S213, transmitting the view angle of the virtual camera of the virtual avatar to the VR device.

In one embodiment, the view and sound of the virtual camera of the avatar can be streamed to the VR device via the WebRTC video streaming protocol, and the virtual camera view of the virtual representative streams the background-removed images of the virtual avatar and virtual character to the AR device, and the AR device superimposes these background-removed images on the real scene captured by its camera to present an AR effect.

In step S214, the users of the AR device and the VR device interact on the same timeline to achieve interaction between the virtual world and the real world.

In addition to one or more of the above embodiments, this case provides a computer program product that executes the above method after the program is loaded into a computer. In addition, the computer program (product) can be stored in a recording medium or directly transmitted and provided on the Internet. The computer program (product) is a computer-readable program and is not limited to an external form. The computer includes but is not limited to an electronic device with a processor, such as a mobile phone or a tablet.

In addition, the present case also provides a computer-readable recording medium, which is applied to a computing device or a computer having a processor and/or a memory, and the computer-readable recording medium stores instructions, and the computing device or the computer can execute the computer-readable recording medium through the processor and/or the memory, so as to execute the above-mentioned method and/or content when executing the computer-readable recording medium. The computer-readable recording medium (such as a hard disk, a floppy disk, an optical disk, a USB flash drive) stores the computer program (product). In one embodiment, the computer-readable recording medium is a non-transitory computer-readable recording storage medium.

Next, based on one or more embodiments of the system, method, computer program product, and computer-readable recording medium for assisting multiple people in selecting activities in a virtual world, a specific embodiment of a demonstration scenario is provided here.

Please refer to Figure 3. Several cameras 1 are set up around a specific area of the event site to shoot the audience in the area, and transmit the images and timestamps to the remote rendering server 2 for analysis via the Internet.

The remote rendering server 2 uses image recognition to identify the position and gesture of the audience, and applies the analyzed gesture (such as raising a hand) to the virtual character. The coordinate information can be calculated from the camera image at the same time point in multiple angles to calculate the position of the audience body from the camera 1 and the relative position in the scene, and finally place the virtual character in the relative position of the virtual three-dimensional space. Similarly, if there are audience members wearing AR devices 3, the corresponding virtual representatives are also placed in the relative position of the virtual three-dimensional space. The remote rendering server 2 renders a virtual three-dimensional space that is consistent with the real world and renders a virtual avatar of the VR device 4, and then puts it into the virtual three-dimensional space.

AR device 3 or VR device 4 refers to a device that can have a built-in Metaverse application (APP) or can be downloaded through the Internet. Audiences of real-world events and audiences experiencing virtual events can wear AR device 3 and VR device 4 respectively when interacting. The Metaverse APP transmits the DoF (Degrees of Freedom) data of AR device 3 and VR device 4 and the information of controllers or gestures to the remote rendering server 2 in real time through the Internet.

The remote rendering server 2 applies the gestures or actions detected by the controllers of the AR device 3 and the VR device 4 to the virtual representative and the virtual avatar, respectively, so that the virtual representative and the virtual avatar can make corresponding changes according to the detected gestures or actions, and the DoF data of the AR device 3 and the VR device 4 are applied to the virtual cameras on the virtual representative and the virtual avatar. The AR device 3 and VR device 4 on the head can update the virtual camera's view in real time. At this time, no matter the user of the AR device 3 or VR device 4, since their corresponding virtual representatives and virtual avatars are in the same virtual three-dimensional space, the virtual cameras on each other can capture each other's movements and the scene perspective, achieving interaction between the virtual world and the real world.

The virtual camera images of the virtual representative and the avatar are encoded by the remote rendering server 2, and then the images and sounds are streamed to the APP in the AR device 3 and VR device 4 in real time through the WebRTC communication protocol for decoding, and finally presented on the display screen in the AR device 3 and VR device 4, so that the interactive images of the users of the AR device 3 and VR device 4 are guaranteed to be real-time and synchronized. For the user of the VR device 4, he can see the virtual scene, the virtual character corresponding to the audience, and the virtual representative corresponding to the AR device 3 through the display screen in the device, that is, the user of the VR device 4 is in the first-person perspective. For users of AR device 3, remote rendering server 2 will only stream background-removed images with virtual avatars, that is, the audience can see themselves in the AR perspective screen. Then, AR device 3 will overlay the streamed background-removed images with the real-life images captured by the camera to present an AR effect.

In summary, the system, method, computer program product, and computer-readable recording medium for synchronizing virtual and real world scene contents in this case have the following characteristics or functions:

1. This solution can capture the audience in real-world activities through camera shooting and remote rendering server analysis. Based on the analysis results, the entity and its posture are established as virtual representatives and virtual characters, and are configured in the virtual world scene.

2. This case allows users of VR devices to have corresponding virtual avatars in the virtual world and interact with users of AR devices in the real world through a virtual representative as a medium.

3. This solution can apply the DoF data of the AR/VR device to the virtual representative on the remote rendering server and the virtual camera on the avatar in real time, and simultaneously stream the interactive images captured by the virtual camera back to the AR/VR device for presentation, achieving interaction between the virtual world and the real world.

The above embodiments are only illustrative of the effects of this case, and are not intended to limit this case. Anyone familiar with this technology can modify and change the above embodiments without violating the spirit and scope of this case. Therefore, the scope of protection of this case should be as listed in the scope of the patent application described below.

S201~S214: Steps

Claims (13)

A system for synchronizing virtual and real world scene contents includes: a virtual avatar rendering module for rendering a virtual avatar of a virtual reality device; a virtual representative rendering module for rendering a virtual representative of an augmented reality device; a virtual scene rendering module for rendering a virtual scene; The virtual avatar and the virtual representative are placed in the virtual scene; the parameter receiving and applying module receives the parameters and timestamp from the virtual reality device to apply to the virtual avatar, so that the virtual camera of the virtual avatar updates the view angle of the virtual camera of the virtual avatar according to the parameters and timestamp. The parameter receiving and applying module receives the parameters and timestamp from the augmented reality device to apply to the virtual representation, so that the virtual camera of the virtual representation updates the view of the virtual camera of the virtual representation according to the parameters and timestamp; and the streaming module transmits the view of the virtual camera of the virtual avatar to the virtual reality device, and the streaming module transmits the view of the virtual camera of the virtual representation to the augmented reality device, so that the virtual reality device and the user of the augmented reality device can interact on the same timeline. The system as described in claim 1 further includes an image receiving and analyzing module, which receives an image and a timestamp of a user wearing the augmented reality device taken by an image capture and transmission device on site, and analyzes the position and movement posture of the user from the image and the timestamp of the user wearing the augmented reality device, so that the virtual representative rendering module generates the virtual representative according to the movement posture analyzed by the image receiving and analyzing module, and the virtual scene rendering module places the virtual representative into the virtual scene according to the position analyzed by the image receiving and analyzing module. The system as claimed in claim 1, wherein the streaming module removes the background of the virtual avatar in the view angle of the virtual camera represented by the virtual representation to form a virtual avatar background-removed image, thereby streaming the virtual avatar background-removed image to the augmented reality device, so that the augmented reality device can overlay the virtual avatar background-removed image on the real scene image captured by the augmented reality device. The system as claimed in claim 1 further comprises an image receiving and analyzing module, which receives images and timestamps of spectators captured by an image capture and transmission device at the scene, and analyzes the positions and movement postures of the spectators from the images and timestamps of the spectators, so that the system generates a virtual character according to the movement postures analyzed by the image receiving and analyzing module, and the virtual scene rendering module generates a virtual character according to the movement postures analyzed by the image receiving and analyzing module. The virtual character is placed into the virtual scene at the position analyzed by the group, and the streaming module removes the background of the virtual character in the view angle of the virtual camera represented by the virtual representation to form a virtual character background-removed image, thereby streaming the virtual character background-removed image to the augmented reality device, so that the augmented reality device can overlay the virtual character background-removed image on the real scene image taken by the augmented reality device. The system as described in claim 1 further includes a degree of freedom parameter receiving and applying module, wherein the parameter of the virtual reality device is the degree of freedom data of the virtual reality device itself, and the parameter of the augmented reality device is the degree of freedom data of the augmented reality device itself, and wherein the degree of freedom parameter receiving and applying module uses the degree of freedom data of the virtual reality device to control the viewing angle of the virtual camera of the virtual avatar, and uses the degree of freedom data of the augmented reality device to control the viewing angle of the virtual camera of the virtual representation. The system as described in claim 1 further includes an action parameter receiving and applying module, wherein the parameter of the virtual reality device is the controller data of the virtual reality device, and the parameter of the augmented reality device is the controller data of the augmented reality device, and wherein the action parameter receiving and applying module uses the controller data of the virtual reality device to control the action of the virtual avatar, and uses the controller data of the augmented reality device to control the action of the virtual representative. A method for synchronizing virtual and real world scene contents is performed by a system for synchronizing virtual and real world scene contents, the method comprising: causing the system to render a virtual scene; causing the system to render a virtual avatar of a virtual reality device to place the virtual avatar in the virtual scene; causing the system to render a virtual representation of an augmented reality device to place the virtual representation in the virtual scene; causing the system to receive parameters and timestamps from the virtual reality device to apply to the avatar, causing the virtual camera of the avatar to update according to the parameters and timestamps The system receives the parameters and timestamp from the augmented reality device and applies them to the virtual representation, so that the virtual camera of the virtual representation updates the perspective of the virtual camera of the virtual representation according to the parameters and timestamp; and The system transmits the view angle of the virtual camera of the virtual avatar to the virtual reality device and transmits the view angle of the virtual camera of the virtual representative to the augmented reality device, so that the virtual reality device and the user of the augmented reality device can interact on the same time axis. The method as described in claim 7, wherein the step of causing the system to render a virtual representation of the augmented reality device to place the virtual representation in the virtual scene further includes: causing the system to receive an image and a timestamp of a user wearing the augmented reality device taken by an image capture and transmission device at the scene; causing the system to analyze the position and movement posture of the user wearing the augmented reality device from the image and the timestamp; and causing the system to generate the virtual representation according to the analyzed movement posture, so as to place the virtual representation in the virtual scene according to the analyzed position. The method as described in claim 7, wherein the step of causing the system to transmit the view angle image of the virtual camera of the virtual representative to the augmented reality device comprises: causing the system to remove the background of the virtual avatar in the view angle image of the virtual camera of the virtual representative to form a virtual avatar background-removed image; and causing the system to stream the virtual avatar background-removed image to the augmented reality device, so that the augmented reality device can overlay the virtual avatar background-removed image on the real scene image captured by the augmented reality device. The method as described in claim 7 further includes: allowing the system to receive an image and a time stamp of a spectator captured by an image capture and transmission device at the scene; allowing the system to analyze the position and movement posture of the spectator from the image and the time stamp; allowing the system to generate a virtual character based on the analyzed movement posture, and placing the virtual character according to the analyzed position. into the virtual scene; the system removes the background of the virtual character in the view angle of the virtual camera represented by the virtual character to form a virtual character background-removed image; and the system streams the virtual character background-removed image to the augmented reality device so that the augmented reality device can overlay the virtual character background-removed image on the real scene image captured by the augmented reality device. The method as claimed in claim 7, wherein the parameter of the virtual reality device is the degree of freedom data of the virtual reality device itself, and the parameter of the augmented reality device is the degree of freedom data of the augmented reality device itself, and wherein the degree of freedom data of the virtual reality device is used to control the viewing angle of the virtual camera of the virtual avatar, and the degree of freedom data of the augmented reality device is used to control the viewing angle of the virtual camera of the virtual representation. The method of claim 7, wherein the parameter of the virtual reality device is controller data of the virtual reality device, and the parameter of the augmented reality device is controller data of the augmented reality device, and wherein the controller data of the virtual reality device is used to control the movement of the avatar, and the controller data of the augmented reality device is used to control the movement of the virtual representation. A computer program product, which is loaded into a computer to execute the method described in any one of claim 7-12.