WO2018233533A1 - Online integration of augmented reality editing devices and systems - Google Patents

Abstract

An editing device and system for on-line integrated augmented reality. The editing device can achieve real-time and on-line acquisition and editing of an AR temporary file. The editing system of the present invention comprises a cloud server, an editor end, and a mobile device end. The editor end can select an AR temporary file and an interaction object in the cloud server and store the same as an AR target file after editing is completed. The editor end uploads the AR target file to the cloud server. The mobile device end acquires, from the cloud server, the AR target file and displays the acquired AR target file. In addition, the mobile device end can capture different content, use the captured content to generate a new AR temporary file, and deliver the new AR temporary file to the cloud server for storage.

Description

Online integration of augmented reality editing devices and systems

The present application claims priority to Chinese Patent Application No. CN201710487618.9, filed on Jun. 23,,,,,,,,,,,,,,,,,,,,

Technical field

The present invention relates to a media editing apparatus and system, and in particular to an editing apparatus and system for integrating an augmented reality on-line.

Background technique

As mobile computing power increases, handsets become thinner, smaller, and can capture and play back images or movies. However, in the prior art, still images or dynamic movies can only display the current scene and cannot interact with the user. In other words, the user can only view the image or the movie simply. In order to provide the interactive function of the images, some manufacturers began to add the functions of Augmented Reality (AR). The AR function is a technique for calculating the position and angle of a camera image in real time, plus the corresponding image, image, sound or other multimedia object. The goal of this technology is to apply virtual world images to virtual objects on the display screen, allowing users to interact with virtual objects.

In the prior art, the development of virtual objects or images of AR needs to be jointly developed by 3D artists and related professional engineers. Therefore, AR technology requires high development costs and a large amount of manpower input; this has caused the entry threshold of AR to be reduced, hindering the subsequent development of VR technology.

Summary of the invention

The invention provides an editing device for online integration of augmented reality. The editor side of the editing device can obtain the AR temporary storage file from the online in real time, and can also apply the AR temporary storage file to the captured environmental movie. The editor side can also upload the edited AR target file to the cloud server, so that the AR target file becomes a new AR temporary file in the cloud server.

The online integrated augmented reality editing device of the present invention comprises a communication unit, a display unit, an object operating unit and a processing unit. The communication unit network is connected to the cloud server, and the AR target file or the interactive object can be obtained from the cloud server; the display unit is used to display the environment image, the AR temporary storage file, the AR target file or the interactive object; and the object operation unit is used to control the interactive object. The display position and the display angle in the display unit; the processing unit and the communication unit, the display unit and the object operation unit are electrically connected; the processing unit can drive the communication unit to obtain the phase of the identification information from the cloud server according to the information of the identification information. Corresponding index information of the AR temporary storage file, and downloading any AR target file according to the index information; the processing unit may generate an AR temporary storage file according to the interactive object and each step generated in the editing process of the selected AR target file. Select any AR temporary file and generate an AR target file, and the processing unit sends the generated AR target file to the cloud server through the communication unit.

The invention also provides an editing system for online integration of augmented reality, the editing system comprising a cloud server, at least one editor end and a mobile device end.

The cloud server is used to record multiple AR target files, multiple interactive objects or user databases, and the user database records the identification information of each user. The editor end network is connected to the cloud server, and the editor end comprises a first communication unit, a first storage unit, a first display unit, a first processing unit and an object operation unit, a first processing unit and a first communication unit, and a first The storage unit, the first display unit and the object operation unit are electrically connected, and the first communication unit acquires index information of the AR target file corresponding to the identification information from the cloud server. The first storage unit is configured to store an object editing interface, at least one AR temporary file, an interactive object, or an AR target file. The first display unit is used to edit the object editing interface and the selected interactive object. The object operating unit is configured to control a placement position and a display angle of the interactive object in the first display unit, and generate an AR temporary storage file corresponding to the interactive object in the editing process. The first processing unit generates an AR target file according to the interactive object and the selected AR temporary storage file, and the first processing unit sends the generated AR target file to the cloud server through the first communication unit.

The mobile device includes a second processing unit, a second communication unit, a second display unit, an image selection unit and a second storage unit, a second processing unit and a second communication unit, a second display unit, an image selection unit, and a second storage unit. The unit is electrically connected. The second communication unit network is connected to the cloud server, the image selection unit is used for capturing the target to be identified, and the second processing unit sends the target to be identified to the cloud server, and the cloud server identifies whether the target to be identified is a plurality of interactive objects. First, the cloud server sends the AR target file corresponding to the target to be identified to the mobile device end, and plays the received AR target file through the second display unit.

The online integrated and augmented reality editing device and system provided by the invention can enable the editor to obtain different AR temporary storage files from the line, and the AR target file can also be uploaded to the cloud server after the editor side is edited. . For different editors, the editor can trade or exchange various materials in the AR scratch file, such as interactive objects or scenes. The mobile device can download the existing AR target file, or send the AR target file to other users through community sharing. In addition, the mobile device can also capture different objects or images, so that the captured target is converted into the content of the AR temporary file.

DRAWINGS

The present invention will be further described in detail below in conjunction with the drawings and specific embodiments.

1 is a schematic diagram of a system architecture of the present invention.

2 is a schematic diagram of an editing process of the AR temporary storage file of the present invention.

FIG. 3 is a schematic diagram of selecting an interactive object in the object editing interface of the present invention.

4 is a schematic diagram of selecting an AR temporary file in the object editing interface of the present invention.

FIG. 5 is a schematic diagram of the operation of the second image recognition program of the present invention.

Figure 6 is a schematic diagram of the identification code input operation of the present invention.

FIG. 7 is a schematic diagram of the operation of sharing by the community of the present invention.

among them:

Editing system 100; cloud server 110; AR temporary file 111; AR target file 112; user database 114; identification information 115; index information 116; editor terminal 120; first communication unit 121; first storage unit 122 The first display unit 123; the first processing unit 124; the object operating unit 125; the interactive object 126; the object editing interface 127; the mobile device end 130; the second processing unit 131; the second communication unit 132; Image selection unit 134; input unit 135; second storage unit 136; first image recognition program 141; second image recognition program 142; community sharing program 143; simple editing program 144; object to be identified 151; community sharing button 152 ; Community icon 153.

Detailed ways

FIG. 1 is a schematic diagram showing the architecture and composition of the system of the present invention. The online integrated augmented reality editing system 100 of the present invention includes a cloud server 110, at least one editor terminal 120 and a mobile device terminal 130. Both the editor terminal 120 and the mobile device terminal 130 can be connected to the cloud server 110 via a network. The composition structure of the present invention comprises two parts: an editing unit and an execution unit; the editing unit includes an editor terminal 120 for establishing or editing an augmented reality; the execution unit includes a mobile device end 130 for executing or displaying augmented reality content. .

The cloud server 110 is used to record information including a plurality of AR target files 112, a plurality of interactive objects 126, or a user database 114. The AR temporary storage file 111 is used for recording temporary data of each step in the user editing process, and the steps are the editing process of the interactive object 126. The user can select the interactive object 126 and the at least one AR temporary file 111 to be used to generate the AR target file 112. The user database 114 is used to record the identification information 115 of each user, and each user can have a plurality of AR temporary files 111 or AR target files 112. For a user who does not meet the set authority, the user can download the AR target file 112; the user can only view the downloaded AR target file 112, and cannot view the AR target file 112 at the cloud server 110. The AR temporary file 111 to which the AR target file 112 at the cloud server 110 belongs cannot be edited or edited. For the user who has the viewing right, the user can open the AR target file 112 or edit the AR temporary file 111 after downloading the AR target file.

The editor terminal 120 includes a first communication unit 121, a first storage unit 122, a first display unit 123, a first processing unit 124, and an object operation unit 125. The first processing unit 124 is electrically connected to the first communication unit 121, the first storage unit 122, the first display unit 123, and the object operating unit 125. The mobile device end 130 includes a second processing unit 131, a second communication unit 132, a second display unit 133, an image selection unit 134, an input unit 135, and a second storage unit 136. The second processing unit 131 is electrically connected to the second communication unit 132, the second display unit 133, the image selection unit 134, the input unit 135, and the second storage unit 136. The editor terminal 120 can be a personal computer, a notebook computer, a smart phone, a tablet computer, or a wearable device, but is not limited thereto.

The first communication unit 121 is connected to the cloud server 110 via a network. The first communication unit 121 can transmit the information of the identification information 115 and the AR target file 112 to the cloud server 110. The first storage unit 122 is configured to store information of the object editing interface 127, the interactive object 126, the AR target file 112, or the identification information 115. The first display unit 123 can make a selection of the object editing interface 127, and can also edit the selected interactive object 126. The user can select the button, image object, list, or image object of the object editing interface 127 through the object operating unit 125; or move or rotate the interactive object 126 to move to any position of the object editing interface 127. The object operating unit 125 may be a keyboard, a mouse, a touch screen, but is not limited thereto.

When the editor terminal 120 is connected to the cloud server 110, the cloud server 110 performs a confirmation step of the user's identification information 115. After the cloud server 110 completes the verification of the identification information 115, the cloud server 110 transmits the corresponding index information 116 to the user according to the identification information 115. The AR target file 112 that the user can select is recorded in the index information 116. In general, the content stored in the cloud server 110 includes public index information 116 and index information 116 for personal use. After the user selects the AR target file 112, the editor terminal 120 downloads the selected AR target file 112 and displays the scene content of the selected AR target file 112 on the object editing interface 127. In general, the scene of the AR temporary file 111 may be a still image, a festive greeting card, a postcard, or a dynamic movie, but is not limited thereto.

The object editing interface 127 in this embodiment includes at least an AR target file list, an interactive object list, an editing area, a location information, an event list, and an action list. The user can select an object to be edited from the AR target file list and the interactive object list. After selection, the related scene or interactive object 126 of the AR target file 112 is downloaded and displayed in the editing area of the object editing interface 127. The user can select a model in any of the interactive object lists and edit the selected interactive object 126 in the editing area. The user can select at least one interactive object 126 from the list of interactive objects of the object editing interface 127, or can download a new interactive object 126 from an external source. For example, the user downloads any image, picture or 3D model from the hard disk or optical disc of the editor terminal 120 as a new interactive object 126.

The user can edit the scene or object in the AR target file 112 through the object editing interface 127. As shown in FIG. 2, it is a schematic diagram of an editing process of the AR target file 112 of the present invention. In the present invention, the editing process of the AR target file 112 is described in terms of vehicle travel, but is not limited thereto.

First, the user selects at least one interactive object 126 in the object editing interface 127 and moves the interactive object 126 to the scene of the AR target file 112 through the object operating unit 125. Meanwhile, when the user is editing the interactive object 126, the first processing unit 124 records the steps of editing the interactive object 126, and stores it in the AR temporary file 111, as shown in FIG. 2 and FIG.

In FIG. 2, the user drags the interactive object 126 from the left side of the object editing interface 127 into the scene. For example, when the first processing unit 124 selects or operates the interactive object 126, the first processing unit 124 begins to create a corresponding AR temporary file 111. The first processing unit 124 records the above operation process in the AR temporary storage file for the operation of setting the interactive object 126 to move, zoom, change the image, set the object action, change the display menu or add or delete the image in the scene. Here, as shown in Figure 4. The first processing unit 124 records each edited process of the interactive object 126 as a new AR temporary file 111. Further, the user may select at least one AR temporary storage file 111 from the current plurality of AR temporary storage files 111, and associate the selected AR temporary storage file 111 with the interactive object 126 to generate a new AR. Target file 112. The user may also select a plurality of AR temporary files 111 and associate the selected plurality of AR temporary files 111 with the interactive object 126 to generate an AR target file 112. In addition, the editor side 120 can also set the triggering action of the interactive object 126 when clicking or moving, such as an animation, playing a sound, playing a movie, or opening a new window screen, but is not limited thereto.

After the editing of the interactive object 126 and the AR temporary file 111 is completed, the first processing unit 124 generates the result of the editing completion as the AR target file 112. In addition, the editor side 120 can set access rights to the AR target file 112, and the access rights can be publicly used or limited to users. The setting of the rights includes the access rights of the AR target file 112 in the identification information, but is not limited thereto.

After setting the identification information 115, the editor 120 stores the interactive object 126 and the selected AR temporary file 111 as the AR target file 112, and uploads the AR target file 112 to the cloud server 110. The cloud server 110 updates the index information and records the AR target file 112. Finally, the editor terminal 120 uploads the AR target file 112 to the cloud server 110 through the first communication unit 121, and the cloud server 110 synchronously updates the user's identification information 115.

The mobile device end 130 is connected to the cloud server 110 through the second communication unit 132 for transmitting the information of the index information 116, the identification information 115 and the AR target file 112. The second display unit 133 is configured to play the captured image or AR target file 112. The image selection unit 134 is configured to capture an environment image, and the environment image includes a target 151 to be identified. The input unit 135 is for controlling the interaction of the operation of the mobile device side 130 with the AR target file 112. The input unit 135 can be implemented by a touch screen, or can be a button or a joystick.

The target to be identified 151 of the present invention may be a QR code (Quick Response, QR for short), or other graphics or physical objects preset by the cloud server 110, or graphics uploaded by the user. The second storage unit 136 is configured to store the first image recognition program 141, the second image recognition program 142, the community sharing program 143, and the simple editing program 144.

When the second processing unit 131 runs the first image recognition program 141, the image selection unit 134 starts shooting. In the process of the image capturing unit 134, the second processing unit 131 determines in real time whether the captured object 151 to be identified has the identification information 115. The identification information 115 is automatically recognized by the second processing unit 131 in the process of capturing the object 151 to be identified. When the second processing unit 131 detects that the target to be identified 151 has the identification information 115, the second processing unit 131 downloads the corresponding AR target file 112 from the cloud server 110 according to the request of the index information 116. After the mobile device end 130 downloads the AR target file 112, the second display unit 133 will play the AR target file 112.

The user can operate the interactive object 126 of the AR target file 112 through the input unit 135, taking the above-mentioned vehicle (ie, the interactive object 126) as an example. When the image selection unit 134 captures the object to be identified 151 having the identification information 115, as shown in FIG. 5, the second processing unit 131 acquires the related AR target file 112 from the cloud server 110 based on the identification information 115, and then displays the second target file 112. Unit 133 plays AR target file 112.

In addition, the present invention can also download the corresponding AR target file 112 by the second image recognition program 142. When the second processing unit 131 runs the second image recognition program 142, the image selection unit 134 starts shooting. During the shooting, the image selection unit 134 captures the target 151 to be identified. At the same time, a related icon appears in the second display unit 133, as shown in FIG. 6. After the user clicks on the icon, an input box appears. After the user inputs the correct identification code, the second processing unit 131 downloads the user information belonging to the identification code from the cloud server, and scans the corresponding identification image. To generate a corresponding AR target file 112, link the corresponding AR target file 112 to the related activity or page. The second processing unit 131 can identify the presence or absence of the corresponding interactive object 126 from the activity or page. It is also mentioned above that the identification information 115 can identify access rights to the interactive object 126. In order to achieve the non-disclosure purpose, the second image recognition program 142 further protects the access rights of the target 151 to be identified.

The operation of the community sharing program 143 is established on the first image recognition program 141 and the second image recognition program 142 described above. After the user completes the above image recognition process, the user can add the AR target file 112 to the management page. In the management page, each AR target file 112 is arranged in order, as shown in FIG. The user can select the option of any AR target file 112 and the community share button 152 via the input unit 135. When the user clicks the selected community sharing button 152, the second processing unit 131 will run the community sharing program 143 and display the corresponding community icon 153 on the second display unit 133. The user can select any community icon 153. Upon selection, the second processing unit 131 forwards the index information 116 of the AR target file 112 to the selected community.

When the second processing unit 131 executes the simple editing program 144, the user can capture an environmental image or an image object. After the shooting is completed, the second processing unit 131 uploads the captured result to the cloud server 110 in the manner of the AR temporary file 111. In other words, the user can take any object or image through the mobile device end 130 as the material required for the AR temporary file 111. In addition, the mobile device 130 can also upload the externally downloaded movie to the cloud server 110 and store the movie as a new AR temporary file 111.

Embodiments of the invention are as described above. However, the embodiments of the present invention are not limited thereto, and the present invention can also be made without departing from the above-described basic technical idea of the present invention, in accordance with the above-mentioned contents of the present invention, using the ordinary technical knowledge and conventional means in the art. Other various modifications, substitutions and alterations are possible within the scope of the invention.

Claims (10)

An editing device for online integration of augmented reality, characterized in that the device comprises: The communication unit is connected to the cloud server through the network, and is used to obtain an AR target file or an interactive object; a display unit, configured to display an environment image, an AR temporary file, the AR target file, or the interactive object; An object operating unit, configured to control a position and a display angle of the interactive object in the display unit; a processing unit, the processing unit is electrically connected to the communication unit, the display unit, and the object operation unit, and the processing unit drives the communication unit to download and identify information from the cloud server according to the identification information. Corresponding index information of the AR temporary storage file, and downloading any of the AR target files according to the index information; The processing unit selects any of the AR temporary storage files and generates the AR target file according to the interactive object and the AR temporary storage file generated in each step of the editing process of the AR target file, and the processing is performed. The unit sends the generated AR target file to the cloud server through the communication unit. The apparatus for editing an online integrated augmented reality according to claim 1, further comprising: a storage unit, wherein the storage unit is configured to store the AR temporary storage file, the interactive object, the object editing interface, or the AR target file. The online integrated augmented reality editing device according to claim 2, wherein the AR temporary storage file is used for recording movement, image scaling, image deformation, and setting of the interactive object in the object editing interface. Object action, change display options, or add or delete images. The online integrated augmented reality editing apparatus according to claim 3, wherein the processing unit runs the object editing interface, and the object editing unit controls the object editing interface for editing the same. The position, play position, or play time of the AR temporary file in the environment image. The online integrated augmented reality editing device according to claim 1, wherein the editing device is a personal computer, a notebook computer, a smart phone, a tablet computer or a wearable device. The online integrated augmented reality editing apparatus according to claim 1, wherein said processing unit adds said identification information to said generated AR target file and transmits it to said cloud server The cloud server classifies the received AR target file as a new AR temporary file. An editing system for online integration of augmented reality, characterized in that the system comprises: a cloud server for recording a plurality of AR target files, a plurality of interactive objects or a user database; the user database records each user's identification information; At least one editor end connected to the cloud server network; the editor end includes a first communication unit, a first storage unit, a first display unit, a first processing unit and an object operating unit, the first processing The unit is electrically connected to the first communication unit, the first storage unit, the first display unit, and the object operation unit, and the first communication unit acquires an AR corresponding to the identification information to the cloud server. Index information of the target file; the first storage unit is configured to store an object editing interface, at least one AR temporary file, the interactive object or the AR target file; and the first display unit is configured to edit the object editing The interface and the selected interactive object; the object operating unit is configured to control a placement position and a display angle of the interactive object in the first display unit, and generate an AR temporary storage corresponding to the interactive object in the editing process a file; the first processing unit generates the AR target file according to the interactive object and the selected AR temporary file, and the first processing unit passes the a communication unit sends the generated AR target file to the cloud server; a mobile device end, the mobile device end includes a second processing unit, a second communication unit, a second display unit, an image selection unit and a second storage unit, the second processing unit and the second communication unit, The second display unit is electrically connected to the second storage unit, the second communication unit is connected to the cloud server network, and the image selection unit is configured to capture a target to be identified, and the second The processing unit sends the to-be-identified target to the cloud server, and the cloud server identifies whether the target to be identified is one of the plurality of interactive objects, and the cloud server is to be identified The AR target file corresponding to the target is sent to the mobile device end, and the received AR target file is played by the second display unit. The online integrated augmented reality editing system according to claim 7, wherein said first processing unit adds said identification information to said generated AR target file and transmits it to said cloud server. The online integrated augmented reality editing system according to claim 7, wherein the mobile device end inputs an identification code, and the second processing unit identifies the to-be-identified target according to the identification code. And downloading, according to the requirement of the target to be identified, the AR target file corresponding to the target to be identified from the cloud server. The online integrated augmented reality editing system according to claim 7, wherein the mobile device runs a community sharing program, and the second communication unit is connected to the cloud server to select a community. After the website, the second processing unit generates an invitation notification and sends it to the selected social networking website.

Images