KR102174039B1 - Apparatus and Method for Editing Files Anywhere Using Augmented Reality - Google Patents

Abstract

Disclosed is an apparatus and method capable of editing files anywhere using augmented reality.
According to an aspect of the present embodiment, in a file editing apparatus that enables a user to output a file and edit a file anywhere using augmented reality, the file to be edited is received from the outside or the edited file is transmitted to the outside. The communication unit and the image output unit for outputting an augmented reality image of the file received by the communication unit, a camera for photographing a part of the user's body and the surrounding environment of the file editing device, and the surrounding environment of the file editing device photographed by the camera. By mapping a file to generate an augmented reality image of the file, controlling the image output unit to output an augmented reality image of the generated file, and analyzing the motion of a part of the user's body photographed by the camera, the body part of the user It provides a file editing apparatus comprising a control unit for editing the file according to the operation of.

Description

Apparatus and Method for Editing Files Anywhere Using Augmented Reality}

The present invention relates to an apparatus and method for allowing a user to edit a file anywhere by using augmented reality technology.

The content described in this section merely provides background information on the present embodiment and does not constitute the prior art.

With the recent development of communication technology, various services are provided using the Internet. In particular, large files are moved and large amounts of data are transmitted and received using the Internet. Accordingly, technology for sharing various files anytime, anywhere has also developed.

In this way, based on various communication technologies (e.g., the Internet), users who use terminals store data or files in a virtual storage, and provide a service that allows users to exchange and share data or files through the virtual storage. It is called a cloud service. Users can freely use data or files using cloud services. Specifically, a user can download or upload a file without being limited by time and space using a cloud service, and can share the file with other users.

These cloud services are provided based on a shared server, which is a storage for storing and sharing data or files. Specifically, users store and share files using a shared server, and users can edit files with their own terminals or other terminals.

At this time, like a document file, it is a simple file format that can be easily processed even with a mobile terminal. If the main terminal, where the user usually edits files, is a mobile terminal, the user You can edit files anywhere using the terminal.

However, if the file is a format that is mainly edited on a terminal that is difficult to move, such as a video file, or if the main terminal used for editing a file is a terminal that is difficult to move, the user may save and share the file using a cloud service. It is inconvenient to edit the file only in the space where the main terminal is located. For example, if the main terminal for editing a file is a desktop PC, no matter how much the file can be shared with multiple terminals as a cloud service, the user can only use other terminals of the same type as the main terminal. There is a problem that the main terminal cannot be used in the other spaces installed.

Republic of Korea Patent Publication No. 2019-0132191 (title of the invention: augmented reality editing system and augmented reality implementation system capable of producing and editing augmented reality contents)

An object of the present invention is to provide a file editing apparatus and method capable of editing a file in the same environment as the environment in which the main terminal and the main terminal are arranged anytime, anywhere using augmented reality.

According to an aspect of the present invention, in a file editing apparatus that enables a user to output a file and edit a file anywhere using augmented reality, the file to be edited is received from the outside or the edited file is transmitted to the outside. The communication unit and the image output unit for outputting an augmented reality image of the file received by the communication unit, a camera for photographing a part of the user's body and the surrounding environment of the file editing device, and the surrounding environment of the file editing device photographed by the camera. By mapping a file to generate an augmented reality image of the file, controlling the image output unit to output an augmented reality image of the generated file, and analyzing the motion of a part of the user's body photographed by the camera, the body part of the user It provides a file editing apparatus comprising a control unit for editing the file according to the operation of.

According to an aspect of the present invention, the file editing device is characterized in that the smart glass.

According to an aspect of the present invention, the communication unit may receive the file to be edited from a cloud server or transmit the edited file to the cloud server.

According to an aspect of the present invention, the camera is characterized in that it photographs a hand of the user's body.

According to an aspect of the present invention, the control unit may analyze a user's hand motion and edit the file according to the user's hand motion.

According to an aspect of the present invention, in a method of allowing a user to output a file and edit a file from anywhere using augmented reality by a file editing device, the receiving process of receiving a file to be edited from the outside and the user's body A photographing process of photographing a part and the surrounding environment of the file editing device, and a generation process of generating an augmented reality image for the file by mapping the file to the surrounding environment of the file editing device photographed in the photographing process and augmentation of the file An editing process of outputting a real image and an editing process of editing the file according to the motion of the user's body part by analyzing a motion of a user's body part and a transmission process of transmitting the edited file to the outside. Provides a method of editing files.

According to an aspect of the present invention, the file editing device is characterized in that the smart glass.

According to an aspect of the present invention, the receiving process and the transmitting process are characterized by receiving the file to be edited from a cloud server or transmitting the edited file to the cloud server, respectively.

According to an aspect of the present invention, the photographing process is characterized by photographing a hand of the user's body.

According to an aspect of the present invention, in the editing process, the user's hand motion photographed during the photographing process is analyzed and the file is edited according to the user's hand motion.

According to an aspect of the present invention, in a file editing system that enables a user to output a file and edit a file from anywhere using augmented reality, a terminal for editing and uploading a file in a real space and uploading from the terminal A cloud server that receives and stores a file and a file stored in the cloud server is downloaded, and the file is mapped to the surrounding environment of the file editing device by photographing a part of the user's body and the surrounding environment of the file editing device. Generates an augmented reality image for, outputs the generated augmented reality image, analyzes the motion of the user's body part, edits the file according to the motion of the user's body part, and sends the edited file to the cloud server. It provides a file editing system comprising a file editing device for uploading.

According to an aspect of the present invention, the cloud server is characterized in that it updates a pre-stored file using the edited file uploaded to the cloud server.

According to an aspect of the present invention, in a file editing method in which a file editing system including a terminal, a cloud server, and a file editing device uses augmented reality to allow a user to output a file and edit a file anywhere, the terminal A first upload process in which a file is edited and uploaded in this real space, a storage process in which the cloud server receives and stores a file uploaded from the terminal, and a download process in which the file editing device downloads a file to be edited from the cloud server And a photographing process in which the file editing device photographs a part of the user's body and the surrounding environment of the file editing device, and the file editing device maps the file to the surrounding environment of the file editing device photographed in the photographing process. The generation process of generating an augmented reality image for the file, the output process of the file editing device outputting the augmented reality image of the file, and the file editing device analyzes the motion of the user's body part, Accordingly, it provides a file editing method comprising an editing process of editing the file and a second uploading process of uploading the edited file to the cloud server by the file editing device.

According to an aspect of the present invention, the cloud server is characterized in that it updates the file stored in the storage process by using the edited file uploaded to the cloud server.

As described above, according to one aspect of the present invention, by using augmented reality to edit a file in the same environment as the environment in which the main terminal and the main terminal are located, anytime, anywhere, the file can be saved in a familiar environment. There is one advantage to being able to edit it.

1 is a diagram illustrating a configuration of a file editing system according to an embodiment of the present invention.
2 is a diagram illustrating a configuration of a file editing apparatus according to an embodiment of the present invention.
3 is a diagram showing a state of dividing the surrounding environment of the file editing apparatus according to an embodiment of the present invention by space.
4 is a diagram illustrating a state of determining a degree of matching between a surrounding environment and an arrangement environment of a terminal according to an embodiment of the present invention.
5 is a diagram illustrating an environment in which an augmented reality image is output and a surrounding environment of a file editing apparatus according to an embodiment of the present invention.
6 is a timing chart illustrating a method of editing a file by a file editing system according to an embodiment of the present invention.
7 is a flowchart illustrating a method of editing a file by a file editing apparatus according to an exemplary embodiment.

In the present invention, various changes may be made and various embodiments may be provided, and specific embodiments will be illustrated in the drawings and described in detail. However, this is not intended to limit the present invention to a specific embodiment, it is to be understood to include all changes, equivalents, or substitutes included in the spirit and scope of the present invention. In describing each drawing, similar reference numerals have been used for similar elements.

Terms such as first, second, A, and B may be used to describe various elements, but the elements should not be limited by the terms. These terms are used only for the purpose of distinguishing one component from another component. For example, without departing from the scope of the present invention, a first element may be referred to as a second element, and similarly, a second element may be referred to as a first element. The term and/or includes a combination of a plurality of related listed items or any of a plurality of related listed items.

When a component is referred to as being "connected" or "connected" to another component, it is understood that it may be directly connected or connected to the other component, but other components may exist in the middle. Should be. On the other hand, when a component is referred to as being "directly connected" or "directly connected" to another component, it should be understood that there is no other component in the middle.

The terms used in the present application are used only to describe specific embodiments, and are not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In the present application, terms such as "include" or "have" should be understood as not precluding the possibility of existence or addition of features, numbers, steps, actions, components, parts, or combinations thereof described in the specification. .

Unless otherwise defined, all terms, including technical or scientific terms, used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present invention belongs.

Terms as defined in a commonly used dictionary should be interpreted as having a meaning consistent with the meaning in the context of the related technology, and should not be interpreted as an ideal or excessively formal meaning unless explicitly defined in this application. Does not.

In addition, each configuration, process, process, or method included in each embodiment of the present invention may be shared within a range not technically contradicting each other.

1 is a diagram illustrating a configuration of a file editing system according to an embodiment of the present invention.

Referring to FIG. 1, a file editing system 100 according to an embodiment of the present invention includes a main terminal 110, a cloud server 120, and a file editing device 130.

The main terminal 110 is a device that allows a user to edit a file, and is disposed in an optimal place for a user to edit a file.

The main terminal 110 allows a user to edit a file. A file is a generic term for information including content that a user can execute and edit using a main terminal, such as a document file, video file, audio file, or software. The main terminal 110 executes the file, provides information contained in the file to the user, and modifies the information in the file by receiving the user's input. The main terminal 110 is mainly arranged in one place, such as a desktop PC, a software player such as a video game machine, or a music editing device, so that it is difficult to move, and may be implemented as a device having considerable performance in editing files. The main terminal 110 uploads the file to be edited by the user or the edited file to the cloud server 120.

The main terminal 110 is arranged in an optimal place for a user to edit a file. As described above, the main terminal 110 is a device having a considerable performance in editing a file, it is not easy to move and is mainly used by being placed in one place. Accordingly, the main terminal 110 is used by being placed in a familiar or optimal place for a user to edit a file.

The cloud server 120 stores files uploaded from some or all of the main terminal 110 and the file editing device 130, and stores files stored in some or all of the main terminal 110 and the file editing device 130. send.

The cloud server 120 transfers a file from the main terminal 110, a file editing device 130 or a separate device to 3D shape information about the arrangement environment of the main terminal 110 (hereinafter as '3D shape information'). Abbreviated). The main terminal 110 uploads a file that the user intends to edit, is editing, or has completed editing to the cloud server 120, and the cloud server 120 receives and stores the file from the main terminal 110. In addition, a separate device capable of photographing the surrounding environment using the file editing device 130 or a camera and converting it to 3D shape information generates 3D shape information and uploads it to the cloud server 120, and 120) stores the 3D shape information received from the file editing device 130 or a separate device.

The cloud server 120 transmits the file and 3D shape information to the file editing device 130 and receives the edited file from the file editing device 130. In response to the request of the file editing device 130, the cloud server 120 transmits the file and 3D shape information stored in the cloud server 120 to the file editing device 130. At this time, the cloud server 120 uses the identifier to check whether the user of the file editing device 130 is the same person or a related person as the user of the main terminal 110, and only if the user is the same person, the files stored in the cloud server 120 and 3D The shape information may be transmitted to the file editing device 130. The cloud server 120 receives and stores the file edited by the file editing device 130 from the file editing device 130, and when there is a request from the main terminal 110, the file edited by the main terminal 110 To transmit.

The file editing device 130 generates 3D shape information and uploads it to the cloud server 120, and implements the file and 3D shape information received from the cloud server 120 using augmented reality technology, so that the user can Regardless of location and location, files can be edited in an environment similar to that of the main terminal. The file editing apparatus 130 edits the file by recognizing the user's motion with respect to the file implemented by the augmented reality technology and correcting the contents of the file appropriately to the user's motion.

The file editing device 130 generates 3D shape information and uploads it to the cloud server 120. The file editing device 130 photographs the configuration and shape of the main terminal 110 and the arrangement environment in which the main terminal 110 is arranged, generates it as 3D shape information, and uploads it to the cloud server 120. The file editing device 130 may generate and store 3D shape information, but by uploading it to the cloud server 120, any kind of file editing device can download the 3D shape information. The file editing device 130 is connected to the cloud server 120 not only in the file editing device 130 that generated the 3D shape information, but also in other file editing devices of the same or different types as the file editing device 103 to provide a 3D shape. Make information accessible. Accordingly, the file uploaded to the cloud server 120 and the 3D shape information can be used not only in the file editing device 130 possessed by the user, but also in other devices of the same or different types of the file editing device 130. Because of this, the user can escape from the constraints on the file editing device 130. For example, in a situation where the user leaves the file editing device 130 in a certain place and leaves to another place, and there is another device with a file editing function in another place, the user edits the file using the device. can do.

The file editing device 130 downloads the file and 3D shape information stored in the cloud server 120 by the user using the main terminal 110 from the cloud server 120. The file editing device 130 can access the cloud server 120 and authenticate that the user who has uploaded the file stored in the cloud server 120 and the 3D shape information using the main terminal 110 is the same user, and authenticates After that, the file stored in the cloud server 120 and 3D shape information are downloaded.

The file editing device 130 converts the file and 3D shape information into an augmented reality image using augmented reality technology. The file editing device 130 is an augmented reality device capable of generating and implementing an augmented reality image, and may be implemented with, for example, smart glasses. The file editing device 130 recognizes the surrounding environment of the file editing device 130 and searches for a surrounding environment similar to 3D shape information among surrounding environments. The file editing device 130 implements the file and 3D shape information as an augmented reality image using the searched surrounding environment. As the file editing device 130 implements and outputs augmented reality images for files and 3D shape information in an environment similar to 3D shape information, the user is in an environment similar to the arrangement environment in which the actual main terminal 110 is placed. You can check the augmented reality image as if the file is output to the main terminal 110 arranged in.

The file editing apparatus 130 recognizes input information from a user's motion and edits the file according to the input information. The file editing device 130 may check the user's motion by photographing a part of the user's body, and analyze the user's motion to determine which input the user desires. For example, when the file is a document, the file editing apparatus 130 may check input information that the user wants to edit for the document by checking the operation of the user, in particular, the user's finger. The file editing apparatus 130 checks the user's input information and edits the file according to the input information. The file editing device 130 uploads the edited file to the cloud server 120 so that other devices such as the main terminal 110 can subsequently edit or check the file. A detailed description of the file editing apparatus 130 will be described with reference to FIG. 2.

2 is a diagram illustrating a configuration of a file editing apparatus according to an embodiment of the present invention.

2, the file editing apparatus 130 according to an embodiment of the present invention includes a communication unit 210, a 3D modeling unit 220, a camera 230, a control unit 240, and an image output unit 250. Include.

The communication unit 210 uploads the 3D shape information or the edited file to the cloud server 120, or downloads the file and the 3D shape information from the cloud server 120.

The 3D modeling unit 220 models the arrangement environment of the main terminal 110 and the main terminal 110 photographed by the camera 230 as 3D shape information. The 3D modeling unit 220 includes the main terminal 110 and the main terminal 110 so that the arrangement environment of the main terminal 110 and the main terminal 110 can be output as an augmented reality image in the file editing device 130 or other file editing device. The arrangement environment of the terminal 110 is modeled as 3D shape information. For example, the 3D modeling unit 220 may generate 3D shape information in the following manner. First, the 3D modeling unit 220 extracts feature points from an image captured by the camera 230. The 3D modeling unit 220 matches feature points between neighboring images (previous image and current image). The 3D modeling unit 220 estimates the three-dimensional pose of the camera by using the two-dimensional coordinate relationship in the image between the matched feature points. The 3D modeling unit 220 models 3D shape information by back-projecting a 2D feature point into a 3D space based on the estimated 3D pose of the camera. The 3D modeling unit 220 models realistic 3D shape information by reflecting the color distribution to 2D feature points by performing reverse projection. The 3D modeling unit 220 models the arrangement environment of the main terminal 110 and the main terminal 110 photographed by the camera 230 using various methods including the above-described method as 3D shape information.

When the 3D modeling unit 220 generates an augmented reality image from a file and 3D shape information, the file may be included in the shape information of the main terminal 110 included in the 3D shape information. For example, when the file is a document and the main terminal 110 is a desktop PC, the 3D modeling unit 220 generates an augmented reality image as if the document is output on the monitor of the main terminal 110. Accordingly, the 3D modeling unit 220 may provide a more realistic augmented reality image to the user.

The camera 230 photographs the arrangement environment of the main terminal 110 and the main terminal 110, or the surrounding environment of the file editing apparatus 130 in which an augmented reality image may be output and a part of the user's body.

The camera 230 photographs the arrangement environment of the main terminal 110 and the main terminal 110. In order for the 3D modeling unit 220 to model the arrangement environment of the main terminal 110 and the main terminal 110 with 3D shape information, the camera 2300 is the arrangement environment of the main terminal 110 and the main terminal 110 Take a picture.

The camera 230 photographs the surrounding environment of the file editing device 130 so that the control unit 240 can determine the place where the augmented reality image is to be output. When a user edits a file using the file editing device 130, the control unit 240 includes 3D shape information and a file editing device so that the file can be edited in an environment similar to the arrangement environment of the main terminal 110 as much as possible. (130) judge the degree of consistency of the surrounding environment. The camera 230 photographs the surrounding environment of the file editing device 130 and provides it to the control unit 240 so that the control unit 240 can determine the degree of matching with the 3D shape information.

The camera 230 photographs the surrounding environment of the file editing apparatus 130 and simultaneously photographs a part of the user's body to grasp the user's input information. The camera 230 photographs a part of the user's body. As in the above example, when the file is a document, the camera 230 photographs the user's finger, so that the controller 240 can recognize the user's input information by recognizing the motion of the user's finger.

The controller 240 downloads a file and 3D shape information from the cloud server 120 to generate and output an augmented reality image, and controls each component in order to recognize the user's input information and edit the file.

The controller 240 controls the 3D modeling unit 220 to generate an augmented reality image using the file and 3D shape information downloaded from the cloud server 120 by the communication unit 210. In this case, the control unit 240 controls the file to be included in the shape information of the main terminal 110 included in the 3D shape information in order to generate an augmented reality image from the file and the 3D shape information. For example, when the file is a document and the main terminal 110 is a desktop PC, the control unit 240 generates a 3D modeling unit 220 to generate an augmented reality image as if the document is output on the monitor of the main terminal 110. Control. The controller 240 generates an augmented reality image by mapping the file and 3D shape information to the image of the surrounding environment of the file editing device 120 photographed by the camera 230.

The controller 240 determines the degree of matching between the surrounding environment of the file editing device 130 photographed by the camera 230 and the 3D shape information, and searches for a place to output an augmented reality image. A method in which the control unit 240 analyzes the surrounding environment of the file editing apparatus 130 and determines the degree of conformity with the 3D shape information is illustrated in FIGS. 3 and 4.

3 is a diagram showing a state of dividing the surrounding environment of the file editing apparatus according to an embodiment of the present invention by space, and FIG. 4 is a view showing the matching of the surrounding environment and the arrangement environment of a terminal according to an embodiment of the present invention. It is a diagram showing a state of judging degree.

Referring to FIG. 3, the controller 240 divides the image of the surrounding environment of the file editing device 130 photographed by the camera 230 for each space. The control unit 240 provides three-dimensional information about the surrounding environment of the file editing device 130 photographed by the camera 230 in the same or different method as the 3D modeling unit 220, or by any method (hereinafter, ' (Abbreviated as '3D information') is created. The controller 240 divides the 3D information into spaces according to similarities such as inclination or area, and divides each of them.

Thereafter, as shown in FIG. 4, the control unit 240 compares the spaces of the divided 3D information with the arrangement environment of the main terminal in the 3D shape information to determine the consistency of each information. Fig. 4(a) shows a process of determining the consistency of an arrangement environment of a main terminal with any one of the divided 3D information spaces. Referring to Fig. 4(a), it can be seen that the two are significantly different in consistency. On the other hand, referring to FIG. 4(b), it can be seen that the consistency of the arrangement environment of the main terminal with any one of the divided 3D information spaces is quite high. The control unit 240 determines the consistency as described above and searches a space of 3D information having high consistency with the arrangement environment of the main terminal. Thereafter, the control unit 240 controls the image output unit 250 to output an augmented reality image to the searched coherent space, thereby increasing the user's sense of immersion.

Referring back to FIG. 2, the controller 240 controls the image output unit 250 to generate an augmented reality image and output an augmented reality image to the surrounding environment space of the highly consistent file editing device 130. For example, assuming that the file editing device 130 is a smart glass, the controller 240 may operate as follows. When the file editing device 130 is not facing the direction of the highly consistent space, the control unit 240 outputs the direction to the image output unit 250 so that the file editing device 130 determines the direction of the highly consistent space. Can be controlled to face. When the file editing device 130 is facing a direction of a highly consistent space, the controller 240 maps the file and 3D shape information to the image of the surrounding environment of the file editing device 120 photographed by the camera 230 to augment it. The image output unit 250 is controlled to generate a reality image and output the generated augmented reality image. Accordingly, the user of the file editing device 130 receives an augmented reality image for 3D shape information in a highly consistent space, and uses the main terminal 110 in the space where the actual main terminal 110 is placed. You can work like editing An example of this is shown in FIG. 5.

5 is a diagram illustrating an environment in which an augmented reality image is output and a surrounding environment of a file editing apparatus according to an embodiment of the present invention.

FIG. 5(a) is a diagram showing the surrounding environment space of the highly consistent file editing apparatus 130 searched by the controller 240. As shown in Fig. 5(a), substantially no configuration related to the main terminal 110 is arranged, but only a space exists.

5(b) is a diagram in which an augmented reality image is output on a corresponding space by the image output unit 250. As an augmented reality image is output on a space where the configuration related to the main terminal 110 does not exist, the user can feel as if editing a file using his or her main terminal 110, so the user's work efficiency It has the advantage of improving the convenience and convenience, and even if the file is a format that is mainly edited on a terminal that is difficult to move, or the main terminal used by the user for editing a file is a terminal that is difficult to move, the user can be used regardless of time and location. There is an advantage of being able to provide an impression such as editing a file with the main terminal 110.

Referring back to FIG. 2, the controller 240 analyzes an image or an image of a part of the user's body photographed by the camera 230 and checks the user's input information corresponding to the user's motion. The controller 240 analyzes an image or an image of a part of the user's body, and analyzes movement of a feature point with respect to a part of the user's body. As in the above example, when the document is a file, the controller 240 may control the camera 230 to capture a user's finger, and analyze the motion of each joint, which is a feature point within the user's finger. The controller 240 analyzes the motion of the feature points (each joint) to analyze how a part of the user's body is moving, and checks input information corresponding to the user's motion. In the above-described example, the controller 240 can determine what input information corresponding to the user's motion is by grasping to which position the motion of the user's finger moves to the keyboard of the main terminal 110. The control unit 240 edits the file based on the checked user's input information. In this way, the file editing apparatus 130 replaces the input means of the main terminal 110 with the user's motion recognition, and replaces the output means of the main terminal 110 with the image output unit 250, so that the time and place You can allow users to view and edit files without being constrained.

The control unit 240 controls the communication unit 210 to transmit the edited file to the cloud server 120. The file editing device 130 uploads the edited file to the cloud server 120 so that the file edited by the file editing device 130 can be shared with the main terminal 110 as well.

In the above description, it is described that the file editing apparatus 130 photographs the arrangement environment of the main terminal 110 to generate 3D shape information, but is not limited thereto. Generation of 3D shape information and uploading to the cloud server 120 can be operated in devices other than the file editing device 120, and the file editing device 130 downloads the uploaded file and 3D shape information and edits the file. Can be used for

6 is a timing chart illustrating a method of editing a file by a file editing system according to an embodiment of the present invention.

The terminal 110 uploads the file (S610). The main terminal 110 uploads the file to be edited by the user or the edited file to the cloud server 120.

The cloud server 120 stores the uploaded file (S615).

The file editing apparatus 130 acquires 3D shape information by photographing the configuration of the terminal and the arrangement environment of the terminal (S620).

The file editing device 130 uploads the acquired 3D shape information to the cloud server 120 (S625). The file editing device 130 uploads the acquired 3D shape information to the cloud server 120, so that any device that can edit files like the file editing device 130 as well as the file editing device 130 can The file stored in the server 120 and the 3D shape information may be converted into an augmented reality image to edit the file. Accordingly, the user can edit the file regardless of the location or type of the file editing device 130.

The file editing device 130 acquires 3D information by photographing the surrounding environment of the file editing device 130 (S630).

The file editing apparatus 130 searches for a surrounding environment having the highest degree of matching with 3D shape information for the configuration and arrangement environment of the terminal (S635). The controller 240 divides the image of the surrounding environment of the file editing device 130 by space, and searches for the surrounding environment having a high degree of matching with the arrangement environment of the main terminal 110.

The file editing device 130 downloads the file and 3D shape information from the cloud server 120 (S640).

The file editing apparatus 130 photographs the surrounding environment, and generates an augmented reality image of the file and 3D shape information on the configuration and arrangement environment of the main terminal (S645). The control unit 240 generates an augmented reality image by mapping the 3D shape information to the surrounding environment with high alignment with the arrangement environment of the main terminal 110.

The file editing device 130 outputs the generated augmented reality image (S650).

The file editing apparatus 130 captures a part of the user's body and recognizes input information corresponding to the motion of the part of the body (S655).

The file editing device 130 edits the file based on the input information (S660).

The file editing device 130 uploads the edited file to the cloud server 120 (S665).

The cloud server 120 updates the stored file (S670).

The main terminal 110 may download the updated file from the cloud server 120 (S675).

7 is a flowchart illustrating a method of editing a file by a file editing apparatus according to an exemplary embodiment. Since the method of editing a file by the file editing apparatus has been described in detail with reference to FIGS. 1 to 6, detailed descriptions will be omitted.

The file editing apparatus 130 acquires 3D shape information by photographing the configuration of the main terminal and the arrangement environment of the main terminal (S710).

The file editing device 130 uploads the 3D shape information to the cloud server (S715).

The file editing device 130 acquires 3D information by photographing the surrounding environment of the file editing device 130 (S720).

The file editing apparatus 130 searches for the 3D shape information for the configuration and arrangement environment of the main terminal and the surrounding environment with the highest degree of matching (S725).

The file editing device 130 receives the file and 3D shape information from the cloud server (S730).

The file editing apparatus 130 photographs the surrounding environment and generates an augmented reality image of 3D shape information on the file, the configuration and arrangement environment of the main terminal (S735).

The file editing device 130 outputs the generated augmented reality image (S740).

The file editing apparatus 130 captures a part of the user's body and identifies input information corresponding to the motion of the part of the body (S745).

The file editing device 130 edits the file based on the input information (S750).

The file editing device 130 uploads the edited file to the cloud server (S755).

In FIGS. 6 and 7 it is described that each process is sequentially executed, but this is merely illustrative of the technical idea of an embodiment of the present invention. In other words, a person of ordinary skill in the technical field to which an embodiment of the present invention belongs can change the order shown in FIGS. 6 and 7 and execute one or more of each process without departing from the essential characteristics of an embodiment of the present invention. Since the process is executed in parallel and can be applied by various modifications and modifications, FIGS. 6 and 7 are not limited to a time series order.

Meanwhile, the processes illustrated in FIGS. 6 and 7 can be implemented as computer-readable codes on a computer-readable recording medium. The computer-readable recording medium includes all types of recording devices that store data that can be read by a computer system. That is, the computer-readable recording medium is a magnetic storage medium (e.g., ROM, floppy disk, hard disk, etc.), optical reading medium (e.g., CD-ROM, DVD, etc.), and carrier wave (e.g., Internet And storage media such as transmission through In addition, the computer-readable recording medium can be distributed over a computer system connected through a network to store and execute computer-readable codes in a distributed manner.

The above description is merely illustrative of the technical idea of the present embodiment, and those of ordinary skill in the technical field to which the present embodiment belongs will be able to make various modifications and variations without departing from the essential characteristics of the present embodiment. Accordingly, the present embodiments are not intended to limit the technical idea of the present embodiment, but to explain the technical idea, and the scope of the technical idea of the present embodiment is not limited by these embodiments. The scope of protection of this embodiment should be interpreted by the following claims, and all technical ideas within the scope equivalent thereto should be construed as being included in the scope of the present embodiment.

100: file editing system
110: main terminal
120: cloud server
130: file editing device
210: communication department
220: 3D modeling unit
230: camera
240: control unit
250: image output unit

Claims (14)

In a file editing device that enables a user to output a file and edit a file anywhere using augmented reality,
A communication unit that receives a file to be edited from outside or transmits the edited file to the outside;
An image output unit for outputting an augmented reality image of the file received by the communication unit;
A terminal for capturing a part of a user's body and an environment surrounding the file editing device, and for editing and uploading a file in a real space, and a camera for capturing an arrangement environment of the terminal; And
Mapping the file to the surrounding environment of the file editing device photographed by the camera to generate an augmented reality image of the file, controlling the image output unit to output an augmented reality image of the generated file, and the user photographed by the camera And a control unit that analyzes the motion of a part of the body of the body and edits the file according to the motion of the body part of the user,
The control unit,
After modeling with 3D shape information about the terminal and the arrangement environment of the terminal, generating 3D information about the surrounding environment of the file editing device, dividing the 3D information by space, and dividing the 3D information A file editing apparatus, comprising: comparing spaces of dimensional information with an arrangement environment of a terminal in the 3D shape information to determine the consistency of each space, and outputting the augmented reality image in a space having the highest consistency. The method of claim 1,
The file editing device,
File editing device, characterized in that the smart glass. The method of claim 1,
The communication unit,
A file editing apparatus, characterized in that receiving the file to be edited from a cloud server or transmitting the edited file to the cloud server. The method of claim 1,
The camera,
File editing apparatus, characterized in that for photographing a hand of the user's body. The method of claim 4,
The control unit,
A file editing apparatus comprising analyzing a user's hand motion and editing the file according to the user's hand motion. In a method for a file editing device to output a file and edit a file from anywhere by using augmented reality,
A receiving process of receiving a file to be edited from outside;
A photographing process of photographing a user's body part and the surrounding environment of the file editing device, a terminal editing and uploading a file in a real space, and an arrangement environment of the terminal;
A generating process of generating an augmented reality image for the file by mapping the file to the surrounding environment of the file editing device photographed in the photographing process;
An output process of outputting an augmented reality image of the file;
An editing process of analyzing a motion of a user's body part and editing the file according to the motion of the user's body part; And
Including the transfer process of transferring the edited file to the outside,
The generation process,
After modeling with 3D shape information about the terminal and the arrangement environment of the terminal, generating 3D information about the surrounding environment of the file editing device, dividing the 3D information by space, and dividing the 3D information The method further comprises a control process of comparing the spaces of the dimensional information with the arrangement environment of the terminal in the 3D shape information to determine the consistency of each space, and outputting the augmented reality image to a space having the highest consistency. How to edit files The method of claim 6,
The file editing device,
File editing method, characterized in that the smart glass. The method of claim 6,
The receiving process and the transmission process,
Each file editing method comprising receiving the file to be edited from a cloud server or transmitting the edited file to the cloud server. The method of claim 6,
The above shooting process,
A file editing method comprising photographing a hand of a user's body. The method of claim 9,
The above editing process,
Analyzing the user's hand motion photographed in the photographing process, and editing the file according to the user's hand motion. In a file editing system that allows a user to output and edit files anywhere using augmented reality,
A terminal for editing and uploading files in a real space;
A cloud server for receiving and storing a file uploaded from the terminal; And
Downloading the file stored in the cloud server, mapping the file to the surrounding environment of the file editing device photographed by photographing a part of the user's body and the surrounding environment of the file editing device to generate an augmented reality image for the file, Comprising a file editing device for outputting the generated augmented reality image, analyzing a motion of a user's body part, editing the file according to the motion of the user's body part, and uploading the edited file to the cloud server,
The file editing device,
The terminal and the arrangement environment of the terminal are photographed, modeled as 3D shape information on the photographed terminal and the arrangement environment of the terminal, and after generating 3D information about the surrounding environment of the file editing device, the 3D Information is divided and divided by space, and the consistency of each space is determined by comparing the spaces of the divided 3D information with the arrangement environment of the terminal in the 3D shape information, and the file is mapped to the space with the highest consistency. The file editing system, characterized in that generating an augmented reality image for the file. The method of claim 11,
The cloud server,
File editing system, characterized in that for updating a previously stored file using the edited file uploaded to the cloud server. In a file editing method in which a file editing system including a terminal, a cloud server and a file editing device uses augmented reality to allow a user to output and edit a file from anywhere,
A first upload process in which the terminal edits and uploads a file in a real space;
A storage process in which the cloud server receives and stores a file uploaded from the terminal;
A downloading process in which the file editing device downloads a file to be edited from the cloud server;
A photographing process in which the file editing apparatus photographs a part of the user's body and the surrounding environment of the file editing apparatus, and photographs the terminal and the arrangement environment of the terminal;
A generating process in which the file editing device maps the file to the surrounding environment of the file editing device photographed in the photographing process to generate an augmented reality image for the file;
An output process of the file editing device outputting an augmented reality image of the file;
An editing process in which the file editing apparatus analyzes a motion of a user's body part and edits the file according to the motion of the user's body part; And
Including a second upload process in which the file editing device uploads the edited file to the cloud server,
The generation process,
After modeling with 3D shape information about the terminal and the arrangement environment of the terminal, generating 3D information about the surrounding environment of the file editing device, dividing the 3D information by space, and dividing the 3D information The method further comprises a control process of comparing the spaces of the dimensional information with the arrangement environment of the terminal in the 3D shape information to determine the consistency of each space, and outputting the augmented reality image to a space having the highest consistency. How to edit files The method of claim 13,
The cloud server,
And updating the file stored in the storage process by using the edited file uploaded to the cloud server.

Images