KR20140079936A - Method and apparatus for transmitting and receiving augmented reality service - Google Patents

Abstract

The present invention relates to a method and an apparatus for transmitting and receiving an augmented reality service. The present invention organically integrates the augmented information contents that exist independently of each other, and selectively provides the augmented information to the user based on at least one of the user context and the context information.

Description

TECHNICAL FIELD [0001] The present invention relates to a method and apparatus for transmitting and receiving augmented reality service,

The present invention relates to a service transmission and reception method and apparatus, and more particularly, to a method and apparatus for transmitting and receiving an augmented reality service.

Augmented Reality (AR), which is the latest technology trend in various fields such as broadcasting, education, medical, manufacturing, publishing, and games, is a technology derived from virtual reality and combines real world and virtual world. . It is also referred to as a mixed reality because it combines the virtual worlds with additional information in real time in real time into a single image. While the virtual reality realizes the interaction between the human and the computer by establishing the real world environment as a virtual world and immersing it into the computer environment, the augmented reality projects the additional information generated using the computer model to the real world information and interacts It is a technology that enables users to experience various digital information about the real world more realistically.

The earliest augmented reality technology focused on the representation of 3D virtual objects as a part of virtual reality and the natural synthesis of real-time video. Recently, however, the spread of mobile terminals and smart phones with various sensors such as cameras and GPS has spread, and various functions that are fused with Internet services using fast mobile internet have been introduced. In addition to the existing unilateral and uniform service, Personalized augmented information providing service that provides a personalized interface and media contents by selecting a service that the user is interested in utilizing the context information of the user and the environment of the user. Also, in the method of providing the augmented information, much attention has been paid to the development of the augmented information common platform or the augmented information browser which can reproduce various augmented information contents from the method of providing the augmented information by the existing individual application program. This is similar to the relationship between the current web browser and web documents. In other words, instead of installing a separate application program for viewing only individual web documents, it is the same as viewing all web documents using only a common web browser.

1 is a Binary Format for Scene (BIFS) structure diagram.

Binary Format for Scenes (BIFS) is a standard for scene descriptions for all multimedia contents. Specifically, the BIFS is a moving picture experts group (MPEG) system for representing spatial positions, temporal relationships, and interaction with users of various multimedia objects such as audio, video, image, graphic, and text constituting multimedia contents It means scene technology. BIFS expresses scene information using a scene tree composed of various nodes.

The BIFS node may include a node for supporting video and audio standardized in MPEG-4, a node for displaying a 2D scene, and a 3D node for playing a 3D scene. The scene description can also describe a complex scene by combining the 3D scene with a portion of the 2D scene. In addition, in the node technology of the extended VRML (Virtual Reality Modeling Language) format, the scene can be freely updated by directly defining the user operation and the control operation. For example, each node can be directly assigned to a group node and a child node, and each MPEG-4 object creates one node, and the generated node includes attribute nodes corresponding to various characteristics. The node is attached to a child node of the object node, and the object nodes form a group to become a child node of the group node, and can have a hierarchical structure of several stages. Each object node has elements, and the elements have their own attributes, and these elements and properties have a hierarchical structure that can have multiple levels. Such a node may have one or more child nodes, and the nodes may be grouped to specify the location or properties of the node. The nodes defined in the system are group, children, transform, and shape, and they have shape attributes such as Sphere, Cone, Cylinder, and Box. Each node has geometry, position, color, They have the same properties. For example, if it is a Cone node, it has a Shape node as its parent node, and it has property nodes such as bottomRadius, height, side, and bottom. The scene description of an object composed of nodes is a text-based feature, and it is possible to expand a node with additional scene control such as a user event.

BIFS provides a Switch node for optional node indication. Each of the child nodes included in the switch node is assigned a unique index value, and only the child nodes having the same value are displayed on the screen by comparing the index value with the integer value given as the input.

FIG. 2 is a view illustrating an example of displaying the augmented information contents through the augmented information browser.

The concept of prior art for augmented information content playback is the same as playing a web document in a web browser. For example, referring to FIG. 2, in order to reproduce a content providing bus enhancement information, the enhancement information browser 201 receives the bus enhancement information content 203, . This method has a problem that the augmenting information provided to the user relies on the augmenting information content. For example, if a user simultaneously views bus enhancement information, subway enhancement information, and restaurant enhancement information, it is impossible that the enhancement information content does not include all of the three enhancement information. That is, there is no method that can integrally provide the augmentation information that exists independently of each other. In addition, there is a limitation in selectively providing enhancement information most suitable for the current user situation according to user environment or context information in a situation where various enhancement information contents are integrated. In particular, as the type and amount of information provided by the augmented reality service increases, the readability and usefulness of the information may be deteriorated if all the augmenting information is provided to the user all at once. In the case of a mobile terminal such as a smart phone constituted by a relatively small display device, the amount of information that can be displayed on the screen simultaneously becomes more restrictive. Therefore, in order to provide the augmented information, a technology that can automatically select and partially provide the information most suitable for the current user is required.

The present invention provides an apparatus and method for automatically selecting the most suitable information for a current user and partially providing an augmented information service.

The present invention provides an apparatus and method capable of providing a personalized augmented information service by organically integrating the augmented information contents independently of each other and selectively providing the augmented information integrated based on user context or environment context information to the user.

A receiving method according to an embodiment of the present invention is a receiving method of augmented reality service, the method comprising: a step of assigning context information to at least one augmented information contents; Generating a single augmented information content by integrating the at least one augmented information content associated with the context information; And a step of selecting an augmented information content to be provided to a user based on a user situation.

A receiving apparatus according to an exemplary embodiment of the present invention includes a context information associating unit for assigning context information to at least one augmented information contents; An augmented information content integration unit for generating a single augmented information content by integrating the at least one augmented information content associated with the context information; And a context information selection unit for selecting the augmented information contents to be provided to the user based on the user situation.

A transmitting method according to an exemplary embodiment of the present invention is a method for transmitting augmented reality service, comprising: generating at least one augmented information content including a context switch node and a context group node; And transmitting the generated at least one augmented information content.

A transmitting apparatus according to an exemplary embodiment of the present invention includes an augmented information content generating unit for generating at least one augmented information contents including a context switch node and a context group node; And a transmitter for transmitting the generated at least one augmented information contents.

The present invention provides a personalized augmented information service by organically integrating the augmented information contents that are present independently of each other and selectively providing the user with the augmented information integrated according to the user's environment or context information, It is possible to provide a suitable personalized augmented information providing service.

1 is a Binary Format for Scene (BIFS) structure diagram;
FIG. 2 illustrates an example of displaying enhancement information content through an augmentation information browser;
3 is a view showing an example of an augmented reality service;
4 is a block diagram of an augmented reality service receiving apparatus according to an embodiment of the present invention;
5 is a flowchart illustrating an operation of an augmented reality service receiving apparatus according to an embodiment of the present invention;
6 is a detailed format structure diagram of each node according to an embodiment of the present invention;
FIG. 7 is a flowchart illustrating an AR operation receiving operation in consideration of priorities according to an embodiment of the present invention; FIG.
FIG. 8 is an operation flow diagram of an augmented reality service provider according to an embodiment of the present invention; FIG. And
9 is a block diagram of an augmented reality service provider according to an embodiment of the present invention.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. It should be noted that the same components are denoted by the same reference numerals as possible in the accompanying drawings. Further, the detailed description of well-known functions and constructions that may obscure the gist of the present invention will be omitted.

Also, the terms and words used in the present specification and claims should not be construed to be limited to ordinary or dictionary meanings, and the inventor is not limited to the concept of terms in order to describe his invention in the best way. It should be construed in accordance with the meaning and concept consistent with the technical idea of the present invention based on the principle that it can be properly defined.

The mobile terminal described below may be, for example, a personal digital assistant (PDA) having a communication function, a smart phone, a mobile phone, a tablet computer, a notebook, Lt; RTI ID = 0.0 > service provider. ≪ / RTI > Hereinafter, terms such as the user environment and the user situation will be described in combination.

3 is a diagram illustrating an example of an augmented reality service.

For example, augmented reality service X is a service that provides (1) restaurant / cafe information, (2) subway information, (3) bus information, and (4) Suppose you are using. The augmented reality service will be used in combination with the augmentation information below.

As shown in FIG. 3 (a), if the four enhancement information is always provided at once, it may cause confusion for the user due to a large amount of information. However, as shown in FIG. 3 (b), when the restaurant / cafe information and Air Tag information, which are relatively unnecessary at the time of work, are removed and only the subway information and bus information frequently used at that time are displayed, The desired information can be obtained. Therefore, if the augmenting information that is most suitable for the current user is selected and provided based on the user environment or context information, information can be provided to the user more effectively.

The conventional augmented information content reproduction method includes the following restrictions.

(1) It is impossible to organically integrate the augmented information contents existing independently of each other.

(2) It is impossible to selectively provide the augmenting information most suitable for the user based on the context information on the integrated augmenting information.

Also, in the conventional BIFS, there is no method for specifying context information related to the individual node or the augmentation information provided by the nodes. Therefore, as described above, when displaying a node selectively based on user environment or context information, there are many restrictions because a separate algorithm must be implemented for all possible user scenarios using a script node and a switch node. Especially contextual information can be infinite because it contains all the information that can characterize the current situation of an object. In addition, since the subject that extracts, maintains, and manages context information can be a third contextual information service provider rather than an augmented reality service provider, interoperability and service quality deteriorate when content is developed considering only limited contextual information .

Accordingly, the present invention provides a method and apparatus for grouping various augmented information contents by context and selectively providing the augmented information contents to a user according to user environment or context information.

4 is a block diagram of an augmented reality service receiving apparatus according to an embodiment of the present invention.

FIG. 4 is a diagram illustrating an apparatus for providing selective reinforcement information based on context information and context information independently present in the present invention proposed by the present invention.

Referring to FIG. 4, a user may receive various augmented information contents in real time, or store the corresponding contents in a device in the case of augmented information service frequently used.

The augmented reality service receiving apparatus includes a context information managing unit 401, a context information linking unit 403, an augmenting information contents integrating unit 405, an augmenting information selecting unit 411, a screen displaying unit 409, a database 407, And is not limited to these components.

The context information management unit 401 provides various types of context information as objects for maintaining, managing, and providing user environment and context information. The context information management unit 401 proposed by the present invention may exist as a local context information management unit or a remote remote context information management unit not shown in the figure according to a context information management method.

The local context information management unit independently maintains, manages, and provides context information in the augmented reality service directly. On the other hand, the remote context information management unit is managed by another application service in the apparatus or acts as a proxy for delivering context information provided by the third context information service provider 430. [ The remote context information management unit may perform a form and semantic conversion operation suitable for using context information provided by an object providing actual context information, that is, another application service in the apparatus or a remote context information service provider, in the augmented reality service It can also be done.

The database 407 is connected to the context information management unit 401 and stores and extracts various types of context information at the request of the context information management unit 401. [

The context information linking unit 403 selects context information to be associated with each content and designates it for the augmented information content 420 given as input. Selection of N (e.g., 1? N? X) enhancement information contents to be integrated among a given X enhancement information contents can be performed by various methods. For example, all the augmented information contents can be integrated, or only augmented information content that is frequently used by the user can be integrated. The augmentation information content integration unit 405 integrates the augmentation information contents for which the context information is specified by the context information associating unit 403 to regenerate the content as a single augmentation information content.

The augmentation information selection unit 411 extracts the current user environment or context information from the augmented information content integrated with the context information management unit 401, selects only the most appropriate augmentation information based on the extracted user environment or context information, and transmits the selected augmentation information to the screen display unit 409 .

The screen display unit 409 plays back the reinforcement information and user input.

5 is a flowchart of an augmented reality service receiving operation according to an embodiment of the present invention.

FIG. 5 is a diagram illustrating an entire process for providing augmented information content integration and context-based selective augmented reality service on the augmented reality service receiving apparatus (hereinafter referred to as "receiving apparatus") described in FIG.

The receiving apparatus selects the N augmented information contents to be integrated on the X augmented information contents given in step 501. [ At this time, the receiving apparatus can determine which of the augmented information contents to be integrated and can selectively execute the process of acquiring the additional information related to the context information in cooperation with the context information management unit 401. [ When the augmented content to be integrated is determined in step 503, the receiving device selects context information that best describes the individual augmented information content and links them to each other. At this time, the receiving apparatus can selectively execute the process of acquiring the additional information related to the context information in cooperation with the context information management unit 401. For example, this may be a grammar for expressing an arbitrary context, or extracting context information similar to a given context information. When the context information association process is completed for the individual augmented information contents, the receiving apparatus regenerates the single augmented information contents by integrating the N augmented information contents in step 505. [ In step 507, the receiving apparatus receives the current user environment or context information from the context information management unit 401, and selects augmentation information most relevant to the user environment or context information. At this time, the receiving apparatus can selectively execute the process of acquiring additional context information.

Although it has been described above that the augmented information content to be provided to the user is selected based on the user situation or the context information, the augmented information content to be provided to the user can be selected based on at least one of the user situation, context information and user input Of course.

Finally, the receiver displays the augmentation information selected in step 509 on the screen and performs interaction with the user.

6 is a detailed format diagram of a node according to an embodiment of the present invention.

In order to provide the context-based selective augmentation information proposed in the present invention, it is necessary to (1) provide a function of designating related context information for a given augmented information content, (2) organically integrate the augmented information content, A function capable of selecting the enhancement information is required. In order to provide such a function, the present invention defines a ContextGroup node for associating an arbitrary node or group of nodes with context information and a ContextSwitch node for selecting a node based on context information.

The context group information according to the first embodiment of the present invention can be defined as shown in Table 1 below.

The elements of the context group shown in Table 1 are shown in Table 2 below.

Element Explanation addChildren A new child node can be added to the group node as a field for the event. removeChildren As a field for events, you can remove child nodes in group nodes. children And child nodes having the same context information. context Express context information of child nodes. contextRepresentationType Specifies the representation of the context stored in the context attribute. priority Indicates the priority of child nodes. If it is difficult to display all the child nodes on the screen according to the hardware performance of the apparatus, the priority is displayed on the screen. relatedURL Address for obtaining additional information related to contextual information

The context switch information according to the first embodiment of the present invention is shown in Table 3 below.

The elements of the context switch information described in Table 3 are described in Table 4 below.

Element Explanation choice The node or node group to be selected is stored. enabled And indicate whether to selectively display the augmented information based on the context information. contextQuery Context-based query contextQueryRepresentationType Specifies the expression of the query language contained in the contextQuery.

Context information representation techniques exist in various forms, from simplest key-value techniques to extensible XML (eXtensible Markup Language) schema techniques. This phenomenon exists in a form in which the contextual information query is also dependent on a specific context information presentation method. For example, when using the key-value technique, a string matching technique can be used for contextual information comparison and retrieval. For contextual information written in XML, query terms can be expressed using XQuery. Accordingly, in the present invention, the ContextGroup context attribute and the ContextSwitch node contextQuery attribute are configured to accept context information and query expressions that may exist in various forms, and consequently to be neutral to the specific context information presentation method and internal structure, Save the query. However, the actually used context information and the expression method of the query word can be specified by using the contextRepresentationType and the contextQueryRepresentationType attribute defined in the ContextGroup and ContextSwitch nodes to specify which expression method is applied in the device to interpret the context information and the query term. However, in the present invention, any context information and a method for converting a query expression into a character string and the final result are not described in the present invention.

Additional contextual information other than contextual information and query language representation can be obtained using the relatedURL attribute of the ContextGroup node. As shown in FIG. 4, when additional information on context information associated with arbitrary augmentation information is required, additional information can be obtained by sending a request through the relatedURL of the ContextGroup node related to the augmentation information. In particular, the request is handled by an entity that maintains, manages and provides real contextual information, and may thus be a third contextual information service provider present at the remote location as well as other application services within the device.

Context group information according to the second embodiment of the present invention can be defined as shown in Table 5 below.

Since each element of the context group information shown in Table 5 is described in Table 2, a detailed description thereof will be omitted.

The context switch information according to the second embodiment of the present invention is shown in Table 6 below.

Since each element of the context switch information shown in Table 6 is described in Table 4, a detailed description thereof will be omitted.

The second embodiment of the present invention differs from the first embodiment of the present invention in the method of setting the context information and the query expression system. In the first embodiment of the present invention, the contextRepresentationType and contextQueryRepresentationType attributes defined in the ContextGroup and ContextSwitch nodes are used. On the other hand, in the second embodiment of the present invention, as shown in Table 7 below, the context attribute of the ContextGroup node, which stores the actual context information and the query word, and the contextQuery attribute of the ContextSwitch node are used. That is, the same effect as that of the first embodiment of the present invention can be provided by including in the character string an identifier capable of designating the expression system in addition to the actual context information or the query term. For example, context information is expressed in a key-value manner, and context information and a query word are expressed when an identifier capable of expressing the expression is set to " MPEG-CONTEXT-REP ".

Context information MPEG-CONTEXT-REP: LOCATION = SEOUL, CATEGORY = TRANSPORTATION Query language MPEG-CONTEXT-REP: LOCATION = SEOUL || LOCATION = SUWON

FIG. 6 shows a logical structure of a BIFS scene tree created for context-based selective augmentation information providing scenarios described in FIG.

The logical structure of the BIFS scene tree is largely composed of a ContextSwitch node 600 and ContextGroup nodes 610, 612, 614, and 616.

612, 622, and 624 exist as child nodes of the ContextGroup, and the context information associated with the augmented information service is stored in the context of each ContextGroup node 610, 612, 614, and 616 Attribute. The context information directly input by the user or the context information provided by the context handler is set to the contextQuery attribute 626 of the context switch node 600 to provide the enhancement information most suitable for the current user. Accordingly, the node or node group included in the choice attribute of the context switch node 600 is compared with the context information specified in the contextQuery attribute 626 to select the augmented information having the most appropriate context information.

6 illustrates the logical relationship between ContextSwitch and ContextNode and existing BIFS nodes, the context information and the query expression expression stored in the contextQuery attribute of the ContextSwitch node and the context attribute of the ContextGroup node are assumed to be arbitrary strings, for example.

According to FIG. 6, the most suitable augmentation information for the present time can be selected as the subway information and the bus information. When the user terminal providing the augmenting information expresses both information, if the hardware limit is followed, the selected ContextGroup node Based on the priority attribute, the augmentation information to be provided preferentially can be selected.

FIG. 7 is a flowchart illustrating a receiving operation of an augmented reality service in consideration of priorities according to an embodiment of the present invention.

The specific operation of step 507 of FIG. 5 will be described with reference to FIG.

First, in step 701, the receiving apparatus confirms the user status. If the user is at work time, the receiving apparatus selects the sub-augmentation information and the bus augmentation information as the augmentation information most suitable for the current work time in step 703. The receiving apparatus confirms the priority of the subway augmentation information and bus augmentation information selected in step 705. [ If the subway augmentation information has a high priority, the receiving apparatus preferentially selects the subway augmentation information having a high priority in step 707. [

8 is a flowchart illustrating an operation of an augmented reality service provider according to an embodiment of the present invention.

Referring to FIG. 8, in order to provide an augmented reality service according to an embodiment of the present invention, in step 801, an augmented information content including a context switch node and a context group node described in FIG. 6 is generated. In the present invention, a ContextGroup node for associating an arbitrary node or node group with context information and a ContextSwitch node for selecting a node based on context information are defined.

In step 803, the service provider transmits the generated augmented information content to the receiving device.

9 is a block diagram of an augmented reality service provider according to an embodiment of the present invention.

9, a service provider includes an augmented information content generation unit 903, a control unit 901, a memory 905, a transmission / reception unit 907, and the like in order to provide an augmented reality service according to an embodiment of the present invention do.

The augmented information content generation unit 903 generates augmented information content including the ContextSwitch node and the ContextGroup node described with reference to Fig.

The transmitting / receiving unit 907 transmits the generated augmented information content to the receiving apparatus.

The memory 905 stores the augmented information content to be transmitted to the receiving apparatus.

The control unit 901 controls the augmented information content generation unit 903, the memory 905, and the transmission / reception unit 907.

It will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the spirit or scope of the invention. The present invention is not limited to the drawings.

Claims (8)

A method for receiving an augmented reality service,
Designating context information for at least one augmented information content;
Generating a single augmented information content by integrating the at least one augmented information content associated with the context information; And
And selecting the augmented reality information contents to be provided to the user based on the user situation. The method according to claim 1,
Wherein the step of selecting the augmented information content comprises:
Checking at least one of the user context, context information, and user input;
Checking the priority included in the context group information; And
And selecting the augmented information content based on a priority of the augmented reality service. The method according to claim 1,
Context information, and user input based on at least one of context information and user input. 1. An augmented reality service receiving apparatus comprising:
A context information associating unit for assigning context information to at least one augmented information contents;
An augmented information content integration unit for generating a single augmented information content by integrating the at least one augmented information content associated with the context information; And
And a context information selection unit for selecting an augmented information content to be provided to a user based on a user situation. 5. The method of claim 4,
The context information selection unit,
Checking at least one of the user context, context information, and user input;
And determines the priority information included in the context group information and selects the augmented information content based on the priority. 5. The method of claim 4,
The context information selection unit,
Context information, and user input based on at least one of context information, context information, and user input. A method for transmitting augmented reality service,
Generating at least one augmented information content including a context switch node and a context group node; And
And transmitting the generated at least one augmented information contents. 1. An augmented reality service transmitting apparatus comprising:
An augmented information content generation unit for generating at least one augmented information content including a context switch node and a context group node; And
And a transmitter for transmitting the generated at least one augmented information contents.

Images