WO2017188696A1 - Method, device, and recording medium for providing user interface in vr space - Google Patents

Abstract

Disclosed is a method for providing a user interface (UI) in a VR space by a device, the method comprising: receiving, from a user, a request for a content relating to a sports event; acquiring, in response to the received request for a content, a real and a virtual image about a place in which the sports event is held; and displaying, in response to a gesture of the user, corresponding real and virtual image about a particular location in the place in which the sports event is held.

Description

Method, device and recording medium for providing user interface in VR space

The disclosed embodiment relates to a recording medium having recorded thereon a program for performing a method for providing a user interface in a VR space, a device for providing a user interface in a VR space, and a method for providing a user interface in a VR space.

Recently, various technologies for providing an image to a user to experience virtual reality (VR) have been developed. Such virtual reality experience technologies may be realized through a device such as a head mount display (HMD) that allows a user to experience virtual reality by distorting an image output according to the user's field of view through a lens. .

Meanwhile, with the development of devices that allow users to experience virtual reality, interest in services that can be provided to users in the virtual reality world is also increasing. Since virtual reality is a space in which users can indirectly experience things that are difficult to experience in the real world, it is necessary to provide an environment that can be optimized for each user.

By implementing the environment desired by the user in the VR space, a method, a device, and a recording medium for providing a user interface that implements various services required by the user more freely in a physical reality space are provided.

Receives a content request for a sporting event from a user, and as the content request is received, obtains a photorealistic image and a virtual image of a place where a sporting event is held, and according to the user's gesture, A method of providing a user interface (UI) in a VR space by a device displaying a live image and a virtual image regarding a specific location is disclosed.

1 is a conceptual diagram illustrating a method of providing a user interface (UI) in a VR space by a device according to an exemplary embodiment.

2 is a flowchart illustrating a method of providing a UI in a virtual space by a device according to an exemplary embodiment.

3 is a diagram for describing a UI of a user provided by a device, according to an exemplary embodiment.

4 is a diagram for describing a method of providing, by a device, sports training through a UI of a user, according to an exemplary embodiment;

5 is a diagram for describing a method of providing, by a device, a medical service through a UI of a user, according to an exemplary embodiment.

6 is a diagram for describing a method of providing, by a device, a service to a plurality of users through a UI of a user, according to an exemplary embodiment.

FIG. 7 is a diagram for describing a method of providing, by a device, history information of a user through a UI of the user in a VR space according to an embodiment.

8 is a diagram for describing a method of setting, by a device, a UI of a user through other external servers according to an exemplary embodiment.

9 is a diagram for describing a method of providing, by a device, a user interface (UI) in a VR space according to an embodiment.

10 is a flowchart illustrating a method of providing a UI in a VR space by a device according to an exemplary embodiment.

FIG. 11 is a diagram for describing a method of obtaining, by a device, a user's input and displaying a search box in a VR space, according to an exemplary embodiment.

12 is a diagram for describing in more detail a UI in a VR space provided by a device, according to an exemplary embodiment.

FIG. 13 is a diagram for describing a method of providing, by a device, an advertisement by dividing a VR space in a mesh form. FIG.

14 is a diagram for describing a third virtual space in which a device provides an advertisement according to an exemplary embodiment.

15 is a flowchart illustrating a method of displaying, by a device, an advertisement and a search result in a UI on a VR space.

16 is a flowchart illustrating a method of providing content in a VR space by a device according to an exemplary embodiment.

17 is a diagram for describing in more detail a method of providing, by a device, a live action image and a virtual image of a place where a sporting event is performed.

FIG. 18 is a diagram for describing a method of providing, by a device, a combination of a live image and a virtual image of a first location in a VR space; FIG.

FIG. 19 is a diagram for describing a method of providing, by a device, a combination of a live image and a virtual image of a second position in a VR space, according to an exemplary embodiment; FIG.

20 is a diagram for describing a method of providing, by a device, a service to a user at a third location in a VR space, according to an exemplary embodiment.

21 is a diagram for describing a supporter avatar provided by a device, according to an exemplary embodiment.

FIG. 22 is a diagram for describing in detail a method of providing, by a device, content related to a sporting event according to an embodiment.

FIG. 23 is a diagram for describing a method of providing, by a device, history information of content related to a sports game viewed by a user.

24 is a diagram for describing a method of providing an artificial intelligence guide of a sporting event in a device, according to an exemplary embodiment.

FIG. 25 is a diagram illustrating a payment system using a device that provides a virtual reality (VR) space, according to an embodiment.

26 is a flowchart illustrating a payment method of a device for providing a VR space according to an embodiment.

FIG. 27 is a diagram for describing a method of paying, by a device, a right to use a service available through a user interface of a VR space.

FIG. 28 is a diagram for describing a method of allowing a device to pay for a product that can be purchased through a user interface of a VR space, according to an embodiment.

29 is a flowchart for describing a method of performing, by a device, payment for a product or a service using an avatar in a VR space.

30 is a diagram for describing a method of performing, by a device, payment for a product or a service by using an avatar in a VR space.

FIG. 31 is a flowchart illustrating a method of a service providing server paying at least one of a product and a service requested by a device.

32 and 33 are block diagrams of devices providing a UI in a VR space according to an embodiment.

34 is a block diagram of a service providing server according to an exemplary embodiment.

According to an exemplary embodiment, a method of providing a user interface (UI) in a VR space includes: obtaining identification information from a user of the device; Allocating a preset template UI for the user in the VR space based on the obtained identification information; Receiving profile information and information on an object that the user intends to place in a preset template UI from a user; And arranging the virtual object generated based on the information about the avatar and the object of the user generated based on the profile information on the template UI to generate a UI for the user.

According to an embodiment, a method of providing a user interface (UI) in a VR space includes displaying at least one category of an object that can be placed in a template UI; And displaying the plurality of objects included in the selected category as the user selects any one of the at least one category.

According to an embodiment of the present disclosure, a device providing a user interface (UI) in a VR space may include displaying payment information necessary for purchasing an object as information about the object is received; And receiving an input regarding a payment means corresponding to the payment information from the user.

According to an embodiment, a method of providing a user interface (UI) in a VR space may include displaying at least one advertisement on one side of an allocated UI; And

If any one of the displayed at least one advertisement is selected, displaying the web page associated with the selected advertisement on the assigned UI.

According to an embodiment, a method of providing a user interface (UI) in a VR space includes: obtaining biometric information of a user wearing a device according to a preset period; And generating health information of the user based on the obtained biometric information.

According to an embodiment of the present disclosure, a method of providing a user interface (UI) in a VR space may include generating at least one of medical information and recommended sports information about a user based on the generated health information; And displaying the generated information on the assigned UI.

According to an embodiment, a method of providing a user interface (UI) in a VR space includes: detecting a change in a state of a user while displaying a UI in a VR space; And changing the avatar of the user according to the detected change.

According to an embodiment, a device for providing a user interface (UI) in a VR space may include: a sensing unit configured to obtain identification information from a user of a device; On the basis of the obtained identification information, a preset template UI for the user is allocated in the VR space, and profile information and information about an object that the user wants to place in the preset template UI are received from the user, and based on the profile information A processor for generating a UI for the user by disposing a generated virtual object on the template UI based on the generated information about the avatar and the object of the user; And an output unit for displaying a UI for the generated user.

According to an embodiment, a method of providing a UI in a VR space by a device includes: displaying a search box on the VR space as a user input is obtained; Retrieving information about the object as a search request for the object is received through the displayed search box; Displaying at least one advertisement on the VR space while retrieving information about the object; And stopping the display of the at least one advertisement and displaying the information about the found object on the VR space as the search is completed.

According to an embodiment, a method of providing a UI on a VR space by a device includes selecting at least one advertisement from among a plurality of advertisements based on a profile of a user and history information about at least one object previously requested to be searched. It further comprises the step.

In a method of providing a UI on a VR space by a device according to an embodiment, displaying includes moving an image constituting at least one advertisement in a preset direction, wherein a moving speed of the image is The closer the end time of the search for the object is, the faster it is.

According to an embodiment of the present disclosure, a method of providing a UI on a VR space by a device may include receiving information about a product of a selected advertisement when a selection input for selecting at least one advertisement displayed by a user is received while searching for information about an object. And displaying on the VR space.

In the method of providing a UI in the VR space according to an embodiment of the present disclosure, the displaying may include displaying the search result and information about a product of the selected advertisement together in the VR space when a search for an object is completed. do.

In a method of providing a UI in a VR space by a device according to an exemplary embodiment, a user input includes a user's gesture, and the search box determines a size, a position, and a shape of the search box according to the user's gesture. .

According to an embodiment, a device for providing a user interface (UI) in a VR space may include: a sensing unit configured to obtain a user input; An output unit configured to display a search box on the VR space as a user input is obtained; And as a search request for the object is received through the displayed search box, while searching for information about the object and displaying information about the object, displaying at least one advertisement in the VR space, and as the search is completed. And a processor to stop displaying the at least one advertisement, wherein the output unit displays the information about the retrieved object on the VR space.

According to an embodiment, a method of providing a UI in a VR space by a device includes: receiving a content request regarding a sporting event from a user; Acquiring a photo-realistic image and a virtual image of a place at which a sporting event proceeds as the content request is received; And displaying, according to the user's gesture, a live action image and a virtual image of a specific location at a place where a sporting event is held.

According to an embodiment of the present disclosure, a method of providing a UI in a VR space includes: determining a location in a place where the sports event corresponding to a user's gesture is performed; And receiving a live image and a virtual image for the determined position.

In a method of providing a UI in a VR space by a device according to an embodiment, the live-action image includes an image photographed at each of a plurality of viewpoints of a sporting event, and the displaying of the UI is performed at a first viewpoint at a specific location. The photographed image is displayed by combining at least one image photographed at a different point in time from the first view point.

According to an embodiment of the present disclosure, a method of providing content in a VR space includes: transmitting purchase request and payment information about an object displayed on at least one of a live action image and a virtual image of a specific location; And when the purchase of the object is completed, displaying the virtual image of the object on the VR space.

According to an embodiment, there is provided a method of providing a UI in a VR space, the method including: generating history information of at least one content watched by a user before providing content related to a sporting event; If one of the at least one content is selected, displaying history information of the selected content on the VR space; And when the user selects a location included in the displayed history information, displaying a live image and a virtual image of the selected location on the VR space.

According to an embodiment of the present disclosure, a method of providing a UI in a VR space includes selecting a user who uses content related to a sporting event among a plurality of other users connected through a social network service (SNS) account of the user; Transmitting an invitation message to the VR space to a device of the selected user; And as the acceptance message for the invitation message is received, displaying a virtual image representing another user on the VR space.

According to an embodiment, a method of providing a UI in a VR space by a device includes: receiving profile information of a user; And generating an avatar image representing the user based on the received profile information, and displaying the combined avatar image together with the live image and the virtual image to display the avatar image on the VR space.

According to an embodiment, a device for providing a UI in a VR space includes: a sensing unit configured to receive a content request regarding a sports event from a user; A processor for determining a location at which a sporting event proceeds as a content request is received; A communication unit for obtaining a live-action image and a virtual image of the determined place; And an output unit for displaying a live action image and a virtual image of a specific position at a place where a sporting event is conducted according to a gesture of the user.

According to an embodiment, a payment method using a device that provides a UI in a VR space receives a user input of selecting one of items related to at least one of goods and services displayed on a user interface of a VR space provided by the device. Doing; Transmitting payment information of the item and identification information associated with the payment means, as the payment means for paying for the selected item is selected; And when the settlement of the selected item is completed, displaying the settlement completion information on the user interface.

In the payment method using a device that provides a UI in the VR space according to an embodiment, the VR space is divided into a plurality of places according to at least one kind of goods and services provided in the VR space, Receiving, in the VR space, information about costs required to access at least one of the plurality of places.

According to an embodiment, a payment method using a device that provides a UI in a VR space includes displaying payment information regarding a cost required to access a target location selected based on a user's request from among a plurality of locations; And if the payment of the cost required to access the target space is completed, displaying the avatar generated based on the user's profile in the target space.

According to an embodiment, a payment method using a device that provides a UI in a VR space may include identification of a user corresponding to at least one payment means that can be provided by the device to attribute information of an avatar on a VR space created based on a user's profile. The method may further include inputting information, and the transmitting may include transmitting attribute information of the avatar and payment information of the item to a first server providing at least one of goods and services, and extracting the user from the avatar attribute information. The identification information and the payment information of the item are transmitted from the first server to the payment server that performs the payment of the item.

The payment method using a device that provides a UI in a VR space according to an embodiment may further include displaying a virtual image related to the selected item on a user interface of the VR space as the payment is completed.

According to an embodiment, a payment method using a device that provides a UI in a VR space includes: receiving a user input of selecting one of a plurality of payment means that can be provided by the device; And acquiring preset identification information for the selected payment means.

In a payment method using a device that provides a UI in a VR space according to an embodiment, the identification information includes at least one of an iris signal, an EEG signal, and a pulse signal of the user.

According to an embodiment, a device for providing a UI in a VR space includes: a sensing unit configured to receive a user input for selecting one of items related to at least one of a product and a service displayed on a user interface of the VR space; A processor for selecting a payment means for paying for the selected item; As the payment means is selected, the communication unit for transmitting the identification information associated with the payment means and payment information of the item; And an output unit for displaying the settlement completion information on the user interface when the settlement of the selected item is completed.

Terms used herein will be briefly described and the present invention will be described in detail.

The terms used in the present invention have been selected as widely used general terms as possible in consideration of the functions in the present invention, but this may vary according to the intention or precedent of the person skilled in the art, the emergence of new technologies and the like. In addition, in certain cases, there is also a term arbitrarily selected by the applicant, in which case the meaning will be described in detail in the description of the invention. Therefore, the terms used in the present invention should be defined based on the meanings of the terms and the contents throughout the present invention, rather than the names of the simple terms.

When any part of the specification is to "include" any component, this means that it may further include other components, except to exclude other components unless otherwise stated. In addition, the terms "... unit", "module", etc. described in the specification mean a unit for processing at least one function or operation, which may be implemented in hardware or software or a combination of hardware and software. .

DETAILED DESCRIPTION Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art may easily implement the present invention. As those skilled in the art would realize, the described embodiments may be modified in various different ways, all without departing from the spirit or scope of the present invention. In the drawings, parts irrelevant to the description are omitted in order to clearly describe the present invention, and like reference numerals designate like parts throughout the specification.

1 is a conceptual diagram illustrating a method of providing a user interface (UI) in a VR space by a device 100 according to an exemplary embodiment.

The device 100 may implement a VR environment by outputting an image on a display through a pre-made lens in order to make the user feel as if the user is located in a new space different from the space in which the user is currently located.

The device 100 according to an embodiment may provide a UI for allowing a user to directly create a space so that the user may own a space unique to the user in the VR space. For example, the device 100 may provide a preset template UI 100 to the user. Here, the preset template UI 100 may include icons 110, 120, 130, 140, and 150 necessary for the user to create his or her space or control the VR space. Icons 110, 120, 130, 140, and 150 required to control the VR space include a toolbar icon 110, a home icon 120, a favorite icon 130, a settings icon 140, and a preview icon 150. ) May be included.

The toolbar icon 110 may include icons required for placing, removing, and editing an object desired by the user on the template UI 100. The home icon 120 may move a user at another place to the home in the VR space. Here, the home refers to a space that the user directly produces in the VR environment. In the VR space, there may be various places provided by a web server (not shown) connected to the device 100 or an external device. When the user wants to move to a home which is a user's own place in a VR space created by a user, the user may move to the home by touching the home icon 120.

The favorite icon 130 may include location information on the web of a place that the user mainly visits, among various places located on the VR space. For example, the user stores the address information for the first place and the second place that the user frequently visits in the VR space in the favorite icon 130, and selects the favorite icon 130 in another place to record. It is easier to move to the place.

The setting icon 140 may include profile information of the user, profile information of other users connectable to the device 100 of the user, and community information on the VR space including the user. In addition, the user may select the setting icon 140 to update previously stored user profile information, profile information of other users, community information on the VR space, and the like.

The preview icon 150 is a window in which a search result of other places, objects, etc. in the VR space searched by the user is displayed in a preview format. The device 100 may identify and show the VR space where the user is located and the search result by displaying the search result searched by the user through the preview icon 150.

According to an embodiment, the device 100 may generate a UI for the user by arranging a virtual object selected by the user on the preset template 100. In addition, the device 100 may generate not only the virtual object but also the user's avatar together on the template UI to generate a UI for the user.

When implementing the VR space, the device 100 may display the created UI of the user first, and may provide an environment in which the user may enjoy various entertainment activities in his / her own home in the VR space. On the other hand, by placing a variety of virtual objects that do not have in the real world on the user-created UI, it can be made to the user to feel the surrogate satisfaction. A detailed method of setting the UI of the user by arranging the virtual object on the preset template 100 by the user will be described later with reference to FIG. 3.

Meanwhile, the device 100 according to an embodiment may be implemented in various forms. For example, the device 100 described in the present specification may be a mobile phone, a smart phone, a laptop computer, a tablet PC, an electronic book terminal, a digital broadcasting terminal, a personal digital assistant (PDA), a PMP ( A portable multimedia player), a navigation device, a smart TV, a smart car, a consumer electronics (CE) device (eg, a refrigerator having a display panel, an air conditioner, etc.), a head mounted display (HMD), and the like may be used, but is not limited thereto.

2 is a flowchart illustrating a method of providing a UI in a virtual space by a device according to an exemplary embodiment.

In step S210, the device obtains identification information from the user of the device.

The device according to an embodiment may display a message requesting identification information to the user before setting a home space which is a user's own space on the VR space. As the message requesting identification information is displayed in the VR space, the user may input identification information of the user using at least one of a touch, a gesture, and a voice. Here, the identification information of the user may include an ID and a password of the user, but this is only an example, and the identification information is not limited to the above-described example.

In operation S220, the device allocates a preset template UI for the user in the VR space based on the obtained identification information.

According to an embodiment, as the identification information is obtained, the device 100 may allocate a preset template UI to the user. The device 100 may exchange information in real time with a home management server that sets and manages a home space for each of a plurality of users. The device 100 may directly allocate a preset template UI or receive a template UI assigned by the home management server based on user identification information.

In operation S230, the device 100 receives profile information and information about an object that the user intends to place in a preset template UI from the user.

Here, the profile information of the user may include information about a job, age, gender, body information, and a photograph of the user. In addition, the object may include objects such as furniture, transportation means, electronic devices, sports equipment, and clothing, plants such as flowers and trees, and animals such as dogs, cats, and birds.

According to an embodiment, the device 100 may detect a voice, a gesture, a touch signal, or the like of a user, and receive information about an object to be placed by the user. For example, when the user speaks "A car", the device 100 may detect a voice signal "A car" and determine that the object that the user wants to place in the template UI is A car. According to another example, the device 100 may provide images of various objects included in a category when a user inputs a category of an object through a search window on a VR space. For example, when the user inputs the term semi-medium car, the device 100 may display an image of cars included in the semi-medium car. The user may select one of the images of the cars displayed through the device 100 through a touch input or a gesture.

In operation S240, the device 100 generates a UI for the user by arranging the virtual object generated based on the information about the avatar and the object of the user generated based on the profile information on the template UI.

The device 100 according to an embodiment may generate an avatar of the user in the VR space based on the profile information. For example, the device 100 may generate an avatar having a similar body shape and appearance to the user based on the user's body information, age, and photo included in the profile information. According to another example, the device 100 may obtain information about the appearance desired by the user in addition to the profile information of the user and generate the avatar of the user based on the information.

In addition, the device 100 may arrange the generated avatar on the template UI. The user may control the position, movement and posture of the avatar on the template UI through touch, gesture, and voice input.

According to an embodiment, the device 100 may obtain a virtual object corresponding to an object that the user wants to place as information about an object that the user wants to place on the template UI is obtained. Here, the virtual object represents a three-dimensional image of the object in the virtual space.

Meanwhile, the device 100 may request a user to pay for the virtual object in order to obtain the virtual object. For example, the device 100 may request a virtual object from a server of a seller selling a virtual object corresponding to an object selected by a user. The server of the seller who has received the request requests the device 100 to pay a payment means for paying the amount set for the virtual object, and the device 100 pays the requested amount by using a predetermined payment means from the user. Can be. As the amount is paid, the device 100 may obtain a virtual object.

However, this is only an example, and the device 100 may set a session with a server of a seller and a predetermined payment server to pay a cost for obtaining a virtual object.

According to an embodiment, the device 100 may store a template UI in which an avatar and a virtual object are disposed as a UI for a user. Information about the UI of the user stored by the device 100 may be transmitted to the home management server. The home management server may store the received information about the UI of the user by matching the identification information of the user. Accordingly, when the user later accesses the VR space and inputs identification information of the user, the user UI may be received from the home management server.

3 is a diagram for describing a UI 300 of a user provided by the device 100, according to an exemplary embodiment.

Referring to FIG. 3, as the identification information of the user is input, the device 100 may load and display the UI 300 of the user in a VR space previously set by the user. Here, the UI 300 of the user may be stored in the device 100 or may be stored in the home management server.

Meanwhile, the UI 300 of the user corresponds to the toolbar icon 110, the home icon 120, the favorite icon 130, the setting icon 140, and the preview icon 150 described above with reference to FIG. 1. Icons 310, 320, 330, 340, and 350 required to control the VR space may be included.

According to an embodiment, the device 100 may display the advertisement 360 on the UI 300 of the user. Here, the advertisement 360 may select the profile information of the user. The device 100 may not display the advertisement 360 on the UI 300 of the user according to the user's selection. Meanwhile, the user may obtain a point for viewing the advertisement by displaying the advertisement 360 on the UI 300 of the user. The device 100 may obtain information regarding viewing of the advertisement 360 by recording the information about the type and the number of the displayed advertisement 360 and providing the recorded information to the point providing server.

According to an embodiment, the device 100 may display a search box 370 on the UI 300 of the user. The search box 370 may be displayed when the user wants to search for information desired by the user through a pen, finger, keyboard, or the like. The user may display the search box 370 on the UI 300 of the user through signals such as voice, gesture, and touch.

According to an embodiment, the device 100 may display at least one virtual object 380 and 390 on the UI 300 of the user. For example, when the object selected by the user is a boat, the device 100 may display the first virtual object 380, which is a 3D image of the boat, on the UI 300 of the user. Here, the first virtual object 380 may be a live image of a boat of a kind desired by the user or may be a graphic image. In addition, the device 100 may display the second virtual object 390, which is a three-dimensional image of the dock, on the UI 300 of the user.

The device 100 according to an embodiment may provide an environment in which the user may feel intimacy with the VR space by setting the UI 300 of the user as a user's own space on the VR space. Meanwhile, the above-described example is merely an example for describing the UI 300 of the user. In addition to the above-described virtual objects, various virtual objects may be disposed on the UI 300 of the user. For example, virtual objects for various objects such as vehicles, airplanes, pets, physical trainers, personal robots, and attending physicians may be placed on the user's UI 300.

4 is a diagram for describing a method of providing, by the device 100, sports training through the UI 400 of a user, according to an exemplary embodiment.

Referring to FIG. 4, the device 100 may provide training information 415 to the user through the UI 400 of the user. The training information 415 may include an image, text, a diet control table, and the like regarding the exercise motion. The user may enjoy the exercise regardless of the place through the training information 415 without visiting the gym.

Meanwhile, the device 100 according to an exemplary embodiment may arrange the avatar 410 of the user on the UI 400 of the user. In addition, the device 100 may detect a user's motion while exchanging data sensed in real time with the exercise device 405 equipped with a motion sensor including an acceleration sensor, a gyroscope sensor, a barometric pressure sensor, and the like. The device 100 may control the movement of the user's avatar 410 disposed on the UI 400 of the user according to the detected user's motion.

In addition, the device 100 may reflect the effect of the exercise performed by the user on the avatar 410 so that the user may feel a sense of achievement. For example, when the user performs the upper body exercise, the device 100 may change the existing avatar 410 into an avatar 420 having an increased muscle of the upper body.

Meanwhile, the device 100 may provide a place in a VR space in which a group exercise such as soccer or baseball may be performed through connection with at least one other user's device. For example, the device 100 may set a space that can be jointly accessed with at least one other user's device. Here, the method of setting a space that is commonly accessible may correspond to the method described above with reference to FIG. 1. For example, the plurality of users may arrange virtual objects necessary for enjoying a group exercise such as soccer and basketball while communicating with each other in the provided template UI.

5 is a diagram for describing a method of providing, by the device 100, a medical service through a UI 500 of a user, according to an exemplary embodiment.

Referring to FIG. 5, the device 100 may provide a medical service to a user through the UI 500 of the user. For example, the user may display the user's doctor's virtual object 530 on the user's UI 500 through the toolbar icon 110 described above with reference to FIG. 1. In addition, the device 100 may receive information regarding the health state of the user while transmitting and receiving data in real time with the device of the user's doctor or the server of the hospital where the doctor is located. In addition, the device 100 may transmit the biometric information of the user measured through the sensor worn by the user to the device of the user's doctor or the server of the hospital.

The device of the doctor of the user or the server of the hospital may transmit the medical information 510 analyzing the current state of health of the user to the device 100 based on the received biometric information of the user. The device 100 may display the received medical information 510 on the UI 500 of the user.

In addition, on the UI 500 of the user, health care information 520 such as a prescription, a diet, or a method of taking a drug, which is determined based on the user's health state, may be displayed.

According to an embodiment, the device 100 may receive medical treatment and psychological counseling while conducting a call in real time with a user's doctor through a communication session connected with a device of a doctor of a user. In addition, the device 100 may provide the medical service environment in which the user may feel more realistic by displaying the virtual object 530 of the user's doctor on the UI 500 of the user.

6 is a diagram for describing a method of providing, by a device 100, a service to a plurality of users through a UI 600 of a user, according to an exemplary embodiment.

Referring to FIG. 6, the device 100 may be connected to devices 20 and 30 of a plurality of different users. Here, the plurality of other users represent users who are preset to share some information with each other in the VR space with the user. The plurality of other users may be searched through contact information previously stored in the device 100 or may be searched by the user directly inputting profile information on the VR space.

The device 100 may enjoy the entertainment service with other users by displaying the virtual object selected by the user and the virtual objects selected by the other users on the UI 600 of the user. For example, on the UI 600 of the user, the virtual object 610 of the first car selected by the user of the device 100, the virtual object 620 of the second car selected by the user of the second device 20, and the first object. The virtual object 630 of the third vehicle selected by the user of the third device 30 may be displayed. Here, the virtual object 610 of the first car, the virtual object 620 of the second car, and the virtual object 630 of the third car may be virtual objects provided by a Massive Multiplayer Online Role Playing Game (MMORPG) service. have.

According to an exemplary embodiment, the device 100 displays not only a user's virtual object but also another user's virtual object on the user's UI 600, and displays data in real time with another user's device (eg, 20). You can enjoy services such as games by using virtual objects while transmitting and receiving.

FIG. 7 is a diagram for describing a method of providing, by the device 100, history information of a user through the UI 700 of the user in a VR space, according to an exemplary embodiment.

Referring to FIG. 7, the device 100 may reproduce a past situation based on an image, a video, or the like of a user stored in the device 100.

According to an embodiment, when the user selects the thumbnail image 710 for the first history information, the device 100 may reproduce the first history information. For example, when the first history information is a video in which the user cheers a soccer game with friends in the past, the device 100 may edit the video as a 3D image and display the video on the UI 700 of the user.

According to another embodiment, when the user selects the thumbnail image 720 for the second history information, the device 100 may reproduce the second history information. For example, when the second history information is a video photographing a scene sent with a grandmother of a user who has passed in the past, the device 100 reproduces the grandmother of the user who has passed in the past in a VR space, so that the user can remember past memories. It can help you remember.

As in the above-described example, the device 100 allows a user to display a thumbnail image 730 for the third history information, a thumbnail image 750 for the fourth history information, a thumbnail image 750 for the fifth history information, and a sixth. When one of the thumbnail images 760 for the history information is selected, the selected history information may be reproduced and displayed on the user interface 700.

8 is a diagram for describing a method of setting, by the device 100, a UI of a user through other external servers, according to an exemplary embodiment.

Referring to FIG. 8, the device 100 may transmit / receive information related to the UI 810 of the user in real time through the home management server 810. The home management server 810 is a server that manages identification information of a plurality of users and UIs of a plurality of users. For example, the device 100 may load the UI of the pre-stored user by inputting the identification information of the user input from the user to the home management server 810. In addition, when the user's UI does not exist, the UI of the user may be generated by allocating a preset UI template to the user based on the received identification information of the user as described above with reference to FIG. 1. have.

The device 100 may search the web for an object to be placed on the UI of the user. The device 100 may select a desired object among the searched objects and request a virtual object for the selected object. The home management server 810 may provide an environment in which the device 100 may easily purchase a virtual object by providing information of the device 100 to the seller server 820 providing the requested virtual object. The device 100 may pay the virtual object through direct communication with the seller server 820, or may pay the virtual object through a connection with a payment means providing server (not shown) to which the seller server 820 is subscribed.

FIG. 9 is a diagram for describing a method of providing, by a device, a user interface (UI) in a VR space 100 according to an embodiment.

Referring to FIG. 9, the device 100 according to an embodiment may provide a UI including a search box 910 for searching for information in the VR space 900. For example, the device 100 may display the search box 910 on the UI in the VR space 900 as a user input for requesting the display of the search box 910 is obtained from the user.

According to an embodiment, when the user inputs information about a predetermined object through voice, touch, and gesture, the device 100 may display the input information in the search window 910. For example, when the user requests information about the smart device A, the device 900 may display the name smart device A on the search window 910.

In addition, as the object for which the user wants information is displayed in the search box 910, the device 100 may search for information about the object from a web server connected to the device 100. The device 100 changes an image on the VR space 900 while searching for information about an object, thereby providing an environment in which the user may acquire new information on the UI while searching for information about the object. For example, the device 100 may include an advertisement in an image displayed on the VR space 900. Here, the advertisement may be selected based on the user's profile information such as the user's age, gender, occupation, or the like, or may be selected based on a search word previously searched by the user. A method of providing an advertisement while the device 100 searches for information about an object will be described below in detail with reference to FIGS. 12 to 14.

Meanwhile, while the information about the object is retrieved, the device 100 may move the plurality of advertisement images displayed in the VR space 900 in a predetermined direction so that the user may feel like moving to another space. have. Also, the speed at which the advertisement image is moved may be set faster as the end time of the search for the object gets closer.

In addition, when the user selects one of the advertisement images displayed on the VR space 900, the device 100 may receive and display information about the selected advertisement image. The device 100 may continue the search requested by the user while the information about the advertisement image is displayed. For example, when a plurality of advertisement images are displayed while moving in a predetermined direction on the VR space 900 of the device 100, information about a product of the advertisement image touched by the user is displayed on the VR space 900. May be displayed. Meanwhile, a method of selecting one of the plurality of advertisement images by the user is not limited to the touch method. According to another example, the user may select one of the plurality of advertisement images through voice input.

When the user selects any one of the plurality of advertisement images, the device 100 may stop displaying the plurality of advertisement images and display information on goods of the selected advertisement image on the VR space.

When the search is completed, the device 100 may display information about the searched object on the VR space 900. When the user selects a specific advertisement image while searching for information about the object, the device 100 may display the selected advertisement image and the information about the object together. According to another example, when the user does not select a specific advertisement image, the device 100 may display only information about the object on the VR space 900.

According to an embodiment, the device 100 may provide a UI in which a plurality of advertisement images move in one direction on the VR space 900 while the user requests to retrieve information about an object in the VR space 900. It is possible to provide an environment in which a variety of information can be efficiently obtained during the time required for searching.

10 is a flowchart illustrating a method of providing a UI in a VR space by a device according to an exemplary embodiment.

In operation S1010, as a user input is obtained, the device may display a search box on the VR space.

The device according to one embodiment may provide a VR space. Here, the VR space may be implemented by projecting an image displayed on the display of the device through a lens having a predetermined angle of view. The lens having the preset angle of view may be included in the device, but this is only an example, and the lens having the preset angle of view may exist outside the device.

The device may display the search box on the VR space when a user input for requesting the search box is obtained from the user. On the other hand, even if a user input for requesting a search box is obtained while the device is providing a service in a VR space through an application installed on the device, the device stops the service and displays the search box or displays the search box with the service. can do.

In operation S1020, the device may search for information about the object as a search request for the object is received through the displayed search window.

According to an embodiment, the device may receive a search request for an object through the displayed search box. For example, the user may specify an object to be searched by using an input means, a gesture, a voice signal, or the like, which can be touched. When the object is specified, the device may display the specific object in the search box for the user to check. For example, when the user specifies the object A through the smart phone or the wearable device, the device may receive the information about the specified object A and display the object A in the search box.

In addition, the device may search for information about the object A as a search request for the object A is received. The device may connect with external servers, such as a web server, to retrieve information about the object.

In operation S1030, the device may display at least one advertisement on the VR space while searching for information about the object.

According to an embodiment, the device may display an advertisement selected according to the user's profile on the VR space while searching for information about the object. For example, if the user is a male in his thirties, the device may display an advertisement regarding a product preferred by the male in his thirties.

In addition, the device may move and display the advertisement image so that the user may feel as if moving from the current position to the other position in the VR space. For example, the series of advertisement images may be moved in the first direction based on the position of the user. In addition, the device may divide the VR space into a mesh form so that more advertisements may be displayed in each divided area. Accordingly, the user may watch at least one advertisement until the search for the object is completed.

Meanwhile, when the user selects one of the at least one advertisement displayed, the device may stop displaying the advertisement and additionally provide information on the selected advertisement. This will be described later in more detail with reference to FIGS. 14 and 15.

In operation S1040, as the search is completed, the device may stop displaying the at least one advertisement and display information about the found object on the VR space.

According to an embodiment, as the search is completed, the device may stop displaying the at least one advertisement. The device may display at least one of an image, text, and a video included in the information about the searched object on the VR space.

Meanwhile, when the advertisement selected by the user while the search is in progress, the device may display information on the searched object and information on the product of the selected advertisement together in the VR space.

FIG. 11 is a diagram for describing a method of displaying, by a device, a search box 1120 on a VR space 1110 by obtaining a user input. Referring to FIG.

Referring to FIG. 11, the device may detect a gesture of a user. For example, the device may detect a user's gesture of drawing a rectangle with a finger. The device may generate an image projected through a lens having a predetermined angle of view so that the search window 1120 may be displayed at a position where a user gesture is detected in the VR space 1110.

In addition, the device may determine the size and shape of the search box displayed according to the type of the detected gesture of the user. For example, when a gesture for drawing a rectangle is obtained, the device may display a search box 1120 having a rectangular shape. According to another example, when a circular gesture is obtained, the device may display a circular search box. The device may determine the size of the search box displayed based on the user's gesture.

As the user of the device confirms that the search box 1120 is displayed, the user may request information about the object from the device. The device may provide information about the object as text data. According to another example, a user of the device may provide information about an object by dragging an image previously stored in the device to the search window 1120. According to another example, a user of a device may provide information about an object as voice data.

The device may search for the object as the object that the user wants to search through the search box 1120 is specified.

12 is a diagram for describing in more detail a UI in a VR space provided by the device 100, according to an exemplary embodiment.

Referring to FIG. 12, the device 100 may provide a first virtual space 1210, which is a space where a search window is displayed, and a second virtual space 1220, which is a space where a search result is displayed. In addition, the device 100 may provide a third virtual space 1230 in which an advertisement is displayed while the search for the object is in progress.

Here, the first virtual space 1210, the second virtual space 1220, and the third virtual space 1230 may be distinguished according to the type of information displayed in the VR space provided by the device 100. In addition, the device 100 may implement the first virtual space 1210, the second virtual space 1220, and the third virtual space 1230 on the VR space over time.

According to an embodiment, the device 100 may display a search box in a VR space as a user input for requesting a search is received from the user. Here, the space where the search box is displayed according to the user's input may be described as the first virtual space 1210.

Meanwhile, the device 100 may display at least one advertisement in the VR space until the search is completed. Here, the space where the VR space is displayed may be described as the third virtual space 1230. The device 100 may implement the third virtual space 1230 by changing the UI of the VR space to at least one image, text, and video of the advertisement. In addition, the device 100 may display an advertisement image, text, and video constituting an advertisement while moving in a predetermined direction, thereby providing an environment in which the user may feel a movement of the virtual space.

When the search is completed, the device 100 may display the second virtual space 1220 provided with the search result. As the search is completed, the device 100 stops the advertisement displayed while moving on the VR space, thereby passing the user through the third virtual space 1230 and having the same effect as being located in the second virtual space 1220. Can provide.

FIG. 13 is a diagram for describing a method of providing, by a device, an advertisement by dividing a VR space in a mesh form. FIG.

Referring to FIG. 13, as a search for an object for which a user wants information starts, the device may display a third virtual space for providing an advertisement. The device divides the VR space into a mesh form so that a user moves to a third virtual space providing an advertisement so that a plurality of advertisement images can be displayed. In addition, the plurality of advertisement images displayed in the divided area in the mesh form may move in a predetermined direction.

On the other hand, when the user selects at least one of the plurality of advertisement images displayed in the area divided into a mesh form (for example, 522 and 524), the device may display information about the product of the selected advertisement in the VR space. Can be.

According to an exemplary embodiment, the device may change images of an advertisement moving on the third virtual space according to a search time. For example, the speeds of the advertisement image 1310 moving at the start of the search, the advertisement image 1320 moving at the middle of the search, and the advertisement image 1330 moving at the last of the search may be different from each other. In detail, the speed of the advertisement image 1330 moving in the last stage of the search close to the time when the search ends may be the fastest.

14 is a diagram for describing a third virtual space 1410 in which the device 100 provides an advertisement, according to an exemplary embodiment.

Referring to FIG. 14, the device 100 may provide a third virtual space 1410 in which a plurality of advertisements are displayed in the VR space. An image constituting each of the plurality of advertisements may be displayed in the third virtual space 1410.

Also, an image constituting each of the plurality of advertisements may move in a predetermined direction. For example, when the user of the device 100 stands in the first direction, the device 100 detects the gaze direction of the user and displays an image constituting each of the plurality of advertisements in a direction opposite to the gaze direction of the user. You can move it. Here, the series of images constituting each of the plurality of advertisements may move in the second direction opposite to the first direction without disconnection until the device 100 completes the search requested by the user.

Meanwhile, the user of the device 100 may select any one of advertisement images moving by moving around the user. For example, the device 100 may select one of the plurality of advertisements through a gesture of touching one of the advertisement images. Here, the device 100 may detect a gesture of the user and select an advertisement corresponding to the gesture of the user from among advertisement images actually displayed on the display of the device 100. Since the user feels that the image displayed on the device 100 is at a distance through a lens having a preset angle of view, the advertisement selected by the user is matched by matching the user's gesture with the advertisement actually displayed on the device 100. It can determine what kind of advertisement.

In addition, when the user selects the advertisement A, the device 100 may display the information 1420 of the vehicle A, which is a product of the advertisement A, on the VR space. The device 100 may stop displaying other advertisements as the advertisement A is selected, and display the information 620 of the vehicle A on the VR space. According to an embodiment, the device 100 may display the information 1420 of the car A until the search is completed, and when the search is completed, the device 100 may implement a second virtual space in which the search result is displayed on the VR space. have.

 The device 100 may preset the type of advertisement displayed on the third virtual space 1410 based on the profile of the user.

15 is a flowchart illustrating a method of displaying, by a device, an advertisement and a search result in a UI on a VR space.

In operation S1510, the device may search for information about the object as a search request for the object is received through the displayed search window.

In operation S1520, the device may select at least one advertisement from among the plurality of advertisements based on the user's profile and history information about the at least one object previously requested to be searched.

According to an embodiment, the device may provide profile information of a user to an advertisement server that provides an advertisement, and may receive information about products that are preferred by other users having a profile similar to that of the user. Also, according to another exemplary embodiment, the device may be provided with an advertisement about a product determined to be of high interest by providing information of an object previously searched by the user to an advertisement server providing an advertisement.

In operation S1530, the device may display the selected advertisement while searching for information about the object.

According to an embodiment, the device may display the selected advertisement on the VR space while searching for information about the object is in progress.

In operation S1540, as the search is completed, the device may stop displaying the at least one advertisement and display information about the found object on the VR space.

According to an embodiment, as the search is completed, the device may stop displaying the at least one advertisement. The device may display at least one of an image, text, and a video included in the information about the searched object on the VR space.

Meanwhile, when the advertisement selected by the user while the search is in progress, the device may display information on the searched object and information on the product of the selected advertisement together in the VR space.

16 is a flowchart illustrating a method of providing content in a VR space by a device according to an exemplary embodiment.

In operation S1610, the device may receive a content request regarding a sporting event from a user wearing the device.

For example, the device may receive a content request from the user that includes information regarding the type of sporting event. Here, the information on the type of sports event may include information on at least one of a place, a date, and a type of team participating in a sporting event.

In addition, the user may provide information regarding the type of sports event to the device through input signals such as voice and touch. For example, when the user touches one side of the device, the device may display a screen for inputting information regarding the type of sports event. The user may input information regarding the type of sporting event on the displayed screen. According to another example, when the user speaks information on the type of sports event, the device may recognize the user's voice to obtain information on the type of the sports event.

In operation S1620, as the content request is received, the device may acquire a photorealistic image and a virtual image of a place where a sporting event is held.

According to an embodiment of the present disclosure, when a content request is received from a user, the device may request a live image and a virtual image from a server that generates a live image and a virtual image of a place where a sporting event is held.

For example, the device may request a live-action image and a virtual image of the entrance of the stadium A where the sporting event is held. Here, the due diligence image may be an image photographing a situation when the server receives a request from the device. In addition, the virtual image may be a graphic image generated to allow a user to feel a sporting event more realistically in a VR space. For example, the virtual image may include images of advertisements, virtual stores, and virtual museums provided by sponsors sponsoring a game performed in the stadium. According to another example, the virtual image may include a supporter avatar representing the user.

In operation S1630, the device may display a live action image and a virtual image of a specific location at a place where a sporting event is conducted according to a gesture of the user.

According to an exemplary embodiment, as the user's gesture is detected, the device may acquire a photorealistic image and a virtual image of a specific location at a place where a sporting event is conducted. Here, the place where the sporting event is conducted may be distinguished into a plurality of locations. For example, a place where a sporting event is held may be divided into a central station, a central road, a square, and a stadium.

When the user at the location of the central station stands in the first direction and makes a walking gesture, the device may acquire and display a live image and a virtual image of the central road, which is the next location of the central station, from the server. However, this is only an example, and the user may control a device to display a virtual image and a live image of a desired position by inputting a voice to the device at a position that the user wants to move.

According to an embodiment, the device may communicate in real time with at least one server that generates a virtual image and a live image of a plurality of locations included in a place where a sporting event is held, thereby realizing a virtual image of a location corresponding to a user gesture. And live image. Here, the request for the virtual image and the live image transmitted from the device to the server may include identification information regarding a location corresponding to the gesture of the user.

17 is a diagram for describing in more detail a method of providing, by a device, a live action image and a virtual image of a place where a sporting event is performed.

Referring to FIG. 17, when the device receives a content request for a sporting event from a user, the device may obtain a photorealistic image and a virtual image of A1, which is a place where a sporting event is conducted. On the other hand, A1, which is a place where a sports game is conducted, may be distinguished into a plurality of locations. Information about the plurality of locations may be displayed through the device as a map 1700 of the stadium. In the present embodiment, when the device receives a content request for a sporting event from a user, it is assumed that a real image and a virtual image regarding a location A 1710, which is an initial location, are acquired at a location A1.

The device may display the due diligence image and the virtual image for location A 1710. Meanwhile, the virtual image may include a map of place A1, a supporter avatar, and an advertisement. The device may display the live image and the virtual image in combination according to the rendering information received from the server. The rendering information may include stitching information in which the live image and the virtual image are combined, location information on the live image in which the virtual image is disposed, and the like. An example in which the live image and the virtual image are combined will be described later with reference to FIGS. 18 to 20.

According to an embodiment, the device may detect a gesture of a user and determine a location in A1 that the user wants to move. For example, a location within A1 that a user can move may include location A (1710) which is a central station, location B (1720) which is a central road, location C (1730) which is a square, and location D (1740) of a stadium. have. In addition, in the case of the location C (1730) which is a square, a host city public hall (1731), a souvenir shop (1732), a soccer game museum (1733), a toto shop (1734), a local partner showroom (1735), a FIFA world game museum (1736) ), The global partner showroom 1735 and the opponent team square 1738 may be additionally included. In addition, in the position D 1740 of the stadium, the user may move to the position A2 1750 which is the opposing team position of the team selected by the user according to the selection.

In addition, when the user selects the avatar selection and the community invitation room 1705, the device may generate an avatar representing the user or provide information about another user who uses content related to a sporting event.

FIG. 18 is a diagram for describing a method of providing, by a device, a combination of a photorealistic image 1820 and a virtual image 1810 and 1830 for a first location in a VR space.

Referring to FIG. 18, when a content request for a sporting event is received from a user, the device may acquire a photorealistic image 1820 and a virtual image 1810 and 1830 of the first location. For example, the device may receive the live image 1820 and the virtual image 1810, 1830 in real time from a server that generates the live image 1820 and the virtual images 1810, 1830 of the first location. Can be.

Meanwhile, the device may receive an image in which the photorealistic image 1820 and the virtual images 1810 and 1830 are combined, and the stitching information for combining the photorealistic image 1820 and the virtual images 1810 and 1830 together. You may.

The virtual images 1810 and 1830 may include an advertisement image 1832 of a company sponsoring a sporting event or a host city of a sporting event. According to another example, the user may select the first virtual image 1810 and move to another location connected to the first virtual image 1810. For example, the user may move to the avatar selection and community invitation room 1705 described above with reference to FIG. 17 by selecting the first virtual image 1810. The user may move from the moved location to a location where the user can connect with another user to use the content related to the sporting event.

In addition, the user may select the first virtual image 1810 to select an avatar representing the user. For example, when the user selects the first virtual image 1810 and moves to the avatar selection and community invitation room 1705, the device may acquire an image of the user. The image of the user may be obtained from a 3D scanning device or the like that scans the user, but this is only an example, and the image of the user may be stored in advance in the device. The device may generate a supporter avatar based on the obtained user image.

The supporter avatar may be used to perform a process such as authentication when moving to another location later. For example, in a place offering a game, it is necessary to confirm whether a fee is paid to move to a location other than the current location. In this case, the device may recognize the supporter avatar representing the user to the server managing the other location, thereby providing information that the user paid the fee.

The device may more realistically provide an environment required for a user to watch a sporting event in a VR space by displaying an image in which the photorealistic image 1820 and the virtual images 1810 and 1830 for the first location are combined.

FIG. 19 is a diagram for describing a method of providing, by a device, a combination of a live image 1900 and a virtual image 1920, 1940 for a second location in a VR space.

Referring to FIG. 19, when a gesture for requesting a movement to a second location is detected from a user, the device may acquire the photorealistic image 1900 and the virtual images 1920 and 1940 related to the second location. The due diligence image 1900 of the second position represents an image of the second position. The virtual images 1920 and 1940 related to the second location may include an advertisement image 1920 and a second virtual image 1940 for a service reflecting a user's taste. Before moving to the second position, the user may select the type of the second virtual image 1940 to be viewed together with the live image 1900. For example, a performance image, an image of a tourist attraction of a game hosting city, and the like may be included in the second virtual image 1940.

Here, the device may display in advance a type of virtual image that the user may combine with the live image image 1900 to allow the user to select one of the displayed types of the virtual image. In addition, the device may display discount information, point information, etc., which the user can receive by selecting a specific virtual image.

Meanwhile, according to an exemplary embodiment, the device may display a supporter avatar 1910 representing a user on an image in which the photorealistic image 1900 and the virtual images 1920 and 1940 are combined. The device may display the supporter avatar 1910 to move according to the user's gesture, thereby giving the user an effect as if the user is in the actual second position.

In addition, the device may display, along with the supporter avatar 1910 of the user, an avatar 1930 of another user who uses content related to a sporting event together with the user. To this end, the device may transmit and receive information with another user's device in real time. For example, the device may generate the state information of the user based on the voice of the user, the gesture of the user, and provide the same to the device of another user. In addition, the device receives other user's state information generated based on another user's voice, another user's gesture, etc., and displays the supporter avatar 1910 and the other user's avatar 1930 based on the received information. can do. Meanwhile, the user may select another user who uses the content related to the sports event together with the user based on the social network service (SNS) service provided by the device.

20 is a diagram for describing a method of providing, by a device, a service to a user at a third location in a VR space, according to an exemplary embodiment.

Referring to FIG. 20, the device 100 may receive an image 2010 in which a live image and a virtual image of a third location are combined according to a gesture of a user. Here, it is assumed that the information about the position which can move according to the gesture of the user is set in advance. Further, in the present embodiment, the third position is assumed to be the souvenir shop 1732 described above with reference to FIG. In the third location, an image obtained by combining a photorealistic image of a product (eg, 2012) that a user can purchase in the souvenir shop 1732 and a virtual image representing a background of the souvenir shop may be acquired in the device 100. have.

The user of the device 100 may select a desired product (eg, 2012) while watching a product (eg, 2012) displayed on the third location. In addition, when the user selects the product A 2012, the device 100 may request the user to input authentication information for payment in order to execute a payment means for purchasing the product A 2012.

Also, the device 100 may apply the virtual image of the product A 2012 to the supporter avatar before the user purchases the product A 2012 so that the user may experience the product A 2012 in advance. have. In addition, the user may purchase product A 2012 directly or may purchase a virtual image for product A 2012. When the user purchases the virtual image for the merchandise A 2012, the device 100 stores the virtual image for the merchandise A 2012 in the information of the supporter avatar, so that the user may later select another content about the sporting event. In this case, the stored virtual image may be loaded and applied to the supporter avatar.

21 is a diagram for describing a supporter avatar 2110 provided by a device, according to an exemplary embodiment.

Referring to FIG. 21, the device may generate the supporter avatar 2110 based on user information. Here, the user's information may include a profile including the user's gender, age, body information, occupation, and the like, and an image of the user. In addition, for payment in the VR space using the supporter avatar 2110, identification information regarding the user's payment means may be included in the user's information.

When the user moves to a specific location in the VR space, the device may display the supporter avatar 2110 on an image in which the live image and the virtual image of the moved specific location are combined according to the gesture obtained from the user. Here, the method of displaying the supporter avatar 2110 may correspond to the method described above with reference to FIG. 19.

Meanwhile, the device may apply the user's state in the VR space to the supporter avatar 2110. Here, the state of the user may be determined according to product information purchased by the user in the VR space, information of a sports team, and the like that the user supports. For example, the device may include a virtual image 2111 for a flag of a sports team that a user purchased in VR space, a virtual image 2113 for a trophy, virtual images 2115 and 2119 for a race suit, and a team scarf. The virtual image 2117 may be applied to the supporter avatar 2110. Here, the virtual image represents a three-dimensional image of an object such as an actual product.

The device may display an image of the supporter avatar 610 according to the user's request even when the user does not use the content related to the sporting event. In addition, the image of the supporter avatar 610 may be shared with other users through an application such as SNS.

FIG. 22 is a diagram for describing in detail a method of providing, by a device, content related to a sporting event according to an embodiment.

Referring to FIG. 22, the device may acquire a gesture for requesting a change of location to the stadium 1740 described above with reference to FIG. 17. Accordingly, the device may display a combination of photorealistic images 2210, 2220, and 2240 and virtual images 2230 of the stadium 1740.

For example, the device may acquire a photorealistic image 2210 of the stadium at a time corresponding to the location information of the seat table, based on the position information of the seat table in the stadium purchased by the user in advance. Also, the device may use the photo-realistic image 2220 at the first time point and the photo-realistic image 2240 at the second time point to correspond to the user's location information so that the user may view the game from various angles. ) Can be provided with.

On the other hand, in this case, when the live image is provided, the location of the user may be updated according to the changed location of the user. For example, when the device acquires a gesture that moves to the right from the user, the device may obtain a photorealistic image at a position moved by a predetermined distance in the right direction from the current viewpoint.

In addition, the device according to an embodiment may provide the game brokerage information regarding the position and the scoring situation of the player in the ongoing game as the virtual image 2230. However, this is only an example, and the virtual image 2230 may display an advertisement image provided by a sponsor sponsoring a game.

According to another embodiment, when an input for requesting watching another game is received from the user, the device may display the live image and the virtual image of the other game together with the currently displayed image in a multi-screen form.

FIG. 23 is a diagram for describing a method of providing, by a device, history information of content related to a sports game viewed by a user.

Referring to FIG. 23, the device may provide images 2320, 2330, 2340, 2350, 2360, and 2370 related to a sport game that the user has watched in the past to the user in the preset history space 2310. Here, the images 2320, 2330, 2340, 2350, 2360, and 2370 may be provided in at least one form of a still image or a moving image.

According to an embodiment of the present disclosure, when a request for any one of images 2320, 2330, 2340, 2350, 2360, and 2370 related to a sports game that a user has watched is received, the device displays an image corresponding to the received request. can do. Here, the images 2320, 2330, 2340, 2350, 2360, and 2370 related to the sporting event may be images in which the virtual image and the live action image described above with reference to FIGS. 18 to 22 are combined.

In addition, when the user selects a location displayed on the selected image while viewing the selected image, the device may display the virtual image and the live image of the selected location. Accordingly, the user may move to a position where a game that the user has watched in the past is performed, and may be provided with an environment in which the user may experience the past memory again.

24 is a diagram for describing a method of providing an artificial intelligence guide of a sporting event in a device, according to an exemplary embodiment.

According to an embodiment, the device may provide at least one guide 2410 or 2420 in the virtual space 2400 generated by combining a virtual image and a live image of a place where a sports game can be viewed.

The device may set the profile information and the preference information of the user as input values of the neural network to generate at least one guide 2410 or 2420 for providing the user with information about a sporting event. Here, the profile information of the user may include information about the nationality, race, age, gender, etc. of the user. In addition, the preference information may include information about characteristics of the person that the user prefers. The device may set a face, a body type, a key, a voice, etc. of the guide (eg, 2410) based on the user's profile and preference information.

According to another embodiment, the device may receive payment information for use of a copyright or a portrait right of a sports star, a singer, an actor, a competition mascot, or an animation hero of the corresponding item from a server managing the copyright or the portrait right. The device provides the payment information to the user, and when the user's payment approval is completed, at least one guide 2410 or 2420 is generated based on the acquired characteristics such as a sports star, a singer, an actor, a competition mascot, and an animated character. can do.

According to an embodiment, the guide (eg, 2410) generated may provide a user of the device with information about a place and a sporting event to watch a sporting event.

FIG. 25 is a diagram for describing a payment system (hereinafter, referred to as a payment system) using a device 100 that provides a VR (Virtual Reality) space according to an embodiment.

Referring to FIG. 25, a payment system according to an embodiment may include a device 100, a service providing server 2510, and a payment server 2520. However, this is merely an example of a payment system, and the present invention is not limited through FIG. 25. That is, according to various embodiments of the present disclosure, the payment system may be configured differently from FIG. 25.

Each component of the payment system of FIG. 25 is generally connected via a network.

Meanwhile, the device 100 may receive an image for providing a VR space from the service providing server 10. According to an embodiment, an image provided to the device 100 from another service providing server 10 may include an item regarding at least one of goods and services. Here, the product may include various objects that can be traded through the goods, such as smart devices, cars, furniture, clothes and sports equipment. In addition, the service may include a sports relay service, a health care service, a game service, and a community service.

The device 100 may receive a user input for selecting any one of at least one item displayed on the output image. Here, the user input may be in the form of a gesture. For example, the device 100 may detect a gesture of touching the exercise device A in a VR space that the user senses through the output image. However, this is only an example, and the user input may be obtained from an input means such as a touch pen or a smart phone, or may be obtained through a user's voice.

The device 100 may select one of at least one item displayed on the output image based on the received user input. In addition, the device 100 may display information on the amount of money necessary to purchase the selected item. When the user approves the purchase of the selected item, the device 100 may transmit a purchase progress request to the service providing server 10. Accordingly, the service providing server 2510 may receive identification information related to the payment means selected by the user of the device 100 from the device 100 and provide it to the payment server 2520. According to another example, the service providing server 2510 may connect the device 100 and the payment server 2520 so that the device 100 may provide identification information related to a payment method directly to the payment server 2520.

Meanwhile, as a result of the payment of the selected item based on the identification information of the device 100 in the payment server 2520, when the payment is completed, the payment server directly provides the payment completion information or the service providing server 2510. The payment completion information may be provided to the device 100 through. In this case, the payment completion information may include delivery information of the product, information on using the service, payment information, and the like, but this is only an example and the payment completion information is not limited to the above-described example.

FIG. 26 is a flowchart illustrating a payment method of a device 100 that provides a VR space, according to an exemplary embodiment.

In operation S2610, the device 100 may receive a user input of selecting one of items related to at least one of goods and services displayed on the user interface of the VR space provided by the device 100. Here, the item may include identification information representing each of goods and services provided through the user interface of the VR space. For example, an item may include a product name, a service name, a thumbnail image of the service, a catalog of the product, and the like. The item may be displayed in the form of any one of text, an image, and a video.

A user wearing the device 100 may select any one of items related to at least one. For example, a user wearing the device 100 may select a specific item by touching an area in which the specific item is displayed on the sensed VR image or speaking the name of the specific item through voice. However, this is only an example, and a method of selecting one of at least one item by a user wearing the device 100 is not limited to the above-described example.

In operation S2620, as the payment means for paying for the selected item is selected, the device 100 may transmit identification information related to the payment means and payment information of the item.

As the item is selected, the device 100 according to an embodiment may display, on the user interface of the VR space, cost information to be paid for using the selected item. In this case, the cost information about the selected item may be received from one of the service providing server 2510 and the payment server 2520.

In addition, the device 100 may display a window for selecting any one of a plurality of payment means capable of paying for the selected item. The user of the device 100 may select one of a plurality of payment means through the displayed window.

The device 100 may transmit identification information related to the selected payment means and payment information of the item. Here, the identification information related to the payment means may include a password, etc. preset by the user to approve the use of the payment means. In addition, the payment information may include a kind of a product or a service corresponding to the item, shipping address information required to receive the product or use the service.

Meanwhile, the device 100 may transmit identification information related to the payment means and payment information of the item to the payment server 2520 through the service providing server 2510 or directly to the payment server 2520.

In operation S2630, when payment of the selected item is completed, the device 100 may display payment completion information on the user interface.

In the payment server 2520, when the identification information input by the user about the payment method matches the identification information of the pre-stored user, the device 100 may perform payment of the selected item. In addition, when the payment is completed, the device 100 may receive payment completion information from the payment server 2520. In this case, the payment completion information may be directly transmitted from the payment server 2520 to the device 100 or may be transmitted from the payment server 2520 to the device 100 through the service providing server 2510.

Meanwhile, as the payment of the selected item is completed, the device 100 may acquire a right to use a service provided by the service providing server 2510 or obtain a product. Here, the product may be an actual product and a virtual product provided in the VR space. The virtual product may include either a real image of the product or a graphic image of the product.

FIG. 27 is a diagram for describing a method of paying, by the device 100, a user's right to use a service available through a user interface of a VR space.

Referring to FIG. 27, the device 100 may provide various types of services to a user through a VR space. Here, the device 100 may implement at least one place in the VR space in order to provide a service to the user.

For example, when the device 100 provides a sports relay service to a user, the device 100 may change the VR space to a stadium where a sporting event is held. However, this is only an example, and the device 100 may change the VR space to an art gallery, a performance hall, or the like in order to provide a user with an artwork viewing service or a musical viewing service. In order to change the VR space into a place where a service is provided, the device 100 may output a combined image of a live image and a graphic image of a place where a service is provided.

Meanwhile, the user may select one of the at least one service by selecting an item for each of the places where the at least one service displayed on the user interface of the VR space is provided. For example, the user may display an item 2710 for a artwork viewing service, an item 2720 for a musical viewing service, and an item 2730 for viewing a sports event, which the device 100 displays on a user interface of a VR space. By selecting any one of the above, one of the above-described services may be selected.

The device 100 according to an embodiment may receive information necessary to implement a place where the selected service is provided from the service providing server 2510. Meanwhile, in order for the device 100 to obtain information necessary for implementing a place where a service is provided from the service providing server 2510, the device 100 may have to pay a cost designated by the service providing server 2510.

Accordingly, the device 100 may display the cost information to be paid for using the selected service on the user interface. When the user wants to pay for using the selected service, the user may select any one of a plurality of payment means that can be provided through the device 100. When the user selects any one of a plurality of payment means, the device 100 may display a window on which the user's identification information for the selected payment means can be input on the user interface in the VR space.

When the user inputs identification information, the device 100 may transmit the input identification information to the payment server 2520 through the service providing server 2510 or directly to the payment server 2520. As it is confirmed that the identification information of the user received from the payment server 2520 matches the identification information of the preset user, payment for the selected service may be completed.

As the payment for the selected service is completed, the device 100 may set the VR space as a place where the selected service may be provided. For example, when the user pays for a sports relay service, the device 100 may implement a VR space by outputting a combination of live image and graphic image of a stadium where a sporting event is held.

FIG. 28 is a diagram for describing a method of paying, by the device 100, a product that can be purchased through a user interface of a VR space, according to an embodiment.

The device 100 according to an embodiment may display an item regarding at least one product that can be purchased through the VR space. For example, when the above-described sports relay service is provided through the VR space, the device 100 may implement the souvenir hall 2800 in the VR space in the stadium where the sports relay service is provided. The souvenir hall 2800 implemented in the VR space of the device 100 may display an item (eg, 2810) regarding at least one product that can be purchased by the user. Here, the item (for example, 2810) relating to the at least one product may be a photorealistic image, a graphic image, or the like of the at least one product, but this is merely an example, and the item regarding the product is not limited thereto.

The user of the device 100 may select any one of items (eg, 2810) related to the displayed at least one product. For example, a user of the device 100 may touch and select an item 2810 related to the first product.

As the user selects the item 2810 related to the first product, the device 100 may transmit information about the item to the service providing server 2510. As the user of the device 100 selects the item 2810 regarding the first product, the service providing server 2510 includes information about a payment amount of the first product, a purchase option, and a payment method available to the user. Payment information may be provided to the device 100. Here, the purchase option is information indicating whether the purchase of the virtual goods and the actual goods is about.

Meanwhile, the device 100 may display the received payment information on the user interface of the device 100. The user of the device 100 may determine whether to purchase a product based on the displayed payment information. When the user of the device 100 decides to purchase a product, one of a plurality of payment means may be selected, and identification information about the selected payment means may be input to the device 100. In addition, the device 100 may provide input identification information of the user to the payment server 2520 through the service providing server 2510 or directly to the payment server 2520.

29 is a flowchart illustrating a method for the device 100 to make a payment for a product or a service using an avatar in a VR space, according to an exemplary embodiment.

In operation S2910, the device 100 may input identification information of the user corresponding to at least one payment means that can be provided by the device, to attribute information of the avatar on the VR space created based on the user's profile.

According to an embodiment, the device 100 may provide an avatar representing a user in a VR space. Here, the avatar may be produced based on a profile of the user's age, gender, occupation, and the like. In addition, the avatar may be produced based on the image of the user. For example, the device 100 may provide a profile of the user and an image of the user to the service providing server 10 to receive an avatar of the user generated by the service providing server 10. According to another example, the device 100 may generate an avatar of the user based on the profile of the user and the image of the user through the avatar generating application provided from the service providing server 10.

Meanwhile, the device 100 according to an embodiment may input identification information of a payment means for paying a cost of a product or a service to attribute information of an avatar for convenience of payment in a VR space. For example, when the payment means is selected as the xx pay, the device 100 may input identification information of the user for the xx pay to the attribute information of the avatar.

In operation S2920, the device 100 may receive a user input of selecting one of items related to at least one of goods and services displayed on the user interface of the VR space provided by the device 100.

Meanwhile, step S2920 may correspond to step S2610 described above with reference to FIG. 26.

In operation S2930, the device 100 may transmit attribute information of the avatar and payment information of the item to a first server that provides at least one of goods and services.

Here, the first server may be the service providing server 2510 or the payment server 2520 described above with reference to FIG. 1. The device 100 may access an item related to at least one of goods and services using an avatar created based on the user's profile in the VR space.

On the other hand, as described above in step S2920, in order to receive the goods or services corresponding to the item selected by the user has to pay a corresponding cost. The device 100 according to an embodiment may replace the payment process by transmitting attribute information of the avatar and payment information of the item to the first server without going through a separate payment process. For example, the first server that receives the attribute information of the avatar including the information of the payment means and the identification information of the payment means from the device 100 determines that the user approves the payment, and thus the product corresponding to the selected item. Alternatively, the use authority of the service may be granted to the device 100.

In operation S2940, when the settlement of the selected item is completed, the device 100 may display settlement completion information on the user interface.

Meanwhile, step S2940 may correspond to step S2630 described above with reference to FIG. 26.

30 is a diagram for describing a method of performing, by a device, a payment for a product or a service using an avatar in a VR space, according to an exemplary embodiment.

Referring to FIG. 30, the attribute information of the avatar 3010 generated based on the profile of the user of the device 100 may include gender, age, occupation, and identification information of the payment means 1 of the user.

According to an embodiment, the device 100 may display information about payment amount and quantity of the flag 3020 and the trophy 3030 selected by the user, and a request for purchase by the user. The payment information including the information indicating whether the virtual goods or the virtual goods may be transmitted to the first server along with the attribute information. As described above with reference to FIG. 29, the first server may be a service providing server 2510 or a payment server 2520. Here, when the device 100 transmits the payment information and the attribute information to the service providing server 2510, the transmitted payment information and the attribute information may be transmitted to the payment server 2520.

Meanwhile, as payment information and attribute information of the device 100 are transmitted to the first server, a payment process for the selected flag 3020 and the trophy 3030 may be performed. The first server may determine that the purchase of the flag 3020 and the trophy 3030 has been approved by the user, and allow the device 100 to use the flag 3020 and the trophy 3030. In the present embodiment, it is assumed that the user of the device 100 requests the purchase of the virtual goods of the flag 3020 and the trophy 3030.

As payment of the virtual goods related to the flag 3020 and the trophy 3030 of the device 100 is completed, an avatar displayed on the VR space provided by the device 100 includes a purchased flag 3020 and a trophy 3030. ) May be applied.

31 is a flowchart for describing a method of paying at least one of goods and services requested by the device 100 by the service providing server 2510, according to an exemplary embodiment.

In operation S3110, the service providing server 2510 may provide an item regarding at least one of goods and services on a user interface of a VR space provided by the device 100.

The service providing server 2510 according to an exemplary embodiment may transmit an item regarding at least one item representing information on a service available in a VR space and information on a product to be purchased, to the device 100. The device 100 may display an item regarding at least one received on the user interface of the VR space.

In operation S3120, the service providing server 2510 may receive a user input from the device 100 that selects any one of items provided on at least one of the provided items.

In operation S3130, the service providing server 2510 may provide the device 100 with information about a cost to be paid for using a product or a service corresponding to the item selected based on the user input.

In order to use the goods or services provided by the service providing server 2510 according to an exemplary embodiment, a predetermined cost should be paid for the goods or services. Accordingly, the service providing server 2510 may transmit information about the cost to be paid to the device 100.

In operation S3140, the service providing server 2510 may receive identification information related to a payment means and payment information of an item from the device 100.

According to an embodiment, the service providing server 2510 may include payment information including identification information of a payment method selected by the user from the device 100 and information about at least one of a quantity and a use period of a product or service that the user wants to use. Can be received.

In addition, the service providing server 2510 may transmit identification information and payment information to the payment server 2520 for payment of the selected product or service. However, this is only an example, and when the service providing server 2510 may proceed with a payment, the payment process may be performed directly without a separate payment server 2520.

In operation S3150, the service providing server 2510 may approve the right to use the product or the service to the device 100 as the payment for the product or the service is completed.

The service providing server 2510 according to an embodiment may receive information indicating that payment of a product or service selected by a user is completed from the payment server 2520 when the payment is made by the payment server 2520. Accordingly, the service providing server 10 may authorize the user to use the goods or services.

However, this is only an example. When the payment process is directly performed by the service providing server 2510, as the payment is completed, the user may approve the user's right to use the product or service.

32 and 33 are block diagrams of a device 100 providing a UI in a VR space according to an embodiment.

As shown in FIG. 32, the device 100 according to an embodiment may include a sensing unit 110, a processor 120, an output unit 130, and a communication unit 160. However, not all illustrated components are essential components. The device 100 may be implemented by more components than the illustrated components, and the device 100 may be implemented by fewer components.

For example, as shown in FIG. 32, the device 100 according to an embodiment of the present invention may include A / A in addition to the sensing unit 110, the processor 120, the output unit 130, and the communication unit 160. The V input unit 140, the user input unit 150, and the memory 170 may be further included.

Meanwhile, the device 100 illustrated in FIGS. 32 and 33 may perform operations of the device described above with reference to FIGS. 1 to 31.

Hereinafter, the components will be described in turn.

The sensing unit 110 may detect at least one of a state of the device 100, a state around the device 100, and a state of a user wearing the device 100, and transmit the detected information to the processor 120. have.

According to an embodiment, the sensing unit 110 may obtain identification information of a user wearing the device 100. In addition, the sensing unit 110 may track the movement of the user wearing the device 100.

Meanwhile, the sensing unit 110 may acquire biometric information of a user wearing the device 100 at a predetermined cycle. In addition, the sensing unit 110 may detect a state change of the user while displaying the UI in the VR space.

According to an embodiment, the sensing unit 110 may obtain an input of a user wearing the device 100. For example, the sensing unit 110 may detect a user's gesture and may obtain a user's input. According to another example, the sensing unit 110 may receive a content request for a sports event from the user.

According to another example, the sensing unit 110 may receive a user input for selecting any one of items related to at least one of goods and services displayed on the user interface of the VR space.

In addition, the sensing unit 110 may receive a user input for selecting any one of a plurality of payment means that can be provided by the device.

The sensing unit 110 may include a geomagnetic sensor 111, an acceleration sensor 112, a temperature / humidity sensor 113, an infrared sensor 114, a gyroscope sensor 115, a position sensor. (Eg, GPS) 116, barometric pressure sensor 117, proximity sensor 118, and RGB sensor (illuminance sensor) 119, but is not limited thereto. According to another example, the sensing unit 110 may further include a biosignal detection sensor capable of measuring a user's pulse, blood pressure, brain wave, and the like. Since functions of the respective sensors can be intuitively deduced by those skilled in the art from the names, detailed descriptions thereof will be omitted.

The processor 120 typically controls the overall operation of the device 100. For example, the processor 120 executes programs stored in the memory 170 to detect the sensing unit 110, the output unit 130, the A / V input unit 140, the user input unit 150, and the communication unit 160. ) Can be controlled overall.

The processor 120 according to an exemplary embodiment may allocate a preset template UI for the user in the VR space based on the identification information obtained through the sensing unit 110. In addition, the processor 120 may obtain, via the user input unit 150, profile information and information about an object that the user wants to place in the preset template UI from the user. The processor 120 may generate an avatar of the user based on the profile information. In addition, the processor 120 may generate a virtual object based on the information about the object and arrange the virtual object on the template UI. The processor 120 may generate a UI for the user by arranging the avatar and the virtual object on the template UI.

The processor 120 according to an embodiment may set the allocated UI such that at least one advertisement is displayed on one surface of the UI assigned to the user.

The processor 120 according to an exemplary embodiment may generate the user's health information based on the biometric information obtained through the sensing unit 110. The processor 120 may generate at least one of medical information and recommended sports information for the user based on the generated health information.

According to an embodiment, the processor 120 may search for information about an object as a search request for the object is received through the displayed search box. The processor 120 may display at least one advertisement on the VR space through the output unit 130 while searching for information about the object. In addition, as the search is completed, the processor 120 may stop displaying the at least one advertisement.

The processor 120 may select at least one advertisement from among a plurality of advertisements based on a profile of the user and history information about at least one object previously requested to be searched.

The processor 120 according to an embodiment may move an image constituting at least one advertisement in a predetermined direction. For example, the processor 120 changes the speed and the position at which the image projected through the lens having the preset angle of view is output through the output unit 130, thereby shifting the image constituting the at least one advertisement in the preset direction. You can move it. The moving speed of the image may be faster as the end time of the search for the object gets closer.

 According to an embodiment, the processor 120 may determine a place where a sporting event proceeds as a content request is received. In addition, the processor 120 may determine a location in a place where a sporting event corresponding to the user's gesture is performed.

The processor 120 according to an exemplary embodiment may generate history information of at least one content watched by a user before providing content related to a sporting event.

The processor 120 according to an embodiment may select a user who uses content related to a sporting event from among a plurality of other users connected through a user's SNS account.

The processor 120 according to an embodiment may generate an avatar image representing the user based on the profile information of the user.

The processor 120 according to an embodiment may select one of items related to at least one of goods and services based on a user input. In addition, the processor 120 may input identification information of the user corresponding to at least one payment means that can be provided by the device 100 to avatar attribute information on the VR space created based on the profile of the user.

The output unit 130 is for outputting an audio signal, an image signal, or a vibration signal, and may include a display unit 131, a sound output unit 132, and the like.

The display unit 131 displays and outputs information processed by the device 100. For example, the display unit 131 may display a UI for the user generated by the processor 120. In addition, the display unit 131 may display at least one category of objects that can be arranged in the template UI, and may display a plurality of objects included in the selected category as the user selects any one of the at least one category. .

In addition, as the information about the object is received, the display unit 131 may display payment information necessary for purchasing the object.

The display unit 131 may display information on a product of the selected advertisement in the VR space when a selection input for selecting at least one advertisement displayed by the user is received while searching for information about the object.

In addition, when the search for the object is completed, the display unit 131 may display the search result and the information about the product of the selected advertisement together in the VR space.

The display unit 131 according to an exemplary embodiment may display a live action image and a virtual image of a specific location at a place where a sporting event is performed according to a gesture of a user.

In addition, the display unit 131 may display a combination of at least one image photographed at a different point of time from the first viewpoint and the image photographed at the first viewpoint at a specific position.

When the purchase of the object is completed, the display unit 131 according to an embodiment may display a virtual image of the object in the VR space.

According to an exemplary embodiment, when any one of at least one content watched by a user is selected before provision of content related to a sporting event, the display unit 131 may display history information of the selected content on the VR space. . In addition, when the user selects a location included in the displayed history information, the display unit 131 may display a live image and a virtual image of the selected location on the VR space.

The display unit 131 according to an exemplary embodiment may display a virtual image representing another user on the VR space as the acceptance message for the invitation message transmitted to the device of the selected user is received. According to another example, the display unit 131 may combine the avatar image of the user together with the live image and the virtual image of the place where the sporting event is performed, and display the avatar image on the VR space.

According to another embodiment, when the payment of the selected item is completed, the display unit 131 may display the payment completion information on the user interface. The display unit 131 may display payment information about a cost required to access a target place selected based on a user's request among a plurality of places. In addition, the display unit 131 may display the avatar generated based on the user's profile in the target space when payment of the cost required to access the target space is completed.

As the payment is completed, the display unit 131 may display a virtual image related to the selected item on the user interface of the VR space.

Meanwhile, when the display unit 131 and the touch pad form a layered structure and constitute a touch screen, the display unit 131 may be used as an input device in addition to the output device.

The sound output unit 132 outputs audio data received from the communication unit 160 or stored in the memory 170. In addition, the sound output unit 132 outputs a sound signal related to a function (for example, a call signal reception sound, a message reception sound, and a notification sound) performed in the device 100. The sound output unit 132 may include a speaker, a buzzer, and the like.

The A / V input unit 140 is for inputting an audio signal or a video signal, and may include a camera (not shown) and a microphone (not shown). The camera may obtain an image frame such as a still image or a moving image through an image sensor in a video call mode or a shooting mode. The image captured by the image sensor may be processed by the processor 120 or a separate image processor (not shown).

The image frame processed by the camera may be stored in the memory 170 or transmitted to the outside through the communication unit 160. Two or more cameras may be provided according to the configuration aspect of the device 100.

The microphone receives an external sound signal and processes it into electrical voice data.

Meanwhile, the device 100 according to an embodiment may further include a lens (not shown). The user of the device 100 may detect an image output from the display 931 through a lens.

The user input unit 150 means a means for a user to input data for controlling the device 100.

The user input unit 150 may receive a user input. In addition, the user input unit 150 may be linked with the UI module 171 to receive a user input for selecting at least one of items displayed on the sensing area of each of the plurality of sensors. However, this is only an example, and the type of the user input received by the user input unit 150 is not limited to the above-described example.

The communication unit 160 may include one or more components that allow communication between the device 100 and an external device (eg, an HMD). For example, the communicator 160 may include a short range communicator 161, a mobile communicator 162, and a broadcast receiver 163.

The mobile communication unit 162 transmits and receives a radio signal with at least one of a base station, an external terminal, and a server on a mobile communication network. Here, the wireless signal may include various types of data according to transmission and reception of a voice call signal, a video call call signal, or a text / multimedia message.

The broadcast receiving unit 163 receives a broadcast signal and / or broadcast related information from the outside through a broadcast channel. The broadcast channel may include a satellite channel and a terrestrial channel. According to an implementation example, the device 100 may not include the broadcast receiver 163.

The memory 170 may store a program for processing and controlling the processor 120, and may store input / output data.

Programs stored in the memory 170 may be classified into a plurality of modules according to their functions. For example, the programs stored in the memory 170 may be classified into the UI module 171 and the touch screen module 172.

The UI module 171 may provide a specialized UI, GUI, and the like, which are linked to the device 100 for each application. The touch screen module 172 may detect a touch gesture on the user's touch screen and transmit information about the touch gesture to the processor 120.

34 is a block diagram of a service providing server 3400, according to an exemplary embodiment.

As illustrated in FIG. 34, the service providing server 3400 according to an embodiment may include a communication unit 3410, a processor 3420, and a memory 3430. However, not all illustrated components are essential components. The service providing server 3400 may be implemented by more components than the illustrated component, and the service providing server 3400 may be implemented by fewer components.

The service providing server 3400 may perform the above-described operations with reference to FIGS. 25 to 31.

The communicator 3410 may provide an item regarding at least one of goods and services on a user interface of a VR space provided by the device 100. In addition, the communicator 3410 may transmit, to the device 100, an item regarding at least one indicating information on a service available in a VR space and a product to be purchased.

The communicator 3410 may receive a user input from the device 100 that selects any one of items provided for at least one. In addition, the communication unit 3410 may provide the device 100 with information about a cost to be paid for using a product or a service corresponding to the item selected based on the user input.

The communication unit 3410 may receive identification information related to a payment means and payment information of an item from the device 100. In addition, the communication unit 3410 may transmit identification information and payment information to the payment server 2520 that performs the payment of the selected product or service. However, this is only an example, and when the processor 3420 may make a payment, the payment process may be directly performed without having a separate payment server 2520.

The processor 3420 typically controls the overall operation of the service providing server 3400. For example, the processor 3420 may control a process related to payment of a service or a product provided through a VR space of the service providing server 3400 by executing programs stored in the memory 34230. have.

The processor 3420 may authorize the device 100 to use the product or the service as the payment for the product or the service is completed.

The memory 3430 may store a program for processing and controlling the processor 3420, and may store input / output data.

The memory 3430 according to an embodiment may store information about at least one of a service and a product for providing through a user interface in a VR space. For example, the memory 3430 may store information about at least one item of a service and a product.

In addition, the memory 3430 may store identification information and payment information of a user of the device 100.

The device according to the invention comprises a processor, a memory for storing and executing program data, a permanent storage such as a disk drive, a communication port for communicating with an external device, a user interface such as a touch panel, a key, a button and the like. Device and the like. Methods implemented by software modules or algorithms may be stored on a computer readable recording medium as computer readable codes or program instructions executable on the processor. The computer-readable recording medium may be a magnetic storage medium (eg, read-only memory (ROM), random-access memory (RAM), floppy disk, hard disk, etc.) and an optical reading medium (eg, CD-ROM). ) And DVD (Digital Versatile Disc). The computer readable recording medium can be distributed over network coupled computer systems so that the computer readable code is stored and executed in a distributed fashion. The medium is readable by the computer, stored in the memory, and can be executed by the processor.

Claims (15)

In the method for providing a user interface (UI) in a virtual reality (VR) space, Receiving a content request for a sporting event from a user; Acquiring a live-action image and a virtual image of a place where the sporting event proceeds as the content request is received; And Displaying, according to the user's gesture, a live action image and a virtual image of a specific location at a place where the sporting event is held. The method of claim 1, Determining a position in a place where the sporting event is performed corresponding to the gesture of the user; And Receiving a live image and a virtual image for the determined location. The image of claim 1, wherein the live-action image is An image captured by each of a plurality of viewpoints of the sporting event, The displaying step, And displaying at least one image photographed at a different point in time than the first view point on the image photographed at the first view point at the specific position. The method of claim 1, Transmitting purchase request and payment information about an object displayed on at least one of the actual image and the virtual image of the specific location; And If the purchase of the object is completed, displaying the virtual image of the object on the VR space. The method of claim 1, Generating history information of at least one content watched by the user prior to providing the content relating to the sporting event; If one of the at least one content is selected, displaying history information of the selected content on the VR space; And And if the user selects a location included in the displayed history information, displaying a photorealistic image and a virtual image of the selected location on the VR space. The method of claim 1, Selecting a user who uses content related to the sporting event among a plurality of other users connected through the SNS account of the user; Transmitting an invitation message to the VR space to the device of the selected user; And Displaying a virtual image representing the other user on the VR space as an acceptance message for the invitation message is received. The method of claim 1, Receiving profile information of the user; And Generating an avatar image representing the user based on the received profile information, The displaying step, And displaying the avatar image in combination with the live image and the virtual image in the VR space. The method of claim 1, Generating a guide using the neural network to provide information about the sporting event to the user based on the profile information and the preference information of the user, The displaying step, And displaying the generated guide together with the live image and the virtual image. In a device that provides a UI in the VR space, A sensing unit for receiving a content request for a sports event from the user; A processor for determining a place where the sporting event proceeds as the content request is received; A communication unit for obtaining a live-action image and a virtual image of the determined place; And And an output unit configured to display a live image and a virtual image of a specific location at a place where the sporting event is performed according to the gesture of the user. The processor of claim 9, wherein the processor comprises: Determine a location in a place where the sporting event corresponding to the user's gesture is performed, The communication unit, Device for receiving a live image and a virtual image for the determined position. 10. The method of claim 9, wherein the live-action image, An image captured by each of a plurality of viewpoints of the sporting event, The output unit, And displaying at least one image photographed at a different point in time from the image captured at the first view point at the specific position. The method of claim 9, wherein the communication unit, Transmit purchase request and payment information about an object displayed on at least one of the actual image and the virtual image of the specific location; The output unit, And when the purchase of the object is completed, displaying a virtual image of the object on the VR space. The processor of claim 9, wherein the processor comprises: Generating history information of at least one content watched by the user prior to providing the content related to the sporting event, The output unit, When any one of the at least one content is selected, when the history information of the selected content is displayed on the VR space, and when the user selects a location included in the displayed history information, due diligence regarding the selected location A device for displaying an image and a virtual image on the VR space. The processor of claim 9, wherein the processor comprises: Selecting a user who uses content related to the sports event from among a plurality of other users connected through the SNS (Social Network Service) account of the user, The communication unit, Send an invitation message to the VR space to the device of the selected user; The output unit, And displaying a virtual image representing the other user on the VR space as an acceptance message for the invitation message is received. The method of claim 9, wherein the processor, Generating a guide using the neural network to provide the user with information on the sporting event based on the profile information and the preference information of the user, The output unit, And displaying the generated guide together with the live image and the virtual image.

Images