JP2021163479A - Augmented reality system - Google Patents

Abstract

To provide an advertisement in an augmented reality space.SOLUTION: An augmented reality system includes an object management section for disposing a content object in an augmented reality space, an advertisement management section for disposing an advertisement object displaying an advertisement in the augmented reality space, an image generation section for generating an image including the content object and advertisement object, and a posting section for posting an image on a social network.SELECTED DRAWING: Figure 1

Description

The present invention relates to an augmented reality system.

Guide information in an augmented reality form is provided (see Patent Document 1).

Japanese Patent No. 6100496

Advertisements cannot be displayed when using augmented reality guide information.

The present invention has been made in view of such a background, and an object of the present invention is to provide a technique capable of providing an advertisement in an augmented reality space.

The main invention of the present invention for solving the above problems is an augmented reality system, in which an object management unit that arranges a content object in an augmented reality space and an advertisement that arranges an advertisement object that displays an advertisement in the augmented reality space are arranged. It is characterized by including a management unit, an image generation unit that generates an image including the content object and the advertisement object, and a posting unit that posts the image to a social network.

Other problems disclosed in the present application and solutions thereof will be clarified in the columns and drawings of the embodiments of the invention.

According to the present invention, it is possible to provide an advertisement in an augmented reality space.

It is an overall block diagram of the augmented reality system 1 which concerns on this embodiment. It is a system block diagram of the augmented reality system 1 which concerns on this embodiment. It is a figure which shows the flow of providing the augmented reality content in the augmented reality system of this embodiment. It is a figure explaining the display of augmented reality content. It is a figure explaining the flow of the posting process to SNS. It is a figure explaining the posting of augmented reality content to SNS. It is a figure which shows the configuration example of the augmented reality system which concerns on 2nd Embodiment. It is a figure explaining the operation of the augmented reality system which concerns on 2nd Embodiment.

<Outline of the invention>
The contents of the embodiments of the present invention will be described in a list. The present invention includes, for example, the following configuration.
[Item 1]
An object management unit that places content objects in augmented reality space,
An advertisement management unit that arranges an advertisement object that displays an advertisement in the augmented reality space,
An image generation unit that generates an image including the content object and the advertisement object,
The posting section that posts the image to the social network,
An augmented reality system characterized by being equipped with.
[Item 2]
The augmented reality system described in item 1
The content object is a character
The image generation unit synthesizes the image including the character and the advertisement object with the image in the real space taken by the user.
An augmented reality system featuring.
[Item 3]
The augmented reality system described in item 1
To manage the right information indicating that the user has the right to post the image including the content and the advertisement object in the blockchain network.
An augmented reality system featuring.

A specific example of the augmented reality system 1 according to the embodiment of the present invention will be described below with reference to the drawings. It should be noted that the present invention is not limited to these examples, and is indicated by the scope of claims, and is intended to include all modifications within the meaning and scope equivalent to the scope of claims. In the following description, the same elements will be designated by the same reference numerals in the description of the drawings, and duplicate description will be omitted.

<First Embodiment>
FIG. 1 is an overall configuration diagram of the augmented reality system 1 according to the present embodiment. FIG. 2 is a system configuration diagram of the augmented reality system 1 according to the present embodiment. The augmented reality system 1 includes a server 2, and the server 2 is bidirectionally connected to the user terminal 3 via the Internet 4. The SNS server 5 is also communicably connected to the Internet 4.

The augmented reality system 1 of the present embodiment is intended to provide content (hereinafter, referred to as augmented reality content) for displaying as so-called augmented reality (AR).

The server 2 is a computer that provides augmented reality content to the user terminal 3. The server 2 may be a general-purpose computer such as a workstation or a personal computer, or may be logically realized by cloud computing.

The server 2 is also connected to the blockchain network 6. The blockchain network 6 is a distributed ledger system constructed by a plurality of node computers. The blockchain network 6 makes it possible to manage the ledger with difficulty in tampering.

The user terminal 3 is a computer operated by a user who wants to browse augmented reality content. The user terminal 3 can be an electronic terminal such as a smartphone, a tablet or a laptop computer.

The SNS server 5 is a computer that provides a social network service, and can, for example, accept posts of texts and images from users and share them with other users. The server 2 and the user terminal 3 can communicate with each other with the SNS server 5, and the user can post a text, an image, or the like to the SOS server 5 via, for example, the user terminal 3.

<Server 2>
The server 2 mainly includes a server communication unit 21, a storage unit 22, a providing unit 23, and a rights management unit 24. The communication unit 21, the provision unit 23, and the rights management unit 24 can be realized, for example, by the CPU included in the server 2 executing a program stored in the storage. Further, the storage unit 22 is realized as a part of the storage area provided by the storage provided by the server 2. The storage can be, for example, a hard disk, a solid state disk, a flash memory, a RAM, a ROM, or the like.

The communication unit 21 communicates with an external device such as the customer's user terminal 3. The communication unit 21 can be, for example, a wireless communication means.

The storage unit 22 stores a database, a program, or the like in which various information is stored. For example, the storage unit 22 includes an advertisement DB 22a or the like in which advertisement information used in augmented reality is stored. Advertisements used in augmented reality can include advertisements for goods and services. Further, as a form of the advertisement, visual information including a still image or a moving image, audio information, odor information, taste information, tactile information and the like are included. The advertisement information related to the visual information may be a still image or a moving image for displaying the advertisement, or may be text data. The advertisement information related to the voice information may be, for example, voice data encoded by a format such as PCM or MP3, or text data for generating voice information by synthetic voice. In addition, the advertisement information includes advertisement attribute information directly related to the advertisement, advertisement-related information indirectly related to the advertisement, and the like. In the advertisement attribute information, in the case of product advertisement, for example, information such as product type, shape, size, feature, price, mode, use, effect, etc. is included, and in the case of service service advertisement, for example, service etc. Information such as type, feature, price, mode, effect, etc. is included. The advertisement-related information includes, for example, information such as a place where the product is sold in the case of a product advertisement, and information such as a place where the service can be received in the case of a service service.

The providing unit 23 has a function of extracting the advertisement information from the advertisement DB 22a and transmitting it to the user terminal 3. In addition, the providing unit 23 can acquire various information from the user terminal 3 and has a function of extracting advertisements related to the acquired information.

The rights management unit 24 manages the conditions under which the user is allowed to post the augmented reality content to the social network (SNS). In the present embodiment, it is assumed that the conditions under which the user of the augmented reality content including the object is allowed to post to the SNS are managed in the distributed ledger of the blockchain network 6 for each object. In the present embodiment, it is assumed that an object such as a character is also allowed to be posted on the SNS on the condition that one or more advertising objects are included, but other conditions may be registered.

<User terminal 3>
The user terminal 3 mainly includes a communication unit 31, a camera 32, an input unit 33, a display unit 34, an object storage unit 35a, an object management unit 35b, an advertisement receiving unit 36, an advertisement management unit 37, and the like. It includes an information output unit 38 and a position specifying unit 39. The advertisement management unit 36, the advertisement response unit 37, the information output unit 38, and the position identification unit 39 can be realized by the CPU included in the user terminal 3 reading and executing a program stored in the storage. .. The object storage unit 35a can be realized, for example, as a part of the storage area provided by the storage. The storage can be, for example, a hard disk, a solid state disk, a flash memory, a RAM, a ROM, or the like.

The communication unit 31 is a portion capable of communicating with an external device, and is, for example, a wireless communication means.

The camera 32 captures the space in the real world. The camera 32 is a part that performs a shooting operation or the like, and has an image pickup element such as a CCD or CMOS element that converts an optical image into an electric signal, and a shutter. Moreover, not only still images but also moving images can be taken.

The input unit 33 can input via a user operation, such as a touch panel or a keyboard. Further, the input unit 33 may be input not only by hand but also by voice, for example.

The display unit 34 is, for example, a liquid crystal monitor and can display various types of information. Further, the display unit 34 may be integrated with the input unit 33.

The object storage unit 35a stores, for example, model information necessary for generating a virtual object with a motif such as a virtual character or daily necessities. The model information can be, for example, data in a format such as USDZ or glTF. Further, the model information includes, for example, attribute information directly related to the virtual object, advertisement attribute information, and advertisement correspondence information corresponding to the advertisement-related information. The attribute information includes, for example, information such as appearance, type, mode, and component configuration.

The object management unit 35b creates augmented reality content. Specifically, the object management unit 35b creates an augmented reality image to be superimposed and displayed on an image in the real space (an image taken by the camera 32). The object management unit 35b can create a virtual object based on the model information stored in the object storage unit 35a and arrange it in the augmented reality space. The object management unit 35b can also arrange the advertisement object created by the advertisement management unit 36, which will be described later, in the augmented reality space.

The advertisement management unit 37 applies the received advertisement information to an advertisement object suitable for the advertisement information. The advertisement object is a virtual object created by the object management unit 35b. The advertising object may be a virtual object that originally displays content on its surface, such as a picture frame or television, or around a virtual object that is the main content such as a character, such as a wall, desk, or door. It may be a virtual object representing an article to be distributed, or it may be another virtual object such as a T-shirt or a cup that is possessed by a virtual object that is the main content such as a character. The advertisement management unit 37 can display the advertisement on the appearance of the advertisement object by, for example, pasting the advertisement information of the still image or the moving image as a texture on the surface of the advertisement object. Further, for example, when the advertisement information is voice information, the voice information can be attached to the advertisement object so that the voice information can be reproduced when the advertisement object is reproduced. For example, when the advertisement management unit 37 receives the advertisement information regarding the new clothes, the advertisement management unit 37 can have the new clothes design attached to the surface of the advertisement object having the clothes as a motif. The advertisement management unit 37 can attach the advertisement information to the advertisement object of the attribute corresponding to the attribute information of the advertisement information. For example, when the advertisement management unit 37 receives the advertisement information of a new song, since the advertisement attribute information is music, it can be attached to an advertisement object having a radio case as a motif, which includes the advertisement correspondence information corresponding to the music. ..

The information output unit 38 superimposes the augmented reality image created by the object management unit 35b on the image of the camera 32 that captures the real space and outputs the image. The information output unit 38 can also output visual information such as moving images, audio information, odor information, tactile information, taste information, and the like. The information output unit can output visual information such as a video via the display unit 34.

The position specifying unit 39 has a function of acquiring the position information of the user terminal 3 by, for example, a beacon for positioning using satellite positioning such as GPS or short-range wireless technology. In addition, the position specifying unit 39 can also transmit the acquired position information to the server 2.

<Flowchart>
FIG. 3 is a diagram showing a flow of providing augmented reality content in the augmented reality system of the present embodiment. FIG. 4 is a diagram illustrating the display of augmented reality content.

First, as step S1, the user activates the augmented reality mode on the user terminal 3. In the augmented reality mode, the camera 32 (shooting unit) of the user terminal 3 is activated, and the surrounding real space image P (information in the real space) taken by the camera 32 and the object management unit 35b are generated. A virtual drink D (virtual object) with a virtual character O and a drink as a motif is displayed on the display unit 34.

Next, in step S2, the user terminal 3 transmits the location information to the server 2.

Next, in step S3, the providing unit 23 of the server 2 extracts the advertisement related to the location information from the advertisement DB 22a and transmits it to the user terminal 3. In the present embodiment, an advertisement related to a drink sold at a cafe within a predetermined range is extracted from the position of the user terminal 3 based on the received position information.

Next, in step S4, the advertisement management unit 37 causes the received advertisement to be associated with the appropriate virtual drink D to generate the advertisement object C.

Next, in step S5, the object management unit 35b controls the virtual character O to react to the advertisement object C. In the present embodiment, the virtual character O raises the advertisement object C, but the present invention is not limited to this. For example, the virtual character may apply an external force to the advertising object such as touching or hitting the advertising object. Virtual characters may also refer to advertising objects. For example, when an advertisement is received or an advertisement object is generated, the virtual character may output a voice to that effect. Further, the virtual character may be used depending on its purpose. For example, if the advertising object is a drink, the virtual character can make a drinking gesture on the advertising object.

Next, in step S6, the user displays and reproduces the content of the advertisement.

As described above, the augmented reality system 1 in the present embodiment can output an advertisement together with the augmented reality content to the user.

In the augmented reality system of the present embodiment, the appearance of the augmented reality content displayed on the user terminal 3 can be posted on the social network (SNS). FIG. 5 is a diagram illustrating a flow of posting processing to SNS.

When the user requests the user terminal 3 to post to the SNS, the user terminal 3 transmits the request to post to the SNS to the server 2 (S41). The server 2 confirms whether or not the user has the right to post augmented reality content to the SNS upon request. In the present embodiment, the rights related to the augmented reality content of the user are manually managed in the blockchain network 6, and the server 2 refers to the distributed ledger managed by the blockchain network 6 and whether or not the user is allowed to post. Can be determined (S42). Here, the server 2 can determine, for example, whether or not to post to the SNS according to the characters and the advertisement included in the augmented reality content. The server 2 can disallow posting to the SNS, for example, when the augmented reality content does not include the advertising content. In addition, the object included in the augmented reality content may be transmitted to the server 2 in response to the request from the user terminal 3, or the snapshot image of the augmented reality content is transmitted from the user terminal 3 to the server 2. , The server 2 may analyze the snapshot image to determine whether or not the advertisement is displayed.

The server 2 responds to the determination result to the user terminal 3 (S43), and when posting is permitted, the user terminal 3 can transmit the augmented reality content to the SNS server 5 to post (S43). S44). The user terminal 3 can post, for example, a snapshot image of augmented reality content to the SNS. In addition, the user terminal 3 may accept input of text or the like and attach it to the posting. The SNS server 5 performs a process of sharing the posted content with other users (S45).

As described above, according to the augmented reality system 1 of the present embodiment, the user can legally post the augmented reality content viewed on the user terminal 3 to the SNS. In the present embodiment, when a predetermined condition is satisfied, such as when the augmented reality content includes an advertisement according to the condition managed by the blockchain network 6, the augmented reality content is sent to the SNS. Posting is also allowed. Here, the advertising fee from the advertiser can be returned to the provider of the object.
FIG. 6 is a diagram illustrating posting of augmented reality content to SNS. From the user terminal 3, all or part of the augmented reality content displayed on the display unit 34 is snapshotted, and the snapshot image 35 is posted to the SNS. Here, the user can also point the camera 32 of the user terminal 3 toward himself / herself so that the user U is reflected in the snapshot image 35 together with the character O. As a result, it is possible to virtually post a photo-like image of the character O and the user U to the SNS. At this time, the text 36 may be posted. The snapshot image 35 also includes a drink D, which is an advertising object, which makes it possible to spread the advertisement to the SNS. In consideration of this advertisement, the user is given the right to post on the SNS, and the provider of the character O can be returned money.

<Second embodiment>
FIG. 7 is a diagram showing a configuration example of the augmented reality system according to the second embodiment. In the second embodiment, the user terminal 3 omits the advertisement receiving unit 36 and the advertisement management unit 37. Further, the storage unit 22 of the server 2 includes an object DB 22b in addition to the advertisement DB 22a.

The object DB 22b stores model information (hereinafter, also simply referred to as a virtual object) of a virtual object arranged in the augmented reality space. The model information can be, for example, data in a format such as USDZ or glTF.

In the second embodiment, the providing unit 23 provides augmented reality content. The providing unit 23 can provide augmented reality content in response to a request from the user terminal 3, for example. In the request, a virtual object (hereinafter referred to as a content object) that the user wants to view, such as a character, is specified. The providing unit 23 reads the content object specified in the request from the storage unit 22, and responds to the user terminal 3 with augmented reality content including the content object and scene information for arranging the content object in the augmented reality space. .. At this time, the providing unit 23 can respond to the user terminal 3 by including a virtual object for displaying the advertisement (hereinafter, referred to as an advertisement object) in the augmented reality content. The providing unit 23 can paste an advertising image as a texture of an advertising object, for example.

FIG. 8 is a diagram illustrating the operation of the augmented reality system according to the second embodiment.

In step S2, the user terminal 3 transmits a request for augmented reality content to the server 2. The request includes the specification of a virtual object such as a character that the user wants to view.

In step S3, the server 2 creates augmented reality content in response to the request. The server 2 can create augmented reality content by reading model information corresponding to a virtual object (hereinafter referred to as a content object) such as a character specified in a request from the object DB 22b, for example. The server 2 can also generate an object such as a background set or an accessory that is not specified by the user as an advertisement object, attach the advertisement information to the texture of the advertisement object, and then include it in the augmented reality content. Further, the server 2 can include scene information in which the arrangement position and movement of the content object and the advertisement object are specified in the augmented reality content.

In step S4, the user terminal 3 receives the augmented reality content and can create an augmented reality scene based on the received augmented reality content.

As described above, according to the augmented reality system according to the second embodiment, the advertisement object can be provided in association with the content object that the user wants to browse. Therefore, the advertisement can be placed in the same augmented reality space as the content object that the user is interested in.

Also in the second embodiment, the augmented reality content can be posted to the SNS on the user terminal 3. In this case, in the second embodiment, the snapshot image may be created on the server 2.

Although the description of the embodiment of the present invention is completed above, the aspect of the present invention is not limited to this embodiment.

<Modification example>
In the above embodiment, the user reproduces the advertisement by operating the input unit 33, but the present invention is not limited to this. For example, the advertisement management unit 36 has a function of detecting keywords from voice information and text information, and when the advertisement management unit 36 detects keywords related to advertisement information during a voice or text conversation between a virtual character and a user, The advertisement may be provided with a display / playback function.

Further, in the above embodiment, the advertisement information and the advertisement object are associated with each other, but the present invention is not limited to this. For example, the object management unit 35 of the user terminal 3 may have a function of directly converting advertisement information into an advertisement object (virtual object). For example, even if the object management unit creates an advertisement object containing the same, similar or corresponding attribute information as the advertisement attribute information from the object storage unit based on the advertisement attribute information and the advertisement-related information of the received advertisement information. good.

Further, in the augmented reality mode in the above embodiment, the surrounding real space image P (information in the real space) captured by the camera 32 and the virtual character O (virtual object) generated by the object management unit 35b are used. Is displayed on the display unit 34, but the present invention is not limited to this. For example, an augmented reality draft service that recognizes the surrounding real space (buildings, roads, objects, etc.) taken by the camera based on the location information of the user terminal and the direction in which the user terminal is facing (for example, Google map). It may be a service (Live View).

Further, in the above embodiment, the user terminal 3 receives the advertisement when the augmented reality mode is activated, but the present invention is not limited to this. For example, the user terminal 3 may automatically receive (update) an advertisement related to the surrounding environment when the surrounding environment changes.

For example, the user terminal 3 may be provided with an advertisement updating unit (not shown) that updates the advertisement according to a change in the location of the user. Specifically, the advertisement update unit has a function of detecting when the user moves or the like and the location changes, and causing the advertisement management unit 36 to acquire a new advertisement.

In the above embodiment, the advertisement information is displayed as a video, but the advertisement information is not limited to this and may be audio information.

Further, the advertisement in the above embodiment and the environment in the real space may be odor information. For example, the information output unit 38 of the user terminal 3 contains at least one odor element, and the information output unit 38 can emit one or more odor elements. The odorant may be a single chemical substance such as furaneol (strawberry scent) or 1-hexanol (forest scent), or a combination thereof. Further, for example, regardless of the composition of the chemical substance, an odor having a scent equivalent to that of jasmine as one unit may be used. That is, for example, the information output unit 38 may output an equivalent odor by combining a plurality of chemical substances using the odor of strawberry milk as an output method, or the odor of strawberry as one unit and the odor of milk. The odor may be output in combination with the odor of one unit.

Further, the advertisement in the above embodiment and the environment in the real space may be taste information. For example, the information output unit 38 of the user terminal 3 contains at least one taste element, and the information output unit 38 can release one or more taste elements. The taste element may be, for example, a single chemical substance such as inosinic acid or glutamic acid, or a combination thereof. Further, for example, regardless of the composition of the chemical substance, a taste equivalent to the strawberry taste may be used as one unit. That is, for example, the information output unit 38 may output the same taste by combining a plurality of chemical substances by using the taste information of strawberry chocolate as an output method, or the taste with the strawberry taste as one unit and the chocolate taste. May be output in combination with a taste having a unit of.

Further, the advertisement in the above embodiment and the environment in the real space may be tactile information.

Further, in the present embodiment, a virtual space based on augmented reality (AR) is assumed, but the present invention is not limited to this, and is limited to a virtual reality (VR) space or a mixed reality (MR) space. Even if there is, the advertising system of this embodiment can be applied as it is.

1 Advertising system 2 Server 3 User terminal 4 Communication network 5 SOS server 6 Blockchain network 21 Server communication unit
22 Storage unit 22a Advertising DB
23 Providing unit 31 Terminal communication unit 32 Camera 33 Input unit 34 Display unit 35a Object storage unit 35b Object management unit 36 Advertisement receiving unit 37 Advertising management unit 38 Information output unit 39 Positioning unit

Claims (3)

An object management unit that places content objects in augmented reality space,
An advertisement management unit that arranges an advertisement object that displays an advertisement in the augmented reality space,
An image generation unit that generates an image including the content object and the advertisement object,
The posting section that posts the image to the social network,
An augmented reality system characterized by being equipped with. The augmented reality system according to claim 1.
The content object is a character
The image generation unit synthesizes the image including the character and the advertisement object with the image in the real space taken by the user.
An augmented reality system featuring. The augmented reality system according to claim 1.
To manage the right information indicating that the user has the right to post the image including the content and the advertisement object in the blockchain network.
An augmented reality system featuring.

Images