JP2015011360A - Information processing device and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a technique that can further improve a content guide in a device for presenting guidance information about a content to a user.SOLUTION: An information processing device 100 displays a content guide on a display in accordance with an input operation of a user. The content guide includes incidental information such as a map related to an image, voice, a moving image, and a work. A content displaying person installs an AR marker 300 together with each work, which is displayed. The AR marker 300 is commonly installed for each work. The information processing device 100 identifies a content to be displayed on the display in response to an input operation of the user, and reproduces voice data for guiding the identified content. The information processing device 100 receives an operation for starting AR processing from the user, and when detecting the AR marker 300 from captured image data, displays AR data on the display.

Description

  The present invention relates to a technique for presenting guidance information related to content to a user.

  In order for people to better understand various contents such as works of art, animals and plants, tourism resources, etc., guidance information describing explanations of the contents may be provided. Also, in sales operations such as retail stores, guidance information that describes what kind of goods or services are provided to consumers may be provided.

  For example, exhibitors at museums, museums, etc. hold exhibitions that display contents such as works of art, set entrance fees for visitors, and earn income from visitors. Exhibitors help visitors deepen their understanding of content by providing visitors with explanations of content. Exhibitors at art museums, museums, etc., for example, have installed a guide board describing the explanation of the work near the object of appreciation such as a work of art. This provides visitors with an explanation of the content.

  For example, operators such as zoos and botanical gardens have installed information boards and the like that explain the breeding status of animals and habitats, as well as animals and plants that are objects of appreciation. In addition, managers who manage sightseeing spots distribute tourist information via the Internet, etc., and install tourist signs on local tourism resources to provide tourists with tourism information. Provides information including commentary on the history of resources. Thereby, since a visitor can refer to the commentary about contents, such as an appreciation object and sightseeing resources, a visitor can know the background knowledge about a content, and how to enjoy a content. For exhibitors and the like, it is possible to support visitors to understand the object of appreciation and enjoy the appreciation, and to increase the number of visitors who repeatedly visit.

  Information for guidance regarding content may be provided by voice. For example, an exhibitor provides a device (for example, a voice guide device) for assisting a visitor to deepen understanding of content. For example, an exhibition number is assigned in advance to each work to be exhibited. The voice guide device provides a visitor with voice information such as a work commentary by receiving an input of a work number and an operation for reproducing a voice commentary from the visitor.

  Various techniques relating to a guide device for assisting a visitor to deepen their understanding of content are known. For example, Japanese Patent Application Laid-Open No. 11-143419 (Patent Document 1) discloses an audio guide device for explaining an exhibit such as a museum. According to the technique of Patent Document 1, a visitor can listen to an explanation that the visitor wants to listen to with a simple operation. Japanese Patent Laying-Open No. 2006-145936 (Patent Document 2) discloses a voice guide system for improving the service to viewers by improving the efficiency and operativity of holding preparation work such as exhibitions. Disclosure. More specifically, Japanese Patent Application Laid-Open No. 2004-228561 uses an exhibit identification device that is attached near an exhibit to identify an exhibit such as a painting, and a viewer who wants to hear a description of the work by voice. An audio guide device and a management device for remotely registering an identification code of an exhibit in a backyard are disclosed. According to the technique of Patent Document 2, it is possible to remotely manage content in an art museum or a museum, and the operability of a voice guide used for commentary on a work is also improved.

JP-A-11-143419 JP 2006-145936 A

  The voice guide device provides the visitor with information explaining the content corresponding to the content selected by the visitor. However, in order for visitors who are interested in content to understand and enjoy the exhibition objects and the like, it is desirable that the means for providing guides to visitors are more diverse.

  Therefore, there is a need for technology to further enhance content guides for visitors.

  According to one embodiment, an information processing apparatus for displaying guidance information is provided. The information processing apparatus includes an operation receiving unit for receiving an input operation, a camera for obtaining captured image data by shooting, a display unit, a reference image indicating a specific subject, and identification information for a plurality of contents A storage unit for storing content data corresponding to the identification information, and a control unit for controlling the operation of the information processing device based on an input operation. By performing image recognition processing based on the image, a detection processing unit that performs processing for detecting a specific subject from the captured image data and a camera are driven, and the specific processing unit detects the specific subject from the captured image data. A display processing unit that displays content data corresponding to the identification information of the display target content to be displayed on the display unit.

  According to another embodiment, a program for controlling the operation of an information processing apparatus for displaying guidance information is provided. The information processing apparatus includes a processor, a memory, an operation receiving unit for receiving an input operation, a camera for obtaining captured image data by shooting, and a display. The memory stores content data corresponding to the identification information for each of the reference image indicating the specific subject and the identification information of the plurality of contents. The program performs a process for detecting a specific subject from the captured image data by performing an image recognition process on the captured image data based on the reference image, and drives the camera to identify the program. When the subject is detected from the captured image data, the step of causing the display to display content data corresponding to the identification information of the display target content to be displayed on the display is executed.

  According to the above-described embodiment, it is possible to allow a visitor to easily view a content guide not only by voice but also visually. Further, according to the above-described embodiment, for example, when a specific subject is installed near the display object, a visitor can view content data corresponding to the content by photographing the subject with a camera. . Therefore, the information processing apparatus according to the embodiment can prompt the visitor to enter the exhibition hall and appreciate the display object. As a result, the information processing apparatus can contribute to an increase in the number of visitors.

  The above and other objects, features, aspects and advantages of the present invention will become apparent from the following detailed description of the present invention taken in conjunction with the accompanying drawings.

It is a figure which shows an example of the condition where the information processing apparatus 100 concerning embodiment is used. 2 is a block diagram illustrating a configuration of an information processing apparatus 100. FIG. 2 is a diagram illustrating a software configuration of the information processing apparatus 100. FIG. It is a figure which shows the content management table 31. FIG. It is a figure which shows AR (Augmented Reality) marker 300. FIG. It is a figure which shows an example of the user interface for displaying the guide of a content. 3 is a flowchart illustrating processing for displaying AR data by the information processing apparatus 100 according to the first embodiment. It is a figure which shows the example which displays AR data. It is a figure which shows the interface of operation which switches the information incidental to a content, and the information relevant to a content. It is a figure which shows the marker image of Embodiment 2. FIG. It is a figure which shows the content management table 31 of Embodiment 2. FIG. 10 is a flowchart illustrating processing for displaying AR data by the information processing apparatus 100 according to the second embodiment. It is a figure which shows an example of the user interface for accepting input operation which cancels | releases viewing restrictions. It is a figure which shows the example of the path | route for the information processing apparatus 100 to download data, such as a content.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the following description, the same parts are denoted by the same reference numerals. Their names and functions are also the same. Therefore, detailed description thereof will not be repeated.

<Embodiment 1>
FIG. 1 is a diagram illustrating an example of a situation in which the information processing apparatus 100 according to the embodiment is used. The information processing apparatus 100 is a tablet terminal including a touch panel, and displays a content guide on a display. In the first embodiment, the case where the content is a work of art will be described.

<Provision of video guide>
As illustrated in FIG. 1, the information processing apparatus 100 presents guide information including a commentary by video for a display work to a visitor who is a user of the information processing apparatus 100. As a result, the information processing apparatus 100 can provide the user with a video guide having a greater amount of information, compared to a case where a description of only audio is provided. For example, in the information processing apparatus 100, an application for displaying a guide of content that is an exhibition work on the display of the information processing apparatus 100 is installed in advance. An exhibitor such as a museum lends the information processing apparatus 100 to a visitor, so that the visitor can use a content guide by the information processing apparatus 100. The information processing apparatus 100 includes a touch panel display, and accepts user input operations through the touch panel. In addition, the information processing apparatus 100 displays guidance information on a display. A visitor can specify content (art work) that he / she wants to use as a guide by an input operation to the information processing apparatus 100, and can use commentary based on audio data associated with the content or commentary based on video data. it can.

<Installation of AR marker 300 and provision of AR data>
In addition, the information processing apparatus 100 holds an application corresponding to AR (Augmented Reality) technology. The exhibitor installs the AR marker 300 together with the exhibited work. In the first embodiment, the exhibitor installs a common AR marker 300 for each work. When the information processing apparatus 100 receives an input operation for starting an application corresponding to the AR technology from a visitor, the information processing apparatus 100 detects the AR marker 300 photographed as a subject in the photographed image data by image recognition processing, and displays it as an exhibition work. The corresponding AR data is displayed on the display of the information processing apparatus 100. As described above, the visitor can enjoy each content further by designating the content, using the guide by voice or moving image for each content, and viewing the AR data using the AR marker 300. The AR data is video data that is overlaid on the captured image by processing using the AR technology based on the position of the AR marker detected from the captured image by image recognition processing. An example of the AR data is data related to the land, episode, and other contents that are the basis of the landscape as the content.

  In addition, the information processing apparatus 100 receives an input operation for designating content to be displayed on the display from a visitor. The information processing apparatus 100 displays AR data or the like corresponding to the content designated by the visitor on the display. Therefore, the AR marker 300 itself does not need to include information for identifying AR data to be displayed, and an exhibitor can use a common AR marker 300 for each work. It becomes easy to install each work.

  As described above, the exhibitor installs a specific subject such as the AR marker 300 at the place where the content such as the exhibition work is installed. The exhibitor can detect a specific subject such as the AR marker 300 by processing using the AR technology of the information processing apparatus 100, and content (AR data) that can be viewed when the information processing apparatus 100 detects the AR marker 300 Prepare for visitors. As a result, the information processing apparatus 100 can provide the user with usable content by visiting a place where the content is installed and detecting a specific subject such as the AR marker 300, so that the user can display the exhibition place. Can be encouraged to visit.

  In the example of FIG. 1, a router 700 is illustrated as a device that communicates with the information processing apparatus 100. The information processing apparatus 100 may acquire an application for realizing processing for displaying a content guide on the display of the information processing apparatus 100 from an external network via the router 700.

<Configuration>
The configuration of the information processing apparatus 100 will be described with reference to FIGS. 2 and 3. FIG. 2 is a block diagram illustrating a configuration of the information processing apparatus 100. As illustrated in FIG. 2, the information processing apparatus 100 includes antennas 1 and 2, a long-distance communication module 3, a short-distance communication module 4, a touch panel 5, a display 6, a camera 7, and a power control unit. 8, a battery 9, a microphone 10, a speaker 11, an audio processing unit 12, a storage unit 13, and a control unit 14.

  The antenna 1 and the antenna 2 radiate signals emitted from the information processing apparatus 100 as radio waves. The antenna 1 receives radio waves from the space and gives a received signal to the long-distance communication module 3. The antenna 2 receives radio waves from the space and gives a received signal to the near field communication module 4.

  The long-distance communication module 3 and the short-distance communication module 4 perform modulation / demodulation processing for transmitting and receiving signals via the antenna 1 in order for the information processing apparatus 100 to communicate with other wireless devices. The long-distance communication module 3 and the short-distance communication module 4 are communication modules including a tuner, a high-frequency circuit, and the like, perform modulation / demodulation and frequency conversion of radio signals transmitted and received by the information processing apparatus 100, and control received signals. 14 is given.

  The long-distance communication module 3 is a communication module compatible with mobile communication systems such as LTE (Long Term Evolution) and UMTS (Universal Mobile Telecommunications System), and is compatible with relatively long-distance communication.

  The near field communication module 4 is compatible with, for example, near field wireless communication standards such as Bluetooth (registered trademark), wireless LAN (Local Area Network) standards such as IEEE (Institute of Electrical and Electronic Engineers) 802.11, and the like. It is a communication module and supports relatively short distance communication. When the information processing apparatus 100 supports a plurality of communication methods, an antenna and a communication module may be provided corresponding to these communication methods.

  The touch panel 5 is an operation member for receiving a user input operation. As the touch panel 5, for example, a capacitive touch panel is used to accept a user's contact operation. The touch panel 5 outputs an input operation by the user to the control unit 14.

  The display 6 displays icons and background images on the menu screen, and displays still images, moving images, and the like according to the operation of the application. The display 6 is realized by, for example, an LCD (Liquid Crystal Display) or an organic EL (electroluminescence) display. The display 6 displays a content guide designated by the user in response to an input operation.

  The camera 7 is a camera module that receives incident light and converts it into an electrical signal. The camera 7 includes a CMOS (Complementary Metal Oxide Semiconductor) sensor, an ADC (Analog to Digital Converter), and the like, and outputs an image signal to the control unit 14.

  The power control unit 8 controls power supplied to each circuit of the information processing apparatus 100. The power control unit 8 is, for example, a power control IC (Integrated Circuit).

  The battery 9 is a supply source that supplies power for operating each circuit of the information processing apparatus 100. The electric power supplied from the battery 9 is supplied to each circuit according to the control of the power supply control unit 8.

  The microphone 10 receives a voice input by the user and gives a voice signal corresponding to the voice input to the voice processing unit 12.

  The speaker 11 converts the sound signal given from the sound processing unit 12 into sound and outputs the sound to the outside of the information processing apparatus 100.

  The audio processing unit 12 performs modulation / demodulation of the audio signal. The audio processing unit 12 modulates the signal given from the microphone 10 and gives the modulated signal to the control unit 14. In addition, the audio processing unit 12 gives an audio signal to the speaker 11. The voice processing unit 12 is realized by, for example, a voice processing processor.

  The storage unit 13 is configured by a flash memory, for example, and stores data and programs used by the information processing apparatus 100. In one aspect, the storage unit 13 stores a content management table 31, content image data 32, content audio data 33, content moving image data 34, content incidental information 35, and AR data 36.

  As will be described in detail later, the content management table 31 includes identification information for identifying the content (works), the name of the content, an image file indicating the content, and an audio data file and a video data file for providing a guide for the content. And the AR data prepared for each content and the accompanying information (such as map information related to the content) attached to the content.

  The content image data 32 is a group of image data corresponding to each content. The content audio data 33 is an audio data group for guiding each content. The content moving image data 34 is a moving image data group for guiding each content. The content incidental information 35 is an incidental information group incidental to each content. The AR data 36 is AR data prepared for each content.

  The control unit 14 controls the overall operation of the information processing apparatus 100 by reading and executing a control program stored in the storage unit 13. The control unit 14 is, for example, an application processor. The control unit 14 performs the functions of the detection processing unit 40, the video processing unit 41, and the drive control unit 42 by operating according to a program. The detection processing unit 40 causes the information processing apparatus 100 to exhibit the function of image recognition processing, and performs image recognition processing on the captured image data output from the camera 7 to detect a specific subject such as the AR marker 300. To do. The video processing unit 41 controls the display content of the display 6 according to the processing, and displays the content image data 32 and the like on the display 6. The drive control unit 42 performs processing for driving each component (such as the camera 7) of the information processing apparatus 100.

  FIG. 3 is a diagram conceptually illustrating a software configuration of the information processing apparatus 100. As illustrated in FIG. 3, the information processing apparatus 100 executes an OS (Operating System) 51 and software such as an application 52. The OS 51 provides a basic execution environment for the control unit 14 to execute the application 52. The OS 51 is realized by an OS for mobile terminals, for example.

  The application 52 causes the information processing apparatus 100 to exhibit the AR function, detects the AR marker 300 from the captured image, and displays the AR data in an overlay display on the preview of the camera 7 based on the position of the AR marker 300 detected from the captured image. The information processing apparatus 100 is caused to perform processing and processing for providing a content guide or the like to the operator of the information processing apparatus 100 via the display 6. The application 52 includes a user interface display program 53 and an AR processing program 54. The user interface display program 53 is a program for displaying a user interface on the display 6. The AR processing program 54 is a program for causing the information processing apparatus 100 to exhibit the AR function. The captured image acquisition unit 61 drives the camera module (camera 7) and sequentially acquires captured image data. The marker detection unit 62 detects the AR marker 300 by applying an image recognition process to the acquired captured image data. The drawing unit 63 overlay-displays the AR data at the marker position extracted from the captured image data.

<Data structure>
With reference to FIG. 4, data used in the processing of the information processing apparatus 100 will be described. FIG. 4 is a diagram showing the content management table 31.

  As shown in FIG. 4, each record of the content management table 31 includes a work identification number 311, a content name 312, a content image file name 313, an audio data file name 314, a moving image data file name 315, and AR data. 316 and a content-attached data name 317 are included.

  The work identification number 311 is identification information for identifying each content. The content name 312 indicates the name of the content indicated in the identification information. The content image file name 313 is information for specifying the image file of the content indicated by the identification information. The audio data file name 314 is information for specifying the audio data file of the content indicated in the identification information. The moving image data file name 315 is information for specifying the moving image data file of the content indicated by the identification information. The AR data 316 is information for specifying the AR data of the content indicated in the identification information. The content incidental data name 317 is information for specifying incidental information incidental to the content indicated by the identification information. For example, the incidental information is map information indicating a position related to the content (for example, a position of a drawing object drawn in the work or a position where the work is produced).

  The information processing apparatus 100 reproduces each data specified by the content image file name 313, the audio data file name 314, the moving image data file name 315, the AR data 316, or the content incidental data name 317 in accordance with a user input operation. To do.

<AR marker>
The AR marker in the first embodiment will be described with reference to FIG. FIG. 5 is a diagram showing a scene where the AR marker 300 is used. FIG. 5 illustrates a situation in which the user of the information processing apparatus 100 is shooting the AR marker 300 as a subject with a camera. In the present embodiment, a content exhibitor installs an AR marker 300 in the vicinity of each content. The exhibitor installs a common AR marker 300 for each content. The information processing apparatus 100 sequentially acquires captured image data from the camera 7 and displays an image based on the acquired data on the display 6 as a preview image. The information processing apparatus 100 detects the AR marker 300 as a subject from the captured image data by applying an image recognition process each time captured image data is acquired from the camera 7. When detecting the AR marker 300, the information processing apparatus 100 reads the AR data 316 corresponding to the identification information with reference to the content management table 31 according to the content identification information designated by the user. The information processing apparatus 100 displays the read AR data in an overlay manner while displaying the captured image data from the camera 7 on the display 6.

  As shown in FIG. 5, the AR marker 300 has a shape in which the width of the frame line and the like is determined in advance because the information processing apparatus 100 performs the AR function and specifies the shape of the marker by image recognition. . The shape is, for example, a rectangle or a circle, but is not limited thereto.

  The information processing apparatus 100 receives a setting of an image (reference image) showing the AR marker 300. The information processing apparatus 100 reads the setting of the reference image indicating the AR marker 300 and performs a process of detecting the AR marker 300 from the captured image data based on the reference image.

<Example of user interface>
An example of a screen displayed on the display 6 by the information processing apparatus 100 will be described with reference to FIG. FIG. 6 is a diagram illustrating an example of a user interface for displaying a content guide.

  As shown in FIG. 6, the control unit 14 displays a title display area 71, a content designation receiving area 72, a content display area 73, a display switching area 74, a viewing operation receiving area 75, and audio playback on the display 6. A reception area 76, a moving image reproduction reception area 77, and an AR mode reception area 78 are displayed.

  The title display area 71 shows the title of the exhibition contents. The content designation receiving area 72 is an area for displaying a list of identification information for identifying each content and receiving an input operation for designating the content by the user. The information processing apparatus 100 stores the identification information of the content specified by the user in the memory. In the example of FIG. 6, the item 121 of the identification information “27” is designated by the user's input operation, and the item 121 is displayed separately from other contents.

  The display 6 displays a list of identification information in order from the item with the highest identification information in the content designation receiving area 72, and scrolls the identification information to be displayed according to the user's input operation. When receiving an input operation for the item “to top” in the content designation receiving area 72, the display 6 scrolls the display of the content identification information and displays the item of the highest level identification information. Further, when receiving an input operation for the item “to the end” in the content designation receiving area 72, the display 6 scrolls the display of the content identification information and displays the item of the lowest-order identification information. The information processing apparatus 100 accepts, through the touch panel 5, an operation in which the user slides a finger on the screen and an operation in which the finger is touched on the screen. The user scrolls the list of contents displayed in the content designation reception area 72 by touching and sliding the content list displayed in the content designation reception area 72, and the item of the work is displayed. The content to be played can be specified by touching the area.

  The content display area 73 is an area for displaying an image file, a moving image data file, or content supplementary information corresponding to content identification information specified by the user.

  The display switching area 74 is an area for receiving an input operation for switching content to be displayed in the content display area 73 from the user. In response to an input operation on the display switching area 74, the control unit 14 switches the display of the content supplementary information and the display of either the image file or the moving image data file to display the content in the content display area 73.

  The viewing operation reception area 75 indicates the playback status of the audio data file or moving image data file, and is an area for receiving a playback pause operation, an operation for volume adjustment, and other operations. When the control unit 14 accepts an operation to reproduce the audio data file from the user, the control unit 14 displays an image file (content image file name 313) corresponding to the content identification information in the content display area 73, and the audio data file (Audio data file name 314) is reproduced. Further, when the operation of reproducing the moving image data file is received from the user, the control unit 14 displays the video portion of the moving image data in the content display area 73 and reproduces the sound.

  The audio reproduction reception area 76 is an area for receiving an input operation for reproducing or stopping an audio data file from the user. In the example of FIG. 6, since the information processing apparatus 100 is playing back an audio data file, the display in the playback display area 79 indicating that the content is being played is overlaid in the audio playback reception area 76. The display 6 displays that the audio data file is being reproduced.

  The moving image reproduction receiving area 77 is an area for receiving an input operation for reproducing or stopping the moving image data file from the user.

  The AR mode reception area 78 is an area for receiving an operation from a user to start processing (AR processing) for detecting the AR marker 300 from captured image data and displaying the AR data in an overlay manner. When the information processing apparatus 100 receives an input operation to the AR mode reception area 78, the information processing apparatus 100 displays a preview of the captured image data from the camera 7 on the entire screen of the display 6, for example.

  Note that the information processing apparatus 100 inputs to the AR mode reception area 78 together with the AR mode reception area 78 in order to urge the user to display AR data using the AR marker 300 installed in the exhibition hall. A display for prompting the operation may be displayed on the display 6. For example, the information processing apparatus 100 may display character information such as “AR (augmented reality) can be enjoyed by shooting an AR marker with a camera” on the display 6 together with the AR mode reception area 78.

  Note that the information processing apparatus 100 reproduces audio data or the like associated with the content identification information specified by the user when the user receives an operation of specifying the content by touching the screen with a finger. It is good. Further, when the information processing apparatus 100 receives an input operation to the moving image reproduction receiving area 77 and reproduces moving image data, the input operation to the content designation receiving area 72 is received and the content specified by the user is switched. In this case, the information processing apparatus 100 stops the reproduction of the audio data that has already been reproduced or the moving image data that is displayed in the content display area 73, and corresponds to the content identification information newly designated by the user. The content image file name 313 may be displayed in the content display area 73.

  Further, the information processing apparatus 100 determines whether or not there is a moving image data file corresponding to the content specified by the user with reference to the moving image data file name 315. If there is no moving image data file, the information processing apparatus 100 May display to suggest to the user that there is no moving image data file, such as hiding the moving image reproduction receiving area 77.

  Further, when the moving image data file is displayed in the content display area 73 when the input operation to the AR mode receiving area 78 is received, the information processing apparatus 100 stops the playback of the moving picture and starts the camera. The AR process may be started. When the audio data is being reproduced when the input operation to the AR mode reception area 78 is accepted, the information processing apparatus 100 reproduces the AR data while continuing to reproduce the reproduced audio as it is. May be activated.

<Control structure>
The control structure of the information processing apparatus 100 will be described in detail with reference to FIG. FIG. 7 is a flowchart showing processing for displaying AR data by the information processing apparatus 100 according to the first embodiment. In the first embodiment, the AR data is data having a three-dimensional shape. Hereinafter, an operation when the information processing apparatus 100 accepts an input operation from the user and starts the AR process with respect to the AR mode reception area 78 illustrated in FIG. 6 will be described.

  In step S <b> 11, the control unit 14 of the information processing apparatus 100 reads from the storage unit 13 content identification information designated by the user through an input operation on the content designation receiving area 72.

  In step S <b> 13, the control unit 14 sets a marker pattern for detecting the AR marker 300 in order to start the AR process. The control unit 14 reads an image (reference image) showing the AR marker 300 from the storage unit 13 and sets it as a marker pattern.

  In step S15, the control unit 14 specifies the AR data corresponding to the content identification information based on the AR data 316 of the content management table 31, and displays the specified AR data as an overlay on the marker pattern in the AR processing. Set as AR data.

  In step S <b> 17, the control unit 14 sets a capture cycle for acquiring (capturing) a captured image, activates the camera 7, and acquires captured image data according to the capture cycle.

  In step S <b> 19, the control unit 14 detects the AR marker 300 by performing image recognition processing on the captured image data based on the image indicating the AR marker 300 every time captured image data is acquired by the camera 7 according to the capture cycle. Perform the process.

  In step S21, the control unit 14 determines whether or not the AR marker 300 has been successfully detected from the captured image data. If the detection of the AR marker 300 is successful (YES in step S21), the control unit 14 performs the process of step S25. When the AR marker 300 is not detected (NO in step S21), the control unit 14 performs the process of step S19.

  In step S25, the control unit 14 calculates the position and inclination of the AR marker 300 on the captured image based on the area of the AR marker 300 detected from the captured image data. Based on the calculated result, the control unit 14 acquires a coordinate transformation matrix for displaying the AR data of the three-dimensional data on the display 6. When the information processing apparatus 100 acquires a coordinate transformation matrix for displaying the AR data on the display 6, the information processing apparatus 100 takes into account information on the inclination of the information processing apparatus 100 detected by the acceleration sensor or the like of the information processing apparatus 100. Also good.

  In step S27, the control unit 14 displays the AR data of the three-dimensional data as an overlay on the captured image displayed on the display 6 according to the coordinate transformation matrix.

  When the AR data is displayed on the display 6 as an animation, the control unit 14 manages the playback position in the total playback time of the animation and displays the AR data on the display 6 according to the playback position. When the AR marker 300 is detected (step S21: YES), the control unit 14 counts the number of times the AR marker 300 is not successfully detected in the captured image data captured thereafter, and this number of times. Until the number of times reaches a certain number, AR data may be displayed regardless of the detection result of the AR marker 300.

<Example of AR data display>
An example of overlay display of AR data in step S27 will be described with reference to FIG. FIG. 8 is a diagram illustrating an example of displaying AR data. In the example of FIG. 8, the information processing apparatus 100 stores AR data for displaying the state in which the ship is shaken by the waves as animation, as data associated with the art work “the ship floating in the waves”. As shown in screen A, screen B, screen C, and screen D, the AR data of the three-dimensional data changes every moment and is displayed on the display 6.

  In this way, AR data is prepared in association with contents such as artworks, and AR data can be reproduced by performing AR processing using the AR marker 300, so that the user can install the AR marker 300. It is possible to use content that can only be viewed at a specific location. For example, by installing the AR marker 300 in an exhibition hall together with a work of art, AR data with movement can be provided to the user. In addition, for example, in a zoo / botanical garden, by installing the AR marker 300 together with the animals and plants to be viewed, information indicating the ecology and growth process of the animals and plants can be provided to the user. In addition, for example, in the case of a sightseeing spot, by installing the AR marker 300 in the sightseeing spot together with the sightseeing resource, the history related to the sightseeing resource can be provided to the user by voice, a still image, and a moving image. Therefore, the information processing apparatus 100 can prompt the user to visit a place where the AR marker 300 is installed, and can contribute to an increase in visitors such as museums.

<Example of incidental information display>
With reference to FIG. 9, the example which displays the incidental information of a content in the content display area 73 is demonstrated. FIG. 9 is a diagram illustrating an interface for an operation of switching between information incidental to content and information related to the content. In the example of FIG. 9, the information processing apparatus 100 displays, on the display 6, map information indicating the position of the drawing target drawn on the work as supplementary information of the content.

  As shown in FIG. 9, the control unit 14 displays the image data file of the content specified by the user in the content display area 73E according to the content identification information specified by the user. Here, when receiving an input operation on the display switching area 74 from the user, the control unit 14 refers to the content-attached data name 317 corresponding to the content identification information designated by the user. The control unit 14 displays the data (map information which is supplementary information of the content) indicated by the content supplementary data name 317 in the content display region 73F, and displays an image to be displayed in the content display region 73 in the display switching region 74. An operation for switching to an image data file or the like is accepted.

  In the example of FIG. 9, the information processing apparatus 100 displays a map indicating the location of the mountain drawn in the work in the content display area 73F as supplementary information of the work in which the mountain is drawn.

<Embodiment 2>
Next, another embodiment 2 will be described with reference to FIGS. In the second embodiment, a plurality of types of AR markers are used. In the second embodiment, data that is restricted for viewing in the information processing apparatus 100 is also used. The information processing apparatus 100 according to the second embodiment stores flag information indicating whether or not a user can view data with viewing restrictions in a memory, and allows the user to view data with viewing restrictions. The display of data is controlled according to whether or not this is possible.

  In addition, the information processing apparatus 100 displays a display prompting an input operation for releasing the viewing restriction on the data having the viewing restriction. Accordingly, the information processing apparatus 100 provides a user with, for example, age-restricted data, and displays a procedure according to whether or not there is a viewing restriction or a procedure (for example, a charging process) for releasing the viewing restriction. Or provide it to the user.

  FIG. 10 is a diagram illustrating a marker image according to the second embodiment. As shown in FIG. 10, in the second embodiment, two AR markers, an AR marker 300 and an AR marker 301, are used. The type of the AR marker 300 and the type of the AR marker 301 are different from each other. For example, it is assumed that there is no viewing restriction for the AR marker 300, and there is viewing restriction for the AR marker 301. The information processing apparatus 100 can detect the AR marker 300 and the AR marker 301 from the captured image data by previously storing reference images indicating the AR marker 300 and the AR marker 301 in advance.

<Data structure>
With reference to FIG. 11, data used by information processing apparatus 100 in the second embodiment will be described. FIG. 11 is a diagram conceptually showing the structure of the content management table 31 of the second embodiment. Each record of the content management table 31 includes a work identification number 311, a content name 312, a content image file name 313, an audio data file name 314, a moving image data file name 315, AR data 316, and a content incidental data name. 317, an AR marker image 318, and an AR marker image 319 with viewing restriction.

  In the second embodiment, each piece of identification information indicating content is associated with information for specifying an AR marker image. For each piece of identification information indicating content, a marker image (AR marker image 318) for displaying AR data without viewing restrictions and a marker image (AR marker with viewing restrictions) for displaying AR data with viewing restrictions are displayed. Image 319).

  The AR marker image 318 indicates an image for reference of the AR marker corresponding to the content identification information indicated by the work identification number 311. The AR marker image 319 with viewing restriction indicates an image for reference of the AR marker with viewing restriction corresponding to the content identification information indicated by the work identification number 311. Depending on the content, a marker image for displaying an AR marker with viewing restrictions may not be specified. In addition, depending on the content, a marker image for displaying AR data with viewing restrictions may be specified, and a marker image for displaying AR data without viewing restrictions may not be specified.

  In this way, a content exhibitor or the like can specify in the AR data 316 either or both of AR data without viewing restrictions and AR data with viewing restrictions. In addition, the exhibitor or the like displays a marker image (AR marker image 318) for displaying AR data without viewing restrictions and a marker image (AR marker image 319 with viewing restrictions) for displaying AR data with viewing restrictions. Prepare separately. Thereby, the exhibitor of contents can suggest to the user that there is AR data that can be viewed by releasing the viewing restriction. As a result, the information processing apparatus 100 can prompt the user to take action for releasing the viewing restriction. For example, the information processing apparatus 100 may release the viewing restriction on the user by accepting an accounting process, registration of user information, an operation for authenticating the user's age, and the like.

<Control structure>
With reference to FIG. 12, the operation of the information processing apparatus 100 in the second embodiment will be described. The same parts as those in the first embodiment are denoted by the same reference numerals, and description thereof will not be repeated.

  FIG. 12 is a flowchart illustrating processing for displaying AR data by the information processing apparatus 100 according to the second embodiment.

  In step S <b> 12, the control unit 14 refers to the content management table 31, and whether or not a reference image is specified in the AR marker image 319 with viewing restriction corresponding to the identification information of the content that has been specified by the user. Determine whether. If a reference image is designated as the AR marker image 319 with viewing restriction in step S12 (YES in step S12), the control unit 14 performs the process of step S22. In step S12, when a reference image is not specified for the AR marker image 319 with viewing restriction (NO in step S12), the control unit 14 performs the process of step S13.

  In step S <b> 22, the control unit 14 refers to the memory and reads a setting as to whether or not the user can view data with viewing restrictions. When it is set that the user can view data with viewing restrictions (YES in step S22), the control unit 14 performs the process of step S13. When it is set that the user cannot view data with viewing restrictions (NO in step S22), the control unit 14 performs the process of step S24.

  In step S <b> 24, the control unit 14 indicates on the display 6 that the AR data cannot be displayed on the display 6 because the content designated by the user is restricted, and a method for releasing the viewing restriction. indicate.

  In step S <b> 26, the control unit 14 refers to the content management table 31 and determines whether or not a reference image is specified in the AR marker image 318 corresponding to the identification information of the content received from the user. To do. In step S26, when a reference image is not designated in the AR marker image 318 (NO in step S26), the control unit 14 ends the process for displaying the AR data. In step S26, when a reference image is designated as the AR marker image 318 (YES in step S26), the control unit 14 performs the process of step S13.

  In step S <b> 13, the control unit 14 refers to the AR marker image 318 and the viewing-restricted AR marker image 319 in order to start the AR processing, and determines the AR marker corresponding to the content identification information specified by the user. An image is read and set as a marker pattern. For example, the control unit 14 refers to both the AR marker image 318 corresponding to the content identification information and the AR marker image 319 with viewing restriction, reads a reference image indicating the AR marker, and sets it as a marker pattern. . In step S13, the control unit 14 refers to the memory, and when it is set that the user cannot view the data with viewing restrictions, the control unit 14 uses the reference for viewing shown in the AR marker image 319 with viewing restrictions. The image for reference shown in the AR marker image 318 is set as the marker pattern without setting the image as the marker pattern.

  In step S15, the control unit 14 sets AR data corresponding to the content identification information in association with the marker pattern. The control unit 14 refers to the AR data 316 and associates AR data without viewing restriction with the marker pattern indicated in the AR marker image 318. In addition, the control unit 14 associates the AR data with viewing restriction with the marker pattern shown in the AR marker image 319 with viewing restriction.

  In step S21, the control unit 14 determines whether or not the marker image of the marker pattern set in step S13 has been successfully detected from the captured image data. When the detection of the marker image is successful (YES in step S21), the control unit 14 performs the process of step S25. When the marker image is not detected (NO in step S21), the control unit 14 performs the process of step S19.

  In step S27, the control unit 14 overlays the AR data of the three-dimensional data corresponding to the detected marker pattern on the captured image displayed on the display 6 according to the coordinate transformation matrix.

  In the second embodiment described above, the control unit 14 is described as determining whether or not the user can view data with viewing restrictions before the process of detecting the AR marker for the captured image. doing. With the operation of the control unit 14 in this way, there is a possibility that data with viewing restrictions is erroneously displayed on the display 6 when it is set that the user cannot view data with viewing restrictions. Can be reduced.

<Operation acceptance screen for releasing viewing restrictions>
With reference to FIG. 13, an example of display of a screen for accepting an input operation for releasing viewing restriction in order to enable viewing of data with viewing restriction will be described. FIG. 13 is a diagram illustrating an example of a user interface for accepting an input operation for releasing viewing restrictions.

  As illustrated in FIG. 13, the control unit 14 displays on the cancel operation reception area 81 that the AR data can be reproduced by canceling the viewing restriction on the display 6. In the example of FIG. 13, the control unit 14 displays on the display 6 that the content viewing restriction can be removed by the user performing a charging procedure. The coupon purchase button 82 is an input reception button for executing a billing procedure (coupon purchase) for releasing the content viewing restriction. The cancel button 83 is a button for accepting from the user that an operation for releasing the content viewing restriction is not performed. Note that the billing mode is not limited to the purchase of the coupon described above.

  In FIG. 13, the example in which the viewing restriction is canceled by the charging procedure for the data having the viewing restriction has been described. In addition to this, for example, when providing viewing restrictions based on age, the information processing apparatus 100 accepts an input operation for a screen for authenticating the user's age on the display 6 (for example, whether or not the user is over 20 years old). Display screen) may be displayed. Thereby, the information processing apparatus 100 can provide AR data according to the user's age to the user.

<Modification 1 Modification of Each Device>
In the description of each of the above embodiments, the information processing apparatus 100 has been described as a tablet terminal. For example, the information processing apparatus 100 may be a device such as a smartphone.

  FIG. 14 is a diagram illustrating an example of a route for the information processing apparatus 100 to download data such as content. Image data and the like regarding the content is held in the distribution server 900 by an exhibitor or the like who exhibits the content, such as an exhibition organizer. As illustrated in FIG. 14, the distribution server 900 holds image data, audio data, moving image data, AR data, and incidental information attached to the content.

  The information processing apparatus 100A, which is a tablet terminal, can download image data and the like regarding content from the distribution server 900 by connecting to the router 700 by wireless communication and communicating with the distribution server 900 via the network 800. . In addition, the information processing apparatus 100 </ b> B that is a smartphone can download data from the distribution server 900 by connecting to the wireless base station 600 through wireless communication and communicating with the distribution server 900 via the network 800. Also, the information processing apparatus 100A and the information processing apparatus 100B download an application 52 for displaying a guide for these contents from an external server such as the distribution server 900.

  Further, the device that displays the content guide to the user may be the wearable terminal 200 connected to the information processing device 100 by short-range wireless communication. The wearable terminal 200 includes a processor, a memory, a display, and a communication interface. The wearable terminal 200 is connected to the information processing apparatus 100B through, for example, Bluetooth (registered trademark). Wearable terminal 200 is worn by a user, and can display moving image data and the like related to content on the display of wearable terminal 200 to allow the user to view. In the example of FIG. 14, the wearable terminal 200 is illustrated as a glasses-type terminal, but may be other wearable terminals, for example, a terminal that can be worn on an arm.

<Modification 2 Modification of Embodiment 2>
In the second embodiment, the information processing apparatus 100 holds, in a memory, an AR marker image without viewing restriction and an AR marker image for displaying AR data with viewing restriction in advance. Whether or not there is a marker image setting for displaying limited AR data may be held in the content management table 31.

  In the second embodiment, a plurality of types of AR marker images can be set. However, the information processing apparatus 100 holds image files, audio files, and the like in accordance with the types of AR marker images. An image file or the like corresponding to the type of AR marker image detected in the processing may be reproduced. Thereby, the guidance according to the kind of AR marker can be provided to a user.

<Modification 3 Method for Designating Content Displayed on Display 6>
In the description of each of the above embodiments, the information processing apparatus 100 stores content identification information specified by a user input operation, and displays image data or the like corresponding to the content identification information specified by the user on the display 6. doing. In addition to this, the information processing apparatus 100 has a GPS (Global Positioning System) function, a position positioning function by communication with a wireless LAN router, and the like, and stores information regarding the installation position of each content, thereby providing the content installation position. The identification information of the content to be displayed on the display 6 may be specified based on the information processing apparatus 100 and the position of the information processing apparatus 100. For example, the information processing apparatus 100 specifies the content at the installation position closest to the position of the information processing apparatus 100 as identification information of the content to be displayed on the display 6. Thereby, the information processing apparatus 100 can specify the content to be displayed on the display 6 without the user performing an input operation.

<Summary>
As described above, the information processing apparatus 100 according to this embodiment or the modification can provide guidance information related to content to a user with a simple input operation. In addition, by installing a marker image together with the content display, detecting the marker image in the information processing apparatus 100 and displaying the AR data on the display 6, it is possible to provide the user with a guide that can be used at the content display location. The user can be encouraged to visit the exhibition place.

  As described above, the information processing apparatus 100 that provides information for guidance related to content can be used, for example, for explaining a work to be displayed in an art museum or a museum. In addition to this, it can be used for explanation of objects to be appreciated at zoos and botanical gardens, and explanation of tourism resources. For example, in the explanation of tourism resources, the history about tourism resources may be displayed as AR data, and information on movies etc. may be displayed at sightseeing spots used for filming locations, animations, etc. If it is a sightseeing spot used for drawing works such as works, comics, and games, or a sightseeing spot that is the stage of literary works such as novels, characters appearing in these animated works may be displayed.

  The embodiment disclosed this time must be considered as illustrative in all points and not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

  DESCRIPTION OF SYMBOLS 100 Information processing apparatus, 200 Wearable terminal, 600 Wireless base station, 700 Router, 800 Network, 900 Distribution server, 300 AR marker, 301 AR marker, 1, 2 antenna, 3 Long-distance communication module, 4 Short-distance communication module 5 Touch panel, 6 Display, 7 Camera, 8 Power control unit, 9 Battery, 10 Microphone, 11 Speaker, 12 Audio processing unit, 13 Storage unit, 14 Control unit.

Claims (10)

An information processing apparatus for displaying guidance information,
An operation accepting unit for accepting input operations;
A camera for obtaining photographed image data by photographing;
A display unit;
A storage unit for storing first content data corresponding to the identification information for each of the reference image indicating the specific subject and the identification information of the plurality of contents;
A control unit that controls the operation of the information processing device based on the input operation,
The controller is
A detection processing unit that performs processing for detecting the specific subject from the captured image data by performing image recognition processing on the captured image data based on the reference image;
The first content data corresponding to identification information of display target content to be displayed on the display unit when the camera is driven and the specific subject is detected from the captured image data by the detection processing unit An information processing apparatus including a display processing unit that displays a message on the display unit. The operation receiving unit receives an input operation for designating the display target content;
The display processing unit
The identification information of the display target content is set according to the input operation for designating the display target content, and the first content data of the display target content corresponding to the set identification information is specified. The information processing apparatus according to claim 1, wherein when the subject is detected from the captured image data, the subject is displayed on the display unit. The storage unit
Video data to be displayed on the display unit when the specific processing object is detected by the detection processing unit is stored as the first content data in association with the identification information of the content, and second content data Storing at least one of audio data and video data related to the content,
The display processing unit
When the input operation for reproducing the second content data is accepted by the operation accepting unit, the audio data or the moving image data corresponding to the identification information of the display target content is reproduced,
The video data is reproduced according to a detection result of the specific subject from the captured image data when an input operation for executing a detection process by the detection processing unit is received by the operation receiving unit. 1. An information processing apparatus according to 1. The storage unit
As the second content data, audio data related to the content, moving image data, and incidental information attached to the content are stored,
As the first content data, storing video data to be displayed on the display unit when the specific processing object is detected by the detection processing unit,
The display processing unit
The information processing apparatus according to claim 1, wherein when the input operation for displaying the auxiliary information is received by the operation receiving unit, the auxiliary information corresponding to the identification information of the display target content is displayed on the display unit. . The storage unit stores map information related to the content as incidental information attached to the content,
The said display process part displays the said map information corresponding to the identification information of the said display target content on the said display part, when the input operation for displaying the said incidental information by the said operation reception part is received. 4. The information processing apparatus according to 4. The storage unit
For each of the plurality of contents, management information for managing whether or not content display is restricted is stored, and setting information for storing whether or not display of the restricted content is permitted is stored Is,
The display processing unit
The information processing apparatus according to claim 1, wherein the display target content that is restricted in content display is controlled to display the first content data on the display unit according to the setting information. The storage unit stores the reference image indicating each of the specific subjects for a plurality of types of the specific subjects,
The detection processing unit performs processing for detecting each of the plurality of types of the specific subjects from the captured image data based on the reference image,
The display processing unit controls display of the first content data according to a type of the detected specific subject when the specific subject is detected from the captured image data. Information processing device. The storage unit
Storing the reference image of the specific subject indicating that content display is limited, and setting information as to whether or not display of the content with viewing restriction is permitted,
The display processing unit
The display unit of the first content data is read in accordance with the setting information when the setting information is read and the specific subject indicating that the display of the content is limited is detected from the captured image data. The information processing apparatus according to claim 7, which controls display on the screen. The display processing unit
When the specific subject indicating that the display of the content is limited is detected from the photographed image data and display of the content with the viewing limitation is not permitted in the setting information, the limitation is released. The information processing apparatus according to claim 8, wherein a screen for accepting an input operation for displaying is displayed on the display unit. A program for controlling the operation of an information processing device for displaying guidance information, the information processing device taking a picture by photographing, a processor, a memory, an operation accepting unit for accepting an input operation, and It has a camera for obtaining image data and a display,
The memory is for storing content data corresponding to the identification information for each of the reference image indicating a specific subject and the identification information of a plurality of contents,
The program is stored in the processor.
Performing a process for detecting the specific subject from the captured image data by performing an image recognition process on the captured image data based on the reference image;
Driving the camera and displaying the content data corresponding to the identification information of the display target content to be displayed on the display when the specific subject is detected from the captured image data. A program to be executed.

Images