JP4872317B2 - Information processing system - Google Patents

Description

The present invention relates to an information processing system .

  There has been proposed an apparatus that connects to a remote camera server via a network, controls the camera via the camera server, and transfers a captured image to a client. With such a technique, although the image captured by a remote camera can be confirmed, it is impossible to obtain more information than the image of the object for each displayed object.

  For this reason, a technique that can confirm details of an object in a video has been proposed (see Patent Document 1). The technique described in Patent Document 1 determines whether or not an object falls within the shooting field when the camera angle is changed in accordance with an instruction from an external device. In order to transfer information unique to the object when the object is clicked, an anchor is set near the area of the image of the object, and is transferred to the external device together with the photographed image. When an object is designated on the screen of the external device, the file is transferred to the external device in response to a file transfer request related to the object in accordance with the description of the anchor.

In addition, in order to know the details of the objects present in the video, a plurality of IDs capable of detecting the position are assigned to the objects in the real world, and the existence of the objects is recognized by recognizing their positions. When a clickable area is extracted from a video image obtained by capturing the object based on the detected area and the extracted clickable area is clicked, the corresponding URL ( A technique for enabling access to a uniform resource locator has been proposed (see Patent Document 2).
Japanese Patent Laid-Open No. 11-112857 JP 2002-262163 A

  However, in the technique of Patent Document 1 described above, when there are a large number of objects present in the video or when there are many types of objects, it is necessary to perform processing on all the objects present in the video. There is a problem that the processing becomes complicated.

  In the technique of Patent Document 2, a plurality of small IDs must be assigned in advance to an object to be recognized. Since it is necessary to install a sensor for detecting the ID, the entire system becomes very large.

Therefore, the present invention has been made in view of the above-described problems, and the related operations can be realized by a simple operation by a user looking at the display screen with a simple configuration without complicating the processing. an object of the present invention is to provide an information processing system.

In order to solve the above-described problems, the present invention provides a display processing unit that embeds a predetermined image and displays content, an imaging unit that captures a display screen displayed by the display processing unit, and a display captured by the imaging unit. a delivery means for delivering the information on the screen, and recognition means for recognizing the predetermined image embedded in the display screen based on the information of a display screen said distribution means distributes said predetermined recognized by said recognition means Display control means for displaying the predetermined image portion in the display screen as an instructable area on the display screen according to the content of the image, and the predetermined image when the instructable area is instructed Activation means for activating an operation corresponding to the above. According to the present invention, by displaying a predetermined image embedded in the display screen as an area that can be designated on the display screen, a user browsing the display screen can perform a related operation with a simple operation. It becomes. In addition, the processing is not complicated because it is only necessary to perform the processing on the necessary items. Further, since a sensor for installing an ID is not required, the system can be realized with a simple configuration. In addition, operations related to devices installed in a remote work space can be easily realized through captured images.

The information processing system of the present invention further includes a generation unit that generates the predetermined image based on the content. According to the present invention, an image corresponding to each content can be embedded. In the information processing system of the present invention, when the content is a presentation material, the generation unit generates the predetermined image based on URL information and page information in which the presentation material is stored. According to the present invention, it is possible to display synchronously with a remote presentation.

The display processing means may embed the predetermined image at a predetermined time interval of 0.2 seconds or less . Thus, for example, by displaying the code information at a time interval that cannot be recognized by a human as the predetermined time interval, the instructable area can be displayed without being noticed by the user. The display processing means may embed the predetermined image in an invisible state. According to the present invention, for example, an instructable area can be displayed without being noticed by a user by sending a predetermined image in an invisible state that cannot be recognized by a human as a predetermined time interval.

  The display processing means preferably embeds the predetermined image with an optical signal outside the visible region in a form superimposed on video information transmitted by visible light. According to the present invention, the code information is displayed and reproduced with an optical signal outside the visible region such as infrared light or ultraviolet light in a form superimposed on the video information transmitted by visible light. The clickable region can be extracted without noticing that the information is included.

The display control means generates a control panel that can be remotely operated from the information on the display screen, and displays the instructable area in the control panel. According to the present invention, it is possible to operate a device installed at a remote location from a remote location by calling a control panel and starting a program using this control panel. For example, code information can be used for the predetermined image .

According to the present invention, the process without becoming complicated, and a simple configuration, it is possible to provide an information processing system that can realize the operation by the user is viewing the display screen is associated with a simple operation.

  Hereinafter, the best mode for carrying out the present invention will be described.

  A first embodiment of the present invention will be described. FIG. 1 is a diagram showing a configuration of a remote control system according to a first embodiment of the present invention. FIG. 2 is a diagram showing a data structure of data used in the remote control system according to the first embodiment of the present invention. As shown in FIG. 1, a remote control system 100 as an information processing system includes a content distribution server 10 and client PCs 20 and 30. These are connected to the Internet 40 via a LAN (Local Area Network). The content distribution server 10 and the client PCs 20 and 30 are configured using a CPU (Central Processing Unit), a RAM (Random Access Memory), a ROM (Read Only Member), a hard disk device, and the like, and a predetermined program is executed. Each function is realized.

  The content distribution server 10 includes a display processing unit 11 and a distribution unit 12. The display processing unit 11 performs a process of embedding an image image as a predetermined image in a Web page (display screen). Here, for example, code information may be used as the image. The code information includes a two-dimensional code such as a QR code. The distribution unit 12 performs processing for distributing the Web page in which the display processing unit 11 embeds an image.

  The client PCs 20 and 30 include a recognition unit 21 as a recognition unit, a display control unit 22 as a display unit, and an activation unit 23 as an activation unit. The configuration of the client PC 30 is the same as the configuration of the client PC 20, and therefore the details of the configuration are omitted in the figure. FIG. 3 is a diagram showing a display screen displayed on the display device 24 by the client PC. Code information displayed as an image is embedded in a Web page (HTML (HyperText Markup Language) file).

  The recognizing unit 21 recognizes an image embedded in the Web page based on information on the Web page distributed by the distributing unit 12. The display control unit 22 displays the image image portion in the Web page on the Web browser as an instructable area on the Web page according to the content of the image image recognized by the recognition unit 21. At this time, the display control unit 22 displays the image portion as a clickable region. When an instructable area is designated, the display control unit 22 embeds a link so as to jump to a URL corresponding to the image.

  In other words, when the image image represents a URL or information that can be encoded into a URL, the display control unit 22 displays the image image 160 on the Web browser 150 as a clickable area 170 that can be clicked. A link is automatically embedded so that clicking on the clickable area 170 can jump to the URL. The activation unit 23 activates an operation corresponding to the designated image when a clickable area is designated. Here, the operation corresponding to the image includes, for example, operations such as displaying a Web page corresponding to the image and starting a program corresponding to the image.

  FIG. 4 is a process flowchart of the client PC according to the first embodiment of the present invention. The display control unit 22 of the client PCs 20 and 30 analyzes whether all the image images specified by the image tag (<IMG tag>) in the HTML file of the Web page to be displayed represents code information ( Step S101). Next, when the image file represents code information (step S102), the display control unit 22 of the clients 20 and 30 acquires and interprets the code information represented by the image file in step S103, and obtains the URL information. Obtain (step S103). Next, the display control unit 22 of the client PCs 20 and 30 outputs the link tag (A) related to the URL so as to surround the image tag (IMG), and makes the image image portion clickable (step S104). Then, the display control unit 22 of the client PCs 20 and 30 displays the converted HTML file on the browser (step S105). The activation unit 23 jumps to the URL when the clickable area 170 portion is clicked.

  Conventionally, in order to jump to a URL displayed with a QR code or the like in a Web page or the like, an editor of the Web page embeds a link in advance or the user recognizes the code information with a camera or the like. Needed to be displayed. According to the first embodiment of the present invention, the client PCs 20 and 30 indicate the URL when the image shows code information and the content starts with “http: //”. By embedding and displaying as link information, the user can jump to the URL embedded in the image image by clicking the code information portion displayed in the image image without recognizing the code information. can do. Thereby, the user who is looking at the display screen can realize a related operation with a simple operation. In addition, the processing is not complicated because it is only necessary to perform the processing on the necessary items. Further, since a sensor for installing an ID is not required, the system can be realized with a simple configuration.

  Next, a second embodiment of the present invention will be described. In the second embodiment of the present invention, when information on a device installed in a remote work space is acquired through a video image, the control panel is called, and the operation of the device is started from the control panel. It is an example. The device arranged in the remote work space is preferably a device having a display screen.

  FIG. 5 is a diagram showing the configuration of the remote control system according to the second embodiment of the present invention. FIG. 6 is a diagram showing a data structure of data used in the remote control system according to the second embodiment of the present invention. As shown in FIG. 5, the remote control system 200 has a base system 210 and an access base system 220. Base system 210 and access base system 220 are connected via network 230. The base system 210 is provided in a conference room, and a user using the access base system 220 can perform a remote conference with a conference participant in the conference room.

  The base system 210 includes a conference management server 211, display devices 212 to 214, a video camera 215 as an imaging unit, and a video distribution server 216 as a distribution unit.

  The conference management server 211 functions as a code image generation unit 2111 and a display processing unit 2112. The code image generation unit 2111 generates an image as code information representing information related to devices such as the display devices 212 to 214 so that the devices installed in the conference room can be controlled from a remote place.

  The display processing unit 2112 displays the content in which the image images 410 to 412 are embedded on the display devices 212 to 214. FIG. 7 is a diagram showing an example of a display method of an image image as code information displayed by the conference management server 211. FIG. 7A shows an example in which code information is arranged at four points at four corners in the display screen. FIG. 8B is a diagram showing an example when code information is arranged at one point on the display screen, and FIG. 10C is an example when code information is arranged according to the arrangement of objects. FIG.

  The display processing unit 2112 embeds the image images 410 to 412 in content in a state invisible to humans and displays them on the display devices 212 to 214. For example, the display processing unit 2112 embeds and transmits the image images 410 to 412 with an optical signal outside the visible region such as infrared light or ultraviolet light, for example, so as to be superimposed on video information transmitted by visible light. This prevents the user from noticing that code information is included in the video.

  The display devices 212 to 214 are configured by, for example, a CRT display or a liquid crystal display. A projector may be used instead of the display. The video camera 215 images the display screens of the display devices 212 to 214 displayed by the display processing unit 2112. Further, the video camera 215 converts the video shot by the built-in encoder into a digital video. The video distribution server 216 distributes information on a display screen captured by the video camera 215, and specifically distributes encoded video data.

  The access base system 220 includes a client PC 221 and a display device 222. FIG. 8 is a diagram for explaining a screen displayed on the display device 222 by the client PC 221. FIG. 8A shows a Web browser 301 including a clickable area displayed on the display device 222. FIG. 5B is a diagram showing the Web browser 302 including an operation panel opened when a clickable area is designated, and FIG. 5C is a diagram illustrating the Web browser 303 including a Web page displayed when the clickable area is designated. FIG.

  The client PC 221 functions as a recognition unit 2211, a display control unit 2212, and an activation unit 2213. The recognition unit 2211 recognizes an image image embedded in the display screen based on information on the display screen distributed by the video distribution server 216. The display control unit 2112 generates a control panel that can be remotely operated from the display screen, and displays a specifiable area in the control panel. The display control unit 2212 analyzes the image image recognized by the recognition unit 2211. When the image image is code information, the display image unit 2212 can instruct the image image portion in the display screen on the display screen according to the content of the code information. Display as an area. At this time, the display control unit 2212 displays an area which can be clicked with a mouse or the like as an instructable area. The activation unit 2213 activates an operation according to the code information when an instructable area is instructed.

  FIG. 9 is a diagram for explaining the processing of the second embodiment of the present invention. FIG. 10 is a process flowchart of the video distribution server 216 according to the second embodiment of the present invention. FIG. 11 is a processing flowchart of the client PC 221 according to the second embodiment of the present invention. Here, an example will be described in which the conference management server 211 embeds code information in content using infrared light.

  The conference management server 211 displays code information representing information on the device in an invisible form using infrared light on the display screens of the display devices 212 to 214. The video distribution server 216 digitizes and distributes the video input from the video camera using a video encoder (step S201).

  The client PC 221 receives the video data distributed through the network. In FIG. 9, reference numeral 400 indicates all videos of the video data received from the video distribution server 216. Display devices 212 to 214 exist in the entire video 400. Actually, although it cannot be seen, image images 410 to 412 as code information are embedded in the display devices 212 to 214 and the panels 217 and 218, respectively.

  The recognition unit 2211 of the client PC 211 recognizes the code information portion for each frame of the video image distributed from the video distribution server 216, and recognizes the coordinates in the image by using the code information portion as a clickable area (step). S301). Specifically, the recognition unit 2211 of the client PC 221 recognizes the code information portion by applying an infrared extraction filter to the information of the entire video 400. Reference numeral 401 indicates a visible image. Reference numeral 402 indicates an image after the infrared extraction filter is applied. Since only the infrared region image is extracted from the infrared extraction filter image 402, the image images 410 to 412 as code information can be extracted. In this way, by cutting out the area where the code information is displayed as a clickable area, the clickable area in the screen can be easily cut out without modeling the three-dimensional space.

  The client PC 211 extracts the URL of the control panel or the URL related to the correspondence information included in the recognized code information, and stores it in association with the clickable area (step S302). The client PC 221 acquires information included in the code information, and opens an operation panel based on information such as a URL included in the information so that the device can be operated.

  The client PC 211 reproduces the clickable areas 420 to 424 superimposed on the image being reproduced with respect to the extracted coordinate area (step S303). When the activation unit 2213 is instructed by a mouse or the like on a screen area that is being reproduced in synchronization with the clickable area generated by the above processing, a URL corresponding to the control panel or information is opened by the browser (Step S2213) S304). Note that the area cut out here functions as a clickable area even if the code information is not displayed in the portion. In this way, the control screen can be converted into a Web service and used at a remote location.

  Note that, in the above, the display processing unit 2112 may display the code information at a speed that cannot be recognized by humans. That is, the display processing unit 2112 embeds and displays the image images 410 to 412 in the content at time intervals (predetermined time intervals) that cannot be seen by humans. As a result, when code information is displayed on the screen, it cannot be recognized by humans, it is displayed as image data in a time that can be recognized by a computer, and the code information is displayed to the user by cutting it out as a clickable area. It becomes possible to use without being conscious of being.

  Here, a subliminal effect is used so that it cannot be recognized by a human but can be recognized by a computer. This subliminal effect is an effect that appears when stimulating under consciousness. For example, there are things that increase the viewer's willingness to purchase by repeatedly inserting advertisements with a speed and volume that cannot be perceived on TV or radio. . For example, the threshold value is 0.2 seconds or less per second. This is equivalent to inserting about 6 frames in the case of a moving image that normally sends 30 frames / second. Therefore, by generating and inserting an image at a time interval of about 0.2 seconds or less as a predetermined time interval, it can be recognized by a computer and not recognized by a human.

  Further, the code image generation unit 2111 may automatically generate code information to be displayed from display content. That is, the code image generation unit 2111 may automatically generate the image images 410 to 412 serving as code information displayed together with the content based on the content displayed on the display devices 212 to 214. At this time, software for displaying the internal content automatically generates code information from the content of the display content. The content includes, for example, presentation materials, image content, video content, and the like. For example, in the case of presentation materials, URL information and page information in which presentation materials are stored are coded and displayed as code information each time the page is switched. As a result, it is possible to display synchronously with a remote presentation.

  The code image generation unit 2111 may generate the image images 410 to 412 based on information set by the user. When editing content to be displayed on the display devices 212 to 214, the editor may embed code information in the content. Further, code information may be displayed as a function provided in the display devices 212 to 214 itself.

  In addition, video information may be recorded to allow access to related information later. A means for recording video is provided, and a clickable area is cut out from the recorded video by the same method. This makes it possible to obtain the same effect on information exchanged offline.

  Furthermore, the program may be activated and controlled based on the code information. Command information for starting and controlling the program is included in the code information displayed in the video, and when the code information is recognized, the program is started and operated based on the command. As a result, it is possible to realize interactive use such as controlling the turning of the presentation material displayed on the screen and displaying a questionnaire sheet.

  According to the second embodiment of the present invention, a control panel for directly operating an object existing in a remote place can be called through a video image. In addition, it is possible to take out information regarding an object existing in a remote place. Therefore, a user who is viewing the display screen can realize a related operation with a simple operation. In addition, the processing is not complicated because it is only necessary to perform the processing on the necessary items. Further, since a sensor for installing an ID is not required, the system can be realized with a simple configuration.

  Although the preferred embodiment of the present invention has been described in detail above, the present invention is not limited to the specific embodiment, and various modifications, within the scope of the gist of the present invention described in the claims, It can be changed. The information processing method according to the present invention is realized by a remote control system.

It is a figure which shows the structure of the remote control system which concerns on Example 1 of this invention. It is a figure which shows the data structure of the data used with the remote control system which concerns on Example 1 of this invention. It is a figure which shows the display screen which client PC displays on a display apparatus. It is a processing flowchart of client PC concerning Example 1 of the present invention. It is a figure which shows the structure of the remote control system which concerns on Example 2 of this invention. It is a figure which shows the data structure of the data used with the remote control system which concerns on Example 2 of this invention. It is a figure which shows the example of the display system of the code image which the meeting management server 211 displays. 6 is a diagram for explaining a screen displayed on the display device 222 by the client PC 221. FIG. It is a figure for demonstrating the process of 2nd Example of this invention. It is a process flowchart of the video delivery server 216 which concerns on 2nd Example of this invention. It is a process flowchart of client PC221 based on 2nd Example of this invention.

Explanation of symbols

100, 200 Remote control system 10 Content distribution server 20, 30 Client PC
210 Base System 220 Access Base System 211 Conference Management Server 212-214 Display Device 215 Video Camera 216 Video Distribution Server 221 Client PC
222 Display device

Claims (8)

Display processing means for displaying a content by embedding a predetermined image;
Imaging means for imaging a display screen displayed by the display processing means;
Distribution means for distributing information on a display screen imaged by the imaging means;
Recognition means for recognizing the predetermined image embedded in the display screen based on the information of a display screen said distribution means for distributing,
Display control means for displaying the predetermined image portion in the display screen as an instructable area on the display screen according to the content of the predetermined image recognized by the recognition means;
An information processing system comprising: an activation unit that activates an operation corresponding to the predetermined image when the instructable area is instructed. The information processing system according to claim 1 , further comprising generating means for generating the predetermined image based on the content. 3. The information according to claim 2 , wherein when the content is a presentation material, the generation unit generates the predetermined image based on URL information and page information in which the presentation material is stored. Processing system. The information processing system according to claim 1 , wherein the display processing unit embeds the predetermined image at a predetermined time interval of 0.2 seconds or less . The information processing system according to claim 1 , wherein the display processing unit embeds the predetermined image in an invisible state. 2. The information processing system according to claim 1 , wherein the display processing unit embeds the predetermined image with an optical signal outside the visible region in a form superimposed on video information transmitted by visible light. 3. The information processing system according to claim 1 , wherein the display control unit generates a control panel that can be remotely operated from information on the display screen, and displays the instructable area in the control panel. The information processing system according to any one of claims 1 to 7 , wherein the predetermined image is code information.

Images