JP2013016030A - Program reproduction apparatus and program reproduction method - Google Patents

Abstract

[PROBLEMS] To realize a program reproducing apparatus in which a viewer can easily select a desired target from a plurality of targets included in a program video and quickly obtain information on the target via a network.
Image extraction means for extracting one or a plurality of regions from one frame as image data, GUI control means for generating a marker that clearly indicates the position of the extracted image data, and a marker superimposed on one frame Display control means for displaying, operation means for selecting one image data by using a marker by a viewer, image search key generating means for generating an image search key from program sequence information and program specifying information, An image search / collation unit that obtains a reference image from an external server using a search key and collates it with image data, and generates an information search key that indicates the attribute of the image data based on the collation result. And information retrieval means for acquiring related information.
[Selection] Figure 1

Description

  The present invention relates to a program reproducing apparatus capable of reproducing a program composed of characters, still images, moving images, sounds, and the like and searching for information related to the program.

  When viewing a digital broadcast or network-delivered program, if the viewer finds an unknown performer, he may want to know the name and profile of the performer on the spot. For example, in digital broadcasting, program arrangement information (Service Information: hereinafter referred to as SI information) is included in a broadcast stream, and therefore, the electronic program guide (Electronic Program Guide: hereinafter referred to as EPG) is used on the receiver side using this SI information. A screen). The viewer can see the names of the main performers of the program being viewed by referring to the EPG.

  However, since the viewer may not know the face of the performer in the first place, it is often difficult to know which performer is the name of the performer described in the EPG. In order to match the name and the face, it is necessary to know the performer's face by searching for the name of the performer using a Web search site via a personal computer, for example. For example, it is conceivable to search for the name of an EPG performer in order to call a face image corresponding to the performer name on the screen of the personal computer, and collate the face image with the performer video of the program. It is very complicated for the person. In addition, if a search environment such as a personal computer is not at hand, the viewer may eventually give up without knowing who he is. Moreover, since only the main performer names are included in the SI information, the performer names may not be described in the EPG.

  In order to solve such a problem, the face image data included in the broadcast video is extracted, and the extracted face image data is transferred to a device that collates with a reference image stored in a database. A method is disclosed that enables viewers to identify unknown performers by acquiring information (name, gender, profile, etc.) corresponding to an image and displaying it on the screen of the receiver (for example, , See Patent Document 1).

JP 2009-118411 A (page 4-9, FIG. 6)

  However, in the conventional method as described above, in order to cut out the target face image from the image, a frame indicating the region is displayed on the screen, and the viewer moves the frame by remote control operation, etc. Although a method of automatically displaying a frame in a face range by a detection process is shown, any method can cut out a desired face image from a scene in which a plurality of persons are shown on one screen. It is difficult. In addition, there is a possibility that an image of inappropriate quality for face matching, such as small image size, face facing sideways, insufficient brightness and contrast, may be cut out. There is a problem that subsequent verification processing becomes difficult.

  The present invention has been made to solve the above-described problems, and allows a viewer to easily select a desired target from a plurality of targets included in a video of a digital broadcast or network-delivered program. An object of the present invention is to realize a program reproducing apparatus capable of quickly obtaining the target information.

  The program playback apparatus according to the present invention is a program playback apparatus for playing back a received program, wherein a communication means for communicating with an external server via a network, a transmission stream of the program as a video stream, an audio stream and data Demultiplexing means for separating into sections, video decoding means for decoding the video stream and converting it into a video signal, and extracting one or a plurality of areas from one frame included in the video signal as image data An image extracting means for generating, a GUI control means for generating a marker for clearly indicating the position of the extracted one or a plurality of image data in the one frame, and displaying the marker superimposed on the one frame. The display control means and the viewer using the marker or the one place or Operation means for selecting one image data from a plurality of image data, program data extraction means for extracting program arrangement information and program identification information included in the data section, and reference from the program arrangement information and the program identification information An image search key generating means for generating an image search key for searching for an image, and using the image search key, the communication means acquires the reference image from the external server via the network, and the image data And an image search / collation unit for collating the reference image with the reference image, and generating an information search key indicating an attribute of the image data based on the collation result, and the image via the network by the communication unit using the information search key. And an information search means for acquiring information related to the data.

  The present invention displays a marker that clearly indicates the position of one or a plurality of image data extracted from a video of a program so as to be superimposed on the original frame from which the image data has been extracted. By selecting the data and comparing the image data with the reference image, the attribute of the image data selected by the viewer is specified, and the information of the image data selected by the viewer is acquired via the network based on the attribute. With this configuration, the viewer can easily select a specific target from a plurality of targets included in the video of the program, and quickly obtain information on the target.

It is a block block diagram which shows the program reproduction apparatus in Embodiment 1 of this invention. It is a network block diagram in Embodiment 1 of this invention. It is a flowchart which shows operation | movement of the program reproduction apparatus in Embodiment 1 of this invention. It is an external appearance example of the operation input part in Embodiment 1 of this invention. It is an example of a structure of the setting screen in Embodiment 1 of this invention. It is an example of a focus movement of the setting screen in Embodiment 1 of this invention. It is an example of the program reproduction | regeneration screen in Embodiment 1 of this invention. It is an example of the face image cut-out area | region in Embodiment 1 of this invention. It is an example of the extraction image table in Embodiment 1 of this invention. It is an example of the selection screen of the extraction image area | region in Embodiment 1 of this invention. It is an example of the marker setting with respect to the extraction image area | region in Embodiment 1 of this invention. It is an example of the image search key list in Embodiment 1 of this invention. It is an example of the reference image acquisition request message and reference image acquisition response message in Embodiment 1 of this invention. It is an example of the information acquisition request message in Embodiment 1 of this invention. It is an example of the information acquisition response message in Embodiment 1 of this invention. It is an example of the information display screen in Embodiment 1 of this invention. It is a flowchart which shows operation | movement of the program reproduction apparatus in Embodiment 2 of this invention. It is an example of the extraction image table in Embodiment 2 of this invention. It is an example of the selection screen of the extraction image area | region in Embodiment 2 of this invention. It is a block block diagram which shows the program reproduction apparatus in Embodiment 3 of this invention. It is an example of the image search key database in Embodiment 3 of this invention.

Embodiment 1 FIG.
FIG. 1 is a block configuration diagram showing an example of the configuration of a program playback device 1 according to Embodiment 1 for carrying out the present invention. The program playback device 1 includes a tuner / demodulator 10, a descrambler 11, a demultiplexer 12, a video decoder 13, a program data extractor 14, an image search key generator 16, an image extractor 17, as main elements. Image search collation unit 18, information search control unit 19, communication protocol processing unit 20, network communication unit 21, GUI (Graphical User Interface) control unit 22, operation input unit 23, operation analysis unit 24, display control unit 25, display unit 26, an audio decoding unit 27 and an audio output unit 28.

  The tuner / demodulator 10 is a part that performs front-end processing for receiving a broadcast, and extracts stream data from the broadcast wave by selecting and demodulating the broadcast. Since the stream data extracted from the broadcast wave by the tuner / demodulator 10 is encrypted, it is decrypted by the descrambler 11. The demultiplexing unit 12 separates the stream data decoded by the descrambling unit 11 into a section including a video stream, an audio stream, and data such as SI information and program identification information.

  The video decoding unit 13 decodes a video stream that has been compression-encoded in a format such as MPEG-2 (Moving Picture Experts Group phase 2) and converts it into a video signal. The program data extracting unit 14 extracts data such as SI information and program specifying information from the section. The image search key generation unit 16 generates a search key for searching for an image. The image extraction unit 17 cuts out a face image from the video signal decoded by the video decoding unit 13. The image search collation unit 18 compares the face image data cut out by the image extraction unit 17 with the reference image data, collates the face, and generates a search key for information retrieval.

  The information search control unit 19 performs a search based on the search key via the network. The communication protocol processing unit 20 generates a transmission message and analyzes a received message according to a necessary protocol. The network communication unit 21 transmits and receives packets via the network.

  The GUI control unit 22 generates various operation screens and status notification screens, and switches screens according to input commands. The operation input unit 23 is a remote controller, a keyboard, a mouse, or the like, and is a generic name for input devices for a user to input operation instructions. The operation analysis unit 24 analyzes the command input from the operation input unit 23 and notifies the GUI control unit 22 of the command. The display control unit 25 performs display control of the video signal decoded by the video decoding unit 13 and various screens generated by the GUI control unit 22. The display unit 26 is a display that displays an appropriate video according to the display control of the display control unit 25.

  The audio decoding unit 27 decodes an audio stream that has been compression-encoded in a format such as AAC (Advanced Audio Codec) and converts it into audio signal data. The audio output unit 28 converts the audio signal data decoded by the audio decoding unit 27 into an analog signal, amplifies it, and outputs it from the speaker.

  FIG. 2 is a network configuration diagram showing a state in which the program playback device 1 of the present embodiment is connected to the network. The program playback device 1 is connected to the network 2 as shown in FIG. A typical network 2 in this figure is the Internet, but there may be other dedicated intranets and home networks.

  An image database server 3 and an information database server 4 are connected to the network 2. The image database server 3 is a database in which reference image data necessary for face matching is associated with attribute information explaining the contents of the reference image (for example, in the case of a person image, information such as the name, gender, and age of the person). And are stored. The information database server 4 stores detailed information such as the profile of a specific person.

  In FIG. 2, the image database server 3 and the information database server 4 are described as separate servers, but may be integrated into the same physical server. In FIG. 2, one program playback apparatus 1, one image database server 3, and one information database server 4 are shown, but a plurality of each may be connected.

  Next, the operation of the program playback device 1 in the present embodiment will be described. FIG. 3 is a diagram showing a flow of the operation of the program playback device 1 in the present embodiment. FIG. 4 shows an example of the operation input unit 23 used for the operation of the program playback device 1 in the present embodiment. As shown in FIG. 4, the operation input unit 23 will be described using a remote control as an example.

  Remote control keys are arranged in the operation input unit 23. As keys used in this embodiment, keys for directly selecting channel numbers and inputting numbers are arranged in a shape of 3 columns × 4 rows. The names of these keys are number 1 key 41, number 2 key 42, number 3 key 43, number 4 key 44, number 5 key 45, number 6 key 46, number 7 key 47, number 8 key 48, number 9 respectively. The key 49, the number 10/0 key 50, the number 11 key 51, and the number 12 key 52 are used. The twelve keys are collectively referred to as numeric keys.

  In the program playback apparatus 1, operations that are performed while viewing a GUI screen (setting screen) such as a menu screen displayed on the display unit 26, rather than operations directly performed by pressing a remote control key such as channel designation and volume change. In this case, the user selects a desired item by focusing on a portion of the setting screen where selectable items are displayed. Here, “focus” indicates that an arbitrary location is specified from selectable locations included in the display screen.

  FIG. 5 is a configuration example of a setting screen in the present embodiment. As shown in FIG. 5, the color or tone of the focused area is changed or surrounded by a frame. FIG. 5 shows that “network setting” is focused. For the movement of the focus, a method is used in which the selectable portions are sequentially moved using the “cross key (up / down / left / right arrow keys)” of the remote controller.

  A cross key for moving the focus is arranged in the operation input unit 23 shown in FIG. The cross key is actually composed of four types: a key that moves upward, a key that moves downward, a key that moves right, and a key that moves left. These keys correspond to 31, 32, 33 and 34 in FIG. 4, respectively, and are hereinafter referred to as an upper key 31, a lower key 32, a right key 33 and a left key 34 for convenience. The four keys of the upper key 31, the lower key 32, the right key 33, and the left key 34 are collectively referred to as a cross key. FIG. 6 is an example of focus movement on the setting screen in the present embodiment. The example of FIG. 6 shows a case where the focus is moved to the “network setting” from the state where the “reception setting” is focused, and it is necessary to press the down key 32 twice for this movement. In addition to the above keys, a face search key 36 is arranged in the operation input unit 23 as shown in FIG.

  The program reproducing apparatus 1 performs tuning and demodulation of the broadcast wave by the tuner / demodulator 10, decrypts the demodulated stream by the descrambler 11, and then demultiplexes the decrypted stream data by the demultiplexer 12. A video stream, an audio stream, and a section including data such as SI information and program identification information are separated. Note that the stream data may include a data stream (for example, data related to captions). In this case, the demultiplexing unit 12 also separates this data stream. The video stream is decoded into a video signal by the video decoding unit 13, and the video signal is displayed on the display unit 26 via the display control unit 25.

  FIG. 7 shows an example of a program playback screen in the present embodiment. Hereinafter, the operation of the program reproducing apparatus 1 in the present embodiment will be described with reference to FIG. As shown in FIG. 7, the viewer presses the face search key 36 of the operation input unit 23 of the program playback device 1 in step S101 while viewing a program in which four persons are shown. The operation input unit 23 transmits the code information of the pressed key to the operation analysis unit 24. The operation analysis unit 24 requests the GUI control unit 22 to validate the GUI screen of the face search function from the received key code information. The GUI control unit 22 requests the image extraction unit 17 to cut out a face image from the video being output, and simultaneously requests the display control unit 25 to freeze the image.

  In step S <b> 102, the image extraction unit 17 cuts out a face image from an arbitrary frame included in the video signal decoded by the video decoding unit 13. FIG. 8 shows an example of a face image cutout area in the present embodiment. In this example, it is assumed that four faces are detected and the four areas A1, A2, A3, and A4 shown in FIG. 8 are cut out. The image extraction unit 17 generates an extracted image table including coordinate values indicating where the cut-out area is located on the screen. In addition, images of four regions cut out together are generated as individual extracted image files.

  FIG. 9 is an example of an extracted image table in the present embodiment. The elements of the extracted image table include a horizontal coordinate position and a vertical coordinate position for indicating the position of the upper left corner of each of the A1 to A4 areas in the screen, and a horizontal size and a vertical size for indicating the sizes of the areas A1 to A4. And image IDs for specifying each of the areas A1 to A4. The rightmost column shows a code number for distinguishing the region for convenience of explanation, but it is not necessary as an element of the actual extracted image table. As shown in FIG. 8, the coordinate value increases as the horizontal direction advances to the right with the upper left corner of the entire screen as the origin (coordinate 0). In the vertical direction, the coordinate value increases as it goes downward.

  The coordinates are based on the number of pixels. In this example, an image of full high-definition (horizontal direction: 1920 pixels × vertical direction: 1080 pixels) is assumed, so the horizontal position coordinate at the right end is 1919 and the vertical position coordinate at the lower end is 1079. In the extracted image table of FIG. 9, area A1 indicates that the image ID is 1, the position coordinates of the upper left corner are horizontal position coordinates = 320, vertical position coordinates = 212, horizontal size = 330 pixels, and vertical size 340 pixels. ing. Further, in this extracted image table, the frame ID and the extracted image table are associated with each other in order to indicate which frame in the program corresponds to. The frame ID is a number assigned so that each frame can be uniquely specified in a certain long period (for example, a program unit). For example, this can be realized by assigning a counter value that is incremented by 1 for each frame during decoding.

  In step S103, the GUI control unit 22 refers to the extracted image table, and generates marker graphics data for distinguishing the region where the face is displayed so that the viewer can distinguish and select the face image. Then, in order to designate a frame in which the generated marker is superimposed and displayed, the graphics data of the marker is delivered to the display control unit 25 together with the frame ID associated with the extracted image table. The display control unit 25 displays a frame corresponding to the received frame ID on the display unit 26 as a still image, and displays the graphic data of the marker on the frame. FIG. 10 is an example of an extraction image area selection screen in which graphics data is displayed superimposed on a frame in this way.

  In the example of FIG. 10, an ellipse surrounding the periphery of the face is arranged, and numbers “1”, “2”, “3”, and “4” are added and displayed so that the viewer can distinguish them. Yes. FIG. 11 is an example of marker setting for the extracted image region. As shown in FIG. 11, this ellipse is calculated and displayed as an ellipse inscribed in the rectangle of the area A2 from four variables of the horizontal coordinate position, the vertical coordinate position, the horizontal size, and the vertical size in the extracted image table. According to this method, even in a scene where faces are close to each other, it is difficult for the markers to overlap each other, and an effect of increasing the visibility by the viewer can be obtained.

  In step S104, the viewer selects a face to be searched. As a specific operation, a number corresponding to a desired face is selected by pressing a number key of the operation input unit 23 while viewing the selection screen of the extracted image area shown in FIG. For example, when the number 2 key 42 is pressed, the face corresponding to the area A2 is selected. With this method, the viewer can select a desired face only by pressing one of the remote control buttons of the operation input unit 23 once, and an effect that the operation becomes easy can be obtained. For example, in the scene where the faces are not aligned vertically or horizontally as in the screen of FIG. 10, there is a problem that it is difficult to know the direction in which the focus moves when the cross key is pressed (for example, the focus is 1). If the right key 33 is pressed when the key is hit, it is difficult to know whether the focus moves to 2 or 3.) However, an individual number is assigned to the marker as in the present embodiment, and the user can directly input this number with the numeric keys, so that the viewer can select a desired face without hesitation. can get.

  In step S104, the operation analysis unit 24 receives the code of the pressed numeric key and notifies the GUI control unit 22 that the number “2” has been selected from the received key code information. The GUI control unit 22 delivers the image ID of the extracted image selected by the viewer to the image search collation unit 18 in accordance with the notification that the number “2” is selected. At the same time, the GUI control unit requests the display control unit 25 to end image stillness. Upon receiving this request, the display control unit 25 ends the superimposed display of the marker graphics and displays the program video. Resume playback display. As a result, the viewer can recognize that his / her selection operation has been accepted.

  In step S <b> 105, the image search collation unit 18 requests the image search key generation unit 16 to generate an image search key. The image search key generation unit 16 obtains the program identification information extracted from the section data by the program data extraction unit 14, identifies the program currently being viewed, and similarly extracts from the section data by the program data extraction unit 14. An image search key list including a list of performer names is generated with reference to the program arrangement information (SI information). FIG. 12 is an example of an image search key list in the present embodiment. The image search key generation unit 16 delivers the generated image search key list to the image search collation unit 18.

  In step S106, the image search key generation unit 16 refers to the image search key list, and first selects the name of the performer registered at the top (“Kenji Oda” of the image search key number 1 in the example of FIG. 12). The message creation command for the reference image acquisition request is issued to the communication protocol processing unit 20. The communication protocol unit 20 generates a reference image acquisition request message according to a predetermined protocol. The predetermined protocol in this case is, for example, a GET method configured according to HTTP (Hypertext Transfer Protocol). FIG. 13 is an example of a reference image acquisition request message and a reference image acquisition response message. In this message, refimg. A reference image named bmp is requested. Thereafter, the communication protocol unit 20 transmits this reference image acquisition request message to the image database 3 via the network communication unit 21 via the network.

  The image database 3 analyzes the received reference image acquisition request message, and transmits the reference image data matching “Kenji Oda” to the network communication unit 21 of the program playback device 1 as a reference image acquisition response message. FIG. 13 also shows an example of a reference image acquisition response message. This message adopts a response message format for the HTTP GET method. Usually, one reference image data is used, but a plurality of reference image data may be used in order to improve collation accuracy.

  In step S <b> 107, the communication protocol process 20 of the program playback device 1 receives the reference image acquisition response message via the network communication unit 21. The communication protocol process 20 extracts only the reference image data from the reference image acquisition response message and delivers it to the image search collation unit 18. Based on the extracted image file and the extracted image table generated by the image extracting unit 17 in step S102, the image search collating unit 18 acquired in step S106 from the image ID of the extracted image delivered from the GUI control unit 22 in step S104. One extracted image file to be collated with the reference image data is determined, and collation with the reference image data is determined.

  As a method for collation determination between the extracted image file and the reference image data, an algorithm such as a proper face method or an LFA (Local Feature Analysis) method, which is generally known as a face collation processing technique, can be applied. . In these methods, if the score value of the collation result, such as the similarity between the extracted image file and the reference image data, is equal to or greater than a certain threshold, it is determined that they match.

  If the collation determination result is “does not match the reference image” in step S108, that is, if the collation result score does not exceed the predetermined threshold, the process proceeds to step S109. In step S109, if the image search key number is not a value indicating the last image in the image search key list, for example, if the image search key number in the image search key list in FIG. The process proceeds to S110. In step S110, the image search key number is incremented by 1, and the process proceeds to step S106 again.

  If the collation determination result is “matches reference image” in step S108, that is, if the collation result score exceeds a predetermined threshold, the process proceeds to step S111.

  In step S111, the image search collation unit 18 delivers a search key that matches the reference image to the information search control unit 19, and makes an information search request.

  In step S112, the information search control unit 19 issues an information acquisition request message creation command to the communication protocol processing unit 20 based on the received search key. The communication protocol unit 20 generates an information acquisition request message according to a predetermined protocol. The predetermined protocol in this case is, for example, a POST method configured according to HTTP. FIG. 14 is an example of an information acquisition request message. In this example, a message is generated by embedding a search key in an XML (eXtended Markup Language) format document. Thereafter, the communication protocol unit 20 transmits this information acquisition request message to the information database 4 via the network via the network communication unit 21.

  The information database 4 analyzes the received information acquisition request message, and transmits, for example, profile data matching “Oda Kenji” as an information acquisition response message to the program playback device 1. FIG. 15 is an example of an information acquisition request response message. In this example, the message adopts a response message format for the HTTP POST method. The profile data employs an XML text document structure, and information such as the real name, age, sex, career, and representative work is embedded in the XML document as data structured by tags.

  In step S <b> 113, the communication protocol processing unit 20 of the program reproduction device 1 receives the information acquisition response message via the network communication unit 21. The communication protocol processing unit 20 extracts only the profile data in the XML format from the information acquisition response message and passes it to the image search collation unit 18.

  The image search / collation unit 18 analyzes the received profile data in the XML format, extracts individual elements surrounded by tags, and delivers them to the GUI control unit 22. The GUI display unit 22 generates graphics data for displaying the profile and delivers it to the display control unit 25. The display control unit 25 displays the graphic data of the profile display so as to overlap the program reproduction screen. FIG. 16 is an example of an information display screen in the present embodiment.

  In the program playback apparatus configured as described above, a marker display that clearly indicates the position of one or a plurality of image data extracted from the video of the program is displayed so as to overlap the original frame from which the image data is extracted. Selects one image data based on the marker display and collates the image data with the reference image to identify the attribute of the image data selected by the viewer and view it via the network based on the attribute. Since the information of the image data selected by the viewer can be acquired, the viewer can easily select a specific target from a plurality of targets included in the video of the program and quickly obtain the information of the target Is possible.

Embodiment 2. FIG.
In the first embodiment, in step S102 shown in FIG. 3, the image extraction unit 17 cuts out a face image from an arbitrary frame included in the video signal decoded by the video decoding unit 13. In some cases, the face image subjected to the correction is not suitable for face collation, such as a small size. In such a case, an error may occur in the collation determination between the extracted image and the reference image data executed in the subsequent step S107. In the second embodiment, a method capable of avoiding such a problem in advance will be described.

  FIG. 17 is a diagram showing an operation flow of the program reproducing apparatus of the present embodiment. In step S <b> 101, the viewer presses the face search key 36 of the operation input unit 23 of the program playback device 1 while viewing a program in which four people are shown as shown in FIG. 7. The operation input unit 23 transmits the code information of the pressed key to the operation analysis unit 24. The operation analysis unit 24 requests the GUI control unit 22 to validate the GUI screen of the face search function from the received key code information. The GUI control unit 22 requests the image extraction unit 17 to cut out a face image from the video being output, and simultaneously requests the display control unit 25 to freeze the image.

  In step S102, the image extraction unit 17 cuts out a face image from an arbitrary frame included in the video signal decoded by the video decoding unit 13. In the present embodiment, it is assumed that four faces are detected and the four areas A1, A2, A3, and A4 shown in FIG. The image extraction unit 17 generates an extracted image table including coordinate values indicating where the cut-out area is located on the screen. In addition, images of four regions cut out together are generated as individual extracted image files.

  The image extraction unit 17 evaluates whether each of the extracted four extracted image files is appropriate for matching with reference image data. Appropriate evaluation parameters and determination criteria are used according to the performance of the image search / collation unit 18 for collation. For example, the number of pixels of the extracted image, average luminance, and the like. When the size of the extracted image is used as the evaluation parameter, PHref and PVref are set as threshold values for the number of pixels in the horizontal direction and the number of pixels in the vertical direction, respectively, and the number of pixels in the horizontal direction of the extracted image is PHext, When the number of pixels is PVext, if one or both of the following inequality (1) or inequality (29) is not satisfied, it is determined that it is not appropriate for collation.

PHref> PHext (1)
PVref> PVext (2)

  When average luminance is used as the evaluation parameter, the average luminance value of all pixels in the image is defined as average luminance. When the threshold value of the average luminance is set to Bref and the average luminance of the extracted image is set to Bext, if the condition shown in the following inequality (3) is not satisfied, it is determined that it is not appropriate for collation.

Bref> Bext (3)

  Combining the above three inequalities (1), (2), and (3), only when all the conditions are satisfied, it is determined that it is appropriate for collation, otherwise it is determined that it is not appropriate for collation Good.

  In step S201, the image extraction unit 17 determines whether each of the extracted four extracted image files is appropriate for matching with reference image data. If it is determined that matching is appropriate for matching, the process proceeds to step S202 and extraction is performed. Set the collation valid flag of the image table to ON. If it is determined that it is not appropriate for collation, the process proceeds to step S203, and the collation valid flag of the extracted image table is set to OFF.

  FIG. 18 is an example of an extracted image table in the present embodiment. In this example, the area A1, the area A2, and the area A4 are determined to be appropriate for collation, and the area A3 is determined to be inappropriate for collation.

  In step S103, the GUI control unit 22 refers to the extracted image table, and generates marker graphics data for distinguishing the region where the face is displayed so that the viewer can distinguish and select the face image. To do. At this time, the markers are generated only for the areas A1, A2, and A4 that are determined to be appropriate for collation.

  Thereafter, the GUI control unit 22 delivers the graphics data of the marker to the display control unit 25 together with the frame ID associated with the extracted image table in order to designate a frame in which the generated marker is displayed in a superimposed manner. The display control unit 25 displays a frame corresponding to the received frame ID on the display unit 26 as a still image, and displays the marker graphics data on the frame. Thereafter, the process proceeds to step S104, and the subsequent steps are the same as those described in FIG. 3 of the first embodiment.

  FIG. 19 shows an example of an extraction image region selection screen in the present embodiment. A region A3 indicated by reference numeral 3 shown in FIG. 10 different from the first embodiment is determined not to be suitable for collation in the present embodiment. As a result, as shown in FIG. 19, the code 3 marker shown in FIG. 10 is not displayed, and the viewer cannot select this image (face).

  In the program reproducing apparatus configured as described above, when the viewer selects a face to be searched in step S104, it is impossible to select a face in an area determined to be unsuitable for matching. It is possible to avoid the occurrence of an error in the collation determination with the extracted image and reference image data to be executed.

Embodiment 3 FIG.
FIG. 20 is a block configuration diagram showing an example of the configuration of the program playback device 1 according to Embodiment 3 of the present invention. The program playback device of the present embodiment is obtained by adding an image search key database 15 to the image search key generation unit 16 with respect to the program playback device shown in the first embodiment. The image search key database 15 is a list of performers for a specific program that can be updated at any time by means such as a network, and is held in a memory or the like. FIG. 21 is an example of an image search key database in the present embodiment.

  In step S105 in the flowchart shown in FIG. 3 according to the first embodiment, the image search collating unit 18 requests the image search key generating unit 16 to generate an image search key. The image search key generation unit 16 obtains the program identification information extracted from the section data by the program data extraction unit 14 to identify the currently viewed program, and then the program of the currently viewed program with respect to the image search key database 15. Perform a list search request for performers. The image search key database 15 delivers the performer list of the program searched using the designated program name as a key to the image search key generation unit 16. Based on this performer list, the image search key generating unit 16 generates an image search key list composed of a list of performer names. The subsequent steps are the same as those described in FIG. 3 of the first embodiment.

  In the program playback apparatus configured in this way, the image search key database 15 can update the list of performers for a specific program at any time by means of a network or the like, so that it is more abundant than the information included in the program sequence information. Since the information of the image search key can be obtained (for example, including the name of the performer of the end role), more performers can be specified.

DESCRIPTION OF SYMBOLS 1 Program reproduction apparatus 2 Network 3 Image database 4 Information database 10 Tuner / demodulation part 11 Descramble part 12 Demultiplex part 13 Video decoding part 14 Program data extraction part 15 Image search key database 16 Image search key generation part 17 Image extraction part DESCRIPTION OF SYMBOLS 18 Image search collation part 19 Information search control part 20 Communication protocol processing part 21 Network communication part 22 GUI control part 23 Operation input part 24 Operation analysis part 25 Display control part 26 Display part 27 Voice decoding part 28 Voice output part 31 Up key 32 Down key 33 Left key 34 Right key 35 Enter key 36 Face search keys 41-52 Number keys

Claims (18)

A program playback device for playing back a received program,
A communication means for communicating with an external server via a network;
Demultiplexing means for separating the transmission stream of the program into a video stream, an audio stream and a data section;
Video decoding means for decoding the video stream and converting it into a video signal;
Image extracting means for cutting out one or a plurality of regions from one frame included in the video signal and extracting them as image data;
GUI control means for generating a marker that clearly indicates the position of the extracted one or a plurality of image data in the one frame;
Display control means for displaying the marker superimposed on the one frame;
An operation means for selecting one image data from the one place or a plurality of image data using the marker by a viewer;
Program data extraction means for extracting program arrangement information and program identification information included in the data section;
Image search key generation means for generating an image search key for searching for a reference image from the program arrangement information and the program identification information;
Using the image search key, the communication means obtains the reference image from the external server via the network, and the image search collating means for collating the image data with the reference image;
Information search means for generating an information search key indicating an attribute of the image data based on the collation result, and acquiring information related to the image data via the network by the communication means using the information search key. A program playback apparatus characterized by that. The area of the image data cut out by the image extraction means is rectangular,
2. The program reproducing apparatus according to claim 1, wherein the marker generated by the GUI control means has a circular shape inscribed in a rectangular area of the image data. 3. The program reproducing apparatus according to claim 1, wherein the marker generated by the GUI control unit is assigned an individual number to one or a plurality of extracted image data. After the marker is generated by the GUI control means,
4. The program according to claim 1, wherein one frame displayed by the display control means is displayed as a still image until selection screen data is selected by the operation means. Playback device. When at least one of the number of pixels in the horizontal direction of the image data and the number of pixels in the vertical direction of the image data is equal to or less than a predetermined threshold, the image extraction means is not suitable for collation with the reference image The program playback device according to claim 1, wherein the program playback device is determined. The image extraction means determines that the image data is not suitable for collation with a reference image when the average luminance of the image data is equal to or less than a predetermined threshold value. The program playback device according to Item. The image extraction means is configured to detect when at least one of the number of pixels in the horizontal direction of the image data and the number of pixels in the vertical direction of the image data is equal to or smaller than a predetermined threshold value, or when the average luminance of the image data is equal to or smaller than a predetermined threshold value. The program playback apparatus according to claim 1, wherein the image data is determined not to be suitable for collation with a reference image. If the image extraction means determines that the image data is not appropriate for matching with the reference image,
8. The program reproducing apparatus according to claim 5, wherein the GUI control unit does not generate a marker display corresponding to the image data determined not to be suitable for collation. 9. A program playback device for playing back a received program,
A communication means for communicating with an external server via a network;
Demultiplexing means for separating the transmission stream of the program into a video stream, an audio stream and a data section;
Video decoding means for decoding the video stream and converting it into a video signal;
Image extracting means for cutting out one or a plurality of regions from one frame included in the video signal and extracting them as image data;
GUI control means for generating a marker that clearly indicates the position of the extracted one or a plurality of image data in the one frame;
Display control means for displaying the marker in a superimposed manner on the one frame;
An operation means for selecting one image data from the one place or a plurality of image data using the marker by a viewer;
Program data extracting means for extracting program specific information included in the data section;
Database holding means for holding program information relating to the program as a database;
Image search key generation means for generating an image search key for searching for a reference image from the database and the program specifying information;
Using the image search key, the communication means obtains the reference image from the external server via the network, and the image search collating means for collating the image data with the reference image;
Information search means for generating an information search key indicating an attribute of the image data based on the collation result, and acquiring information related to the image data via the network by the communication means using the information search key. A program playback apparatus characterized by that. A receiving step for receiving a program;
A demultiplexing step of separating the received transmission stream of the program into a video stream, an audio stream and a data section;
A video decoding step of decoding the video stream and converting it into a video signal;
An image extraction step of cutting out one or a plurality of regions from one frame included in the video signal and extracting them as image data;
A GUI control step of generating a marker that clearly indicates the position of the extracted one or more image data in the one frame;
A display control step of displaying the marker superimposed on the one frame;
An operation step in which one selected image data is selected from the one place or a plurality of image data using the marker by a viewer;
A program data extraction step for extracting program arrangement information and program identification information included in the data section;
An image search key generating step for generating an image search key for searching for a reference image from the program arrangement information and the program specifying information;
Using the image search key, obtaining the reference image from an external server via a network, and an image search collating step of collating the selected image data with the reference image;
An information search step of generating an information search key indicating an attribute of the selected image data based on the matching result, and acquiring information related to the selected image data via the network using the information search key. A program playback method characterized by the above. The area of the image data cut out by the image extraction step is a rectangular shape,
11. The program reproducing method according to claim 10, wherein the marker display generated by the GUI control step has a circular shape inscribed in a rectangular region of the image data. 12. The program reproduction method according to claim 10 or 11, wherein the marker generated by the GUI control step is assigned an individual number to one or a plurality of extracted image data. After the marker is generated by the GUI control step,
13. The program according to claim 10, wherein one frame displayed by the display control step is displayed as a still image until selection screen data is selected by the operation step. Playback method. In the image extraction step, if at least one of the number of pixels in the horizontal direction of the image data and the number of pixels in the vertical direction of the image data is equal to or less than a predetermined threshold, the image data is not suitable for collation with the reference image. The program playback method according to claim 10, wherein the program playback method is determined. The image extracting step determines that the image data is not suitable for collation with a reference image when the average luminance of the image data is equal to or less than a predetermined threshold value. The program playback method according to item. The image extraction step is performed when at least one of the number of pixels in the horizontal direction of the image data and the number of pixels in the vertical direction of the image data is equal to or smaller than a predetermined threshold value, or when the average luminance of the image data is equal to or smaller than the predetermined threshold value. The program playback method according to claim 10, wherein the image data is determined not to be suitable for collation with a reference image. If the image extraction step determines that the image data is not suitable for matching with the reference image,
The program reproduction method according to any one of claims 14 to 16, wherein the GUI control step does not generate a marker corresponding to the image data determined not to be suitable for collation. A receiving step for receiving a program;
A demultiplexing step of separating the received transmission stream of the program into a video stream, an audio stream and a data section;
A video decoding step of decoding the video stream and converting it into a video signal;
An image extraction step of cutting out one or a plurality of regions from one frame included in the video signal and extracting them as image data;
A GUI control step of generating a marker that clearly indicates the position of the extracted one or more image data in the one frame;
A display control step of displaying the marker superimposed on the one frame;
An operation step in which one selected image data is selected from the one place or a plurality of image data using the marker by a viewer;
A program data extraction step for extracting program specific information included in the data section;
A database holding step of holding program information relating to the program as a database;
An image search key generation step of generating an image search key for searching for a reference image from the database and the program specifying information;
Using the image search key, obtaining the reference image from an external server via a network, and an image search collating step of collating the selected image data with the reference image;
An information search step of generating an information search key indicating an attribute of the selected image data based on the matching result, and acquiring information related to the selected image data via the network using the information search key. A program playback method characterized by the above.

Images