JP2013126170A - Video scene presentation method, device, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To solve the problem in which, when a user instructs to process a content (scene) constituting images being reproduced, it is not necessarily possible to designate a content (scene) the user wants to process.SOLUTION: Weighting is performed in accordance with an operation for a content, importance of subject information on the content is calculated, the importance of the subject information is corrected based on information on a position where processing of the content is instructed, and an optimum content is presented based on the corrected importance of the subject information.

Description

  The present invention relates to a technique for accepting and presenting designation of content constituting an image such as a moving image or a slide show being reproduced.

In recent years, with the increase in storage capacity of hard disk drives and flash media, a large amount of images such as still images and moving images have been registered in devices equipped with storage. In addition, a large amount of images such as still images and moving images have been registered in network storage such as cloud services and Web servers. When a process such as printing is instructed for the content that is being reproduced as described above, the process such as printing is performed on the instructed content.
Hereinafter, in the case of a moving image, the content refers to a scene showing an arbitrary segment that constitutes the moving image. In the case of a slide show, the individual still images constituting the slide show are indicated.
For example, a moving image distributed from a server is played back by a playback device and simultaneously received by a printing device connected to the playback device. The printing apparatus temporarily stores the received predetermined content. Then, in response to a print instruction from the playback device, the temporarily stored content is printed (see, for example, Patent Document 1).

JP 2005-198030 A

However, in the case of the above-described prior art, it may be difficult to specify a content to be printed or the like during image reproduction. For example, when the user is unfamiliar with the operation of the remote controller or the like, it is difficult to specify the desired content to be printed. In addition, it is difficult to indicate content shot from the front of a person during playback of a moving image.
Also, there may be content that suits the user's taste more than content that indicates an image being reproduced. For example, when a plurality of users photograph the same subject with a digital video camera and a digital camera, there may be a case where a desired content exists in the digital camera content rather than the digital video camera content.
Accordingly, the present invention has been made to solve the above-described problems, and presents the content to be processed in consideration of the operation history and the time lag of the instruction operation when instructing the content processing during image reproduction. . It is another object of the present invention to present content that is optimal for processing related to content to be processed.

In order to achieve the above object, a moving picture presentation apparatus according to the present invention comprises the following arrangement.
That is, an operation history holding unit that holds an operation history for each scene of the moving image, a determination unit that determines the importance of subject information appearing in each scene of the moving image using the operation history and its weight, and the moving image Specifying means for specifying the scene, and determining means for determining the scene to be presented based on the importance of the subject information appearing in the specified scene and the neighboring scenes.

  According to the present invention, when processing of moving image content (scene) is instructed, the processing target content intended by the user can be determined in consideration of the time lag of the instruction operation.

It is a block diagram which shows the content presentation apparatus to which embodiment of this invention is applied. It is a block diagram which shows the function structure of the content presentation apparatus in embodiment of this invention. It is a flowchart which shows operation | movement of the content presentation apparatus in embodiment of this invention. 4 is a flowchart for explaining in detail the operation of step S305 of FIG. 3 in the embodiment of the present invention. It is a flowchart explaining in detail the operation of step S306 of FIG. 3 in the embodiment of the present invention. It is a figure which shows the weighting value with respect to operation of the content in image reproduction in embodiment of this invention. It is a figure which shows the importance according to operation in the content reproduction | regeneration for every object information in embodiment of this invention. It is a figure which shows designation | designated of the process target in embodiment of this invention, and the importance of the object information after correction | amendment. It is a flowchart which shows the operation | movement of the count addition for every composition information at the time of printing in Embodiment 2 of this invention. It is a figure which shows the number of contents for every composition information at the time of printing in Embodiment 2 of this invention. It is a figure which shows the display of the optimal content in embodiment of this invention.

[Embodiment 1]
In the first embodiment of the present invention, a case in which processing of a still image among a plurality of still images that are contents constituting a slide show is instructed during playback of a slide show that is an image will be described. In the first embodiment of the present invention, a print process is instructed as a still image process.

  Hereinafter, Embodiment 1 of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a configuration diagram showing a content presentation device 100 to which an embodiment of the present invention is applied. In FIG. 1, reference numeral 101 denotes a microprocessor CPU (Central Processing Unit), which performs operations for information processing, logical judgments, etc., and controls each component connected to those buses via a system bus 111. . Reference numeral 102 denotes a read-only fixed memory ROM (Read Only Memory), which stores control program codes such as processing programs executed in this embodiment. Reference numeral 103 denotes a writable RAM (Random Access Memory), which is used for temporary storage of various data from each component. Reference numeral 104 denotes an input device, and a user instruction is input by the remote controller 105. Reference numeral 107 denotes a display device such as a liquid crystal panel, and the display controller 106 controls display of content information on the display device. An external storage device 108 stores content information. As a storage medium for storing these data, a memory card, HDD (Hard Disk Drive), DVD-RAM (Digital Versatile Disk Random Access Memory), or the like can be used. Reference numeral 109 denotes a communication device, which can be connected to a plurality of content presentation devices via a communication network 110 such as the Internet, a LAN (Local Area Network), or a wireless LAN.

  The content presenting device including such components operates in response to various inputs from the input device 104 and various inputs via the network supplied from the communication device 109. Therefore, when an input from the input device 104 and an input from the communication device 109 are supplied, an interrupt signal is first sent to the CPU 101. Then, the CPU 101 reads out various control signals stored in the ROM 102, RAM 103, and external storage device 108, and various controls are performed in accordance with these control signals.

  The present invention can also be achieved by supplying a storage medium storing a program according to the present invention to a system or apparatus, and the system or apparatus reading and executing the program code stored in the storage medium.

  FIG. 2 is a block diagram showing a functional configuration of the content presentation device 100 according to the embodiment of the present invention. In FIG. 2, reference numeral 200 denotes a content instruction unit which issues an image reproduction instruction and reproduction, a content operation instruction, and a processing instruction. Reference numeral 201 denotes an image reproduction instruction unit which designates images such as moving images and slide shows to be reproduced by the remote controller 105. An image reproduction unit 202 reproduces images such as a moving image and a slide show instructed by the image reproduction instruction unit 201. A content operation instruction unit 203 instructs the remote controller 105 to perform operations such as fast-forward and pause with respect to the content constituting the image being reproduced by the image reproduction unit 202. A content processing instruction unit 204 instructs the remote controller 105 to print, select a still image to be displayed on a photo frame, send an e-mail, and the like for the content of the image being reproduced by the image reproduction unit 202.

  An image importance calculation unit 210 calculates the importance of the subject based on the operation instructed by the content operation instruction unit 203 and the subject information of the operation target. An operation weight value acquisition unit 211 acquires a weight value corresponding to the operation instructed by the content operation instruction unit 203. Reference numeral 212 denotes an operation subject information acquisition unit that acquires subject information of content that is an operation target instructed by the content operation instruction unit 203. Reference numeral 213 denotes a subject importance calculation unit that calculates the importance of a subject based on the operation weight value acquired by the operation weight value acquisition unit 211 and the subject information acquired by the operation subject information acquisition unit 212.

  A content presentation unit 220 corrects the importance of the subject based on the difference information between the subject information of the processing target instructed by the content processing instruction unit 204 and the processing instruction position, and presents the optimum content for processing. To process. Reference numeral 221 denotes a subject information acquisition unit at the time of processing designation, which acquires subject information appearing in a plurality of contents including the content to be processed instructed by the content processing instruction unit 204. Reference numeral 222 denotes a position information acquisition unit, which is position information (in order of reproduction) of the content to be processed instructed by the content processing instruction unit 204 and the contents of the plurality of subject information acquired by the subject information acquisition unit 221 when processing is specified. Get distance). A subject importance level correction unit 223 corrects the importance level of the subject based on the subject information acquired by the subject information acquisition unit 221 at the time of processing designation and the position information acquired by the position information acquisition unit 222. A content acquisition unit 224 acquires optimal content based on the importance of the subject corrected by the subject importance correction unit 223. A content processing unit 225 processes the content acquired by the content acquisition unit 224.

  A communication unit 230 connects the content presentation apparatus 100 to a storage on the communication network 110 such as a cloud service or a Web server in which a large amount of image content such as still images and moving images is registered.

  FIG. 3 is a flowchart showing the operation of the content presentation apparatus in the embodiment of the present invention. Hereinafter, the flowchart is realized by the CPU 101 executing the control program.

  In step S <b> 301, in the content presentation device 100, an image instructed by the remote controller 105 by the image reproduction instruction unit 201 is reproduced by the image reproduction unit 202. The image to be reproduced may be an image registered in the external storage device 108 of the content presentation device 100 or an image registered in a storage on the communication network 110 such as a cloud service or a Web server via the communication unit 230. .

  In step S302, it is determined whether the content constituting the image being reproduced in step S301 has been operated by the remote controller 105. If operated, the process proceeds to step S303, and if not operated, the process is performed. The process proceeds to step S307. For example, the operation may be an operation for instructing processing such as content printing, or may be a content reproduction operation such as fast-forwarding or pausing.

  In step S303, it is determined whether or not the operation in step S302 is an operation for instructing the content processing instructed by the remote control 105 in the content processing instruction unit 204. If it is an operation for instructing content processing, the process proceeds to step S306. If it is not an operation for instructing content processing, the process proceeds to step S304. For example, the operation for instructing the content processing may be an operation for instructing a processing for printing the content, an operation for instructing a processing for selecting a still image to be displayed on the photo frame from the content, or the content by e-mail. The operation of instructing the process to be transmitted may be performed. In step S304, it is determined whether or not the operation in step S302 is a content reproduction operation instruction instructed by the remote control 105 in the content operation instruction unit 203. If the instruction is a content reproduction operation instruction, the process proceeds to step S305. If the instruction is not a content reproduction operation instruction, the process proceeds to step S307. For example, the content playback operation instruction may be fast-forwarding of content or may be paused.

  In step S305, processing is performed when the operation in step S302 is an instruction for content reproduction operation. The description will be given with reference to FIG.

  In step S306, processing is performed when the operation in step S302 is an operation for instructing content processing. The description will be given with reference to FIG.

  In step S307, it is determined whether or not the reproduction of the image has ended. If the reproduction has not ended, the process returns to step S301, and the reproduction of the image is continued. When the reproduction is finished, the process is finished.

  FIG. 4 is a flowchart for explaining in detail the operation of step S305 of FIG. 3 in the embodiment of the present invention.

  In step S401, a weighting value corresponding to the operation instructed in step S302 is acquired from the operation weighting value acquisition unit 211.

  For example, FIG. 6 shows a weighting value for the operation of content during image reproduction. As shown in FIG. 6, the weighting value for “normal reproduction” in the operation of content during image reproduction is “1.0”. In addition, the weight value is set to “10.0” when the content operation during image playback is “pause”, and the weight value is set to “5.0” when “frame advance” and “slow playback”. In the case of “double speed playback”, the weighting value is “0.5”, and in the case of “fast forward” and “fast reverse”, the weighting value is “0.1”.

  In step S402, the subject information of the content that is the operation target in step S401 is acquired from the operation subject information acquisition unit 212. For example, the subject information of the content that is the operation target may be obtained by acquiring subject information from the metadata attached to the content, or the subject is recognized using a recognition technique from a person's face or object. However, the method of using the subject information may be used.

  In step S403, the subject importance calculation unit calculates the importance of the subject information from the weighting value of the content corresponding to the operation acquired in step S401 and the subject information of the content that is the operation target acquired in step S402. FIG. 7 shows an example of the importance according to the operation during the content reproduction for each subject information. The weighting value according to the operation during the content reproduction for each subject information is added to calculate the importance of the subject information. For example, as shown in FIG. 7, the importance of the subject information “A” is “100”, the importance of the subject information “B” is “10”, and the importance of the subject information “C” is “10”. Show.

  FIG. 5 is a flowchart for explaining in detail the operation of step S306 of FIG. 3 in the embodiment of the present invention.

  In step S <b> 501, subject information of a plurality of contents including the content to be processed instructed by the content processing instruction unit 204 is acquired by the subject information acquisition unit 221 when processing is specified. For example, as shown in FIG. 8, when the content instructed as the processing target is 801, subject information of 10 contents including the content instructed as the processing target is acquired in the order of the latest reproduction. The acquired subject information of the 10 contents is “J” to “A”. When a plurality of pieces of subject information exist in one content, a method for processing each piece of subject information or a method for processing a plurality of pieces of subject information collectively may be used. In addition, the method for acquiring subject information of content may be a method of acquiring subject information from metadata attached to the content, or by recognizing a subject using a recognition technology from a person's face or object, and so on. It is also possible to use the method.

  In step S502, the position information acquisition unit 222 acquires position information about the content to be processed instructed by the content processing instruction unit 204 and the contents of the plurality of subject information acquired in step S501. For example, FIG. 8 shows that the content instructed as the processing target is 801 and the content of the subject information “J”. Further, the content of the subject information “I” is closest to the content of the subject information “J”, and the content of the subject information “H” is the next closest.

  In step S503, the importance of the subject information is corrected by the subject importance correction unit 223 based on the plurality of subject information acquired in step S501 and the position information acquired in step S502. For example, the importance of the subject information of the content close to the content instructed as the processing target is increased as indicated by 802 in FIG. The correction ratio of the content of the subject information “J” instructed as the processing target is set to “100%”, and the correction ratio of the content of the subject information “I” in the order closer to the content of the subject information “J” instructed as the processing target. “80%”. Further, the correction ratio of the subject information “H” content is “60%”, the correction ratio of the subject information “G” content is “40%”, the correction ratio of the subject information “F” content is “20%”, Others are invalid. Then, the correction ratio is added to the importance of the subject information to correct the importance of the subject information. As the importance of the subject information after correction, the importance of the subject information “J” instructed as the processing target is “10.0”, and the importance of the subject information “I” is “8.0”. ing. Further, the importance of the subject information “H” is “18.0”, the importance of the subject information “G” is “6.0”, and the importance of the subject information “F” is “12.0”. Show. In step S504, the content acquisition unit 224 acquires the optimum content based on the importance of the subject information corrected in step S503. For example, as shown in FIG. 8, three pieces of subject information “H” content, subject information “F” content, and subject information “G” content with high importance of subject information corrected as shown in FIG. Yes.

  In step S505, the content processing unit 225 performs the process instructed in step S302 on the content acquired in step S504. For example, as shown in FIG. 11, the content acquired as the optimum content for the content presentation device is displayed, and the content selected from the content is printed. Reference numerals 1101, 1102, and 1103 indicate contents acquired as optimum contents. 1101 indicates the contents of the subject information “H”, 1102 indicates the contents of the subject information “F”, and 1103 indicates the contents of the subject information “G”. ing.

  According to the first embodiment of the present invention, when the user instructs the processing of the content constituting the image being reproduced, the system weights in advance according to the operation on the content, and the importance of the subject information of the content is determined. calculate. Then, the importance of the subject information is corrected based on the position information instructed to process the content (the distance between the content and the content), and the optimum content can be presented based on the importance of the corrected subject information. it can. As a result, it is possible to determine the processing target content intended by the user.

[Embodiment 2]
In the first embodiment of the present invention, the case where the optimum content is presented based on the importance of the subject information has been described. In the second embodiment of the present invention, the case where the optimum content is presented based on the composition information is described. To do.

  The flowchart showing the operation of the content presentation device in Embodiment 2 of the present invention is the same as that in FIG. Further, the flowchart showing the operation in the case of the operation for instructing the content processing in the second embodiment of the present invention is the same as FIG.

  In the second embodiment of the present invention, in step S302, the content processing instruction unit 204 instructs the remote control 105 to perform print processing as content processing.

  In step S504, a search is performed based on the composition information, and the search result is acquired by the content acquisition unit 224 as the optimum content. For example, the composition information of the content previously printed by the user is used. In FIG. 10, “person is center” and “within 3 persons” having a large number of contents are acquired in the composition information of the user. Then, a search is made for content having “person is center” or “person is within 3” having a large number of contents in composition information.

  In step S505, the content processing unit 225 performs the printing process instructed in step S302 on the content acquired in step S504.

  FIG. 9 is a flowchart showing a count addition operation for each piece of composition information during printing according to the second embodiment of the present invention.

  In step S901, composition information of content printed by the user is acquired. For example, as the composition information, “person is center”, “person is front-facing”, “within 3 persons”, and the like are acquired.

  In step S902, a count is added for each piece of composition information acquired in step S901. For example, as a result of counting for each piece of composition information, the number of contents “100” of composition information “person is center”, the number of contents “15” of composition information “person is facing front” as shown in FIG. This indicates that the number of contents is “30” or less.

  According to the second embodiment of the present invention, the system acquires composition information when the user prints in advance by instructing the print processing of the content constituting the image being reproduced. By searching based on the composition information, the search result can be presented as the optimum content for the printing process. As a result, it is possible to present optimal related content that suits the user's preference more than the content to be processed being reproduced.

[Embodiment 3]
In the first and second embodiments of the present invention, the case where the optimum content is presented based on the importance of the subject information at the time of content operation and the composition information at the time of printing has been described. In Embodiment 3 of the present invention, while an image is being reproduced by the content presentation device, a print process is instructed in the content constituting the image. A case will be described in which content instructed as a processing target and one or more reproduced content in the vicinity are acquired, and content that is optimal for processing is acquired from the acquired content.

  In Embodiment 3 of the present invention, when an image is reproduced by the content presentation device, if there is a favorite content in the image being reproduced, if the print processing is instructed, the content instructed as the processing target and the adjacent reproduction Get one or more pieces of content. Judgment is made by using the image quality information in order to obtain the optimum content for the printing process among the obtained content. For example, if it is determined that the reproduced content in the vicinity is more suitable as the image quality information than the content instructed as the processing target, the reproduced content in the vicinity is acquired.

  Further, the determination of content suitable for print processing is not limited to image quality information, and resolution information may be used, or composition information may be used.

  According to the third embodiment of the present invention, it is possible to present content more suitable for the instructed processing than the content instructed as the processing target.

[Embodiment 4]
In the first to third embodiments of the present invention, the printing process has been described as the content process. In the fourth embodiment of the present invention, a case where content processing is processing for selecting a still image to be displayed in a photo frame and a case of e-mail transmission processing will be described.

  When processing for selecting a still image to be displayed in a photo frame is instructed as content processing, the history when a still image to be displayed in a photo frame is selected is used. When an e-mail transmission process is instructed, the e-mail transmission history is used.

  According to the fourth embodiment of the present invention, the user instructs the process of selecting a still image to be displayed in the photo frame of the content constituting the image being reproduced and the e-mail transmission process. In any process, it is possible to present related content that is more suitable for the user's preference than the content to be processed being reproduced.

[Embodiment 5]
In the first embodiment of the present invention, the case where the image is a slide show has been described, but in the fifth embodiment of the present invention, the case where the image is a moving image will be described.

  In the case of a moving image, during the reproduction of the moving image, the processing of a scene indicating an arbitrary segment as content is instructed. The subject information of a scene may be a method of acquiring subject information from metadata attached to the scene, or a method of recognizing a subject from a representative image of the scene using a recognition technique to obtain subject information. The reference for acquiring the position information in the reproduction order may be the start point of the scene, the intermediate point of the scene, the end point of the scene, or the position of the representative image of the scene. Others are the same as in the case of the slide show.

  According to the fifth embodiment of the present invention, even when an image is a moving image, it is possible to present related content that is more suitable for the user's preference than the content to be processed that is being reproduced.

[Embodiment 6]
In the first to fifth embodiments of the present invention, the configuration of only the content presentation device has been described. In the sixth embodiment of the present invention, the content presentation device and the configuration of storage on a communication network such as a cloud service or a Web server in which a large amount of image content such as still images and moving images connected to the content presentation device are registered will be described. To do.

  The flowchart showing the operation in the case of the operation for instructing the content processing in the sixth embodiment of the present invention is the same as FIG.

  In the sixth embodiment of the present invention, in step S504, a search is performed based on composition information, image quality information, and resolution information, and the search result is acquired by the content acquisition unit 224 as optimal content.

  In the sixth embodiment of the present invention, the search range may be within the content presentation device 100, or a cloud service or web server in which a large amount of image content such as still images and moving images connected to the content presentation device 100 is registered. Storage on the communication network 110 may be used.

  For example, when the same event is shot with a digital camera and digital video, the still image shot with a digital camera that exists in the storage on the communication network matches the user's preference rather than the content of the video shot with the digital video being played back. The image can be presented as the optimum related content.

  According to the sixth embodiment of the present invention, reproduction is performed by searching storage on a communication network such as a cloud service or a Web server in which a large amount of image content such as still images and moving images connected to the content presentation device is registered. It is possible to present the relevant content that best suits the user's preference rather than the content of the image inside.

[Embodiment 7]
In the first to sixth embodiments of the present invention, a case has been described in which related content that is optimal for a content presentation device that is reproducing an image is presented. In the seventh embodiment of the present invention, a partner connected via a communication network is described. A case where the content is presented on the content presentation device will be described.

  In the seventh embodiment of the present invention, the content presentation device 100 is connected to a plurality of content presentation devices via the communication network 110.

  In Embodiment 7 of the present invention, when a moving image is played back by the content presentation device, if there is content to be presented to the content presentation device of the other party in the image being reproduced, the other party is instructed by instructing display processing. The content is displayed on the content presentation device. In addition, the content may be printed on a printer connected to the content presentation apparatus of the other party by instructing the printing process, or the photo connected to the content presentation apparatus of the other party by instructing the photo frame display process. The content may be displayed in a frame. Further, the destination may be a specific content presentation device or a plurality of content presentation devices.

  According to the seventh embodiment of the present invention, it is possible to present content that is optimal for processing such as printing to a content presentation device that is connected via a communication network.

[Embodiment 8]
In the first and second embodiments of the present invention, the case has been described where optimum related content that suits the user's preference on the content presentation device side that is reproducing an image is presented. In the eighth embodiment of the present invention, a description will be given of a case where optimum related content that suits the user's preference on the side of the content presentation apparatus connected via the communication network is presented.

  The flowchart showing the operation in the case of the operation for instructing the content processing in the eighth embodiment of the present invention is the same as FIG.

  In the eighth embodiment of the present invention, in step S504, a search is performed based on composition information, image quality information, and resolution information, and the search result is acquired by the content acquisition unit 224 as optimal content.

  In the eighth embodiment of the present invention, the user's preference information is used as composition information, image quality information, and resolution information. Therefore, composition information, image quality information, and resolution information are acquired in advance as a user profile.

  In the eighth embodiment of the present invention, it is possible to present the optimum related content that suits the preference of the other party connected via the communication network.

DESCRIPTION OF SYMBOLS 200 Content instruction | indication part 201 Image reproduction instruction part 202 Image reproduction part 203 Content operation instruction part 204 Content processing instruction part 210 Image importance calculation part 211 Operation weight value acquisition part 212 Operation subject information acquisition part 213 Subject importance calculation part 220 Content Presentation unit 221 Process designation subject information acquisition unit 222 Position information acquisition unit 223 Subject importance correction unit 224 Content acquisition unit 225 Content processing unit 230 Communication unit

Claims (6)

Operation history holding means for holding an operation history for each scene of the video;
Determining means for determining the importance of subject information appearing in each scene of the moving image using the operation history and its weight;
A designation means for designating a scene of the moving image;
A moving image scene presentation apparatus comprising: a determination unit that determines a scene to be presented based on importance of subject information appearing in the designated scene and a scene in the vicinity thereof.   The determination unit determines a scene to be presented based on the importance of subject information appearing in the designated scene and a nearby scene and the distance between the designated scene and the neighboring scene. The moving image scene presenting apparatus according to claim 1, wherein: Composition information acquisition means for acquiring composition information of the scene to be presented;
Search means for searching for related scenes using the composition information;
2. The moving image scene presentation apparatus according to claim 1, further comprising presentation means for presenting the presentation scene determined by the determination means and the related scene searched by the search means.   4. The moving image scene presentation apparatus according to claim 3, wherein the retrieval unit retrieves a related scene using the resolution information of the scene. A step of holding an operation history for each scene of the video in the holding means;
A determination step of determining the importance of subject information appearing in each scene of the moving image using the operation history and its weight;
A designation step for designating a scene of the video;
A moving image scene presentation method comprising: a determination step of determining a scene to be presented based on importance of subject information appearing in the designated scene and a scene in the vicinity thereof. Computer
Operation history holding means for holding an operation history for each scene of the video;
Determining means for determining the importance of subject information appearing in each scene of the moving image using the operation history and its weight;
A designation means for designating a scene of the moving image;
Determination means for determining a scene to be presented based on importance of subject information appearing in the designated scene and a scene in the vicinity thereof;
Program to function as.

Images