JP2014170980A - Information processing apparatus, information processing method, and information processing program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To generate an image content in which the atmosphere of a source image remains.SOLUTION: An information processing apparatus comprises: extraction means which extracts at least two partial moving images or static images from a source moving image; determination means which determines, on the basis of the feature of the source moving image, a method for presenting the at least two partial moving images or static images extracted by the extraction means; and generation means which generates an image content including the at least two partial moving images or static images, on the basis of the presentation method determined by the determination means.

Description

  The present invention relates to a technique for extracting a part of a moving image and generating new video content.

  An example of a technique for generating a new video based on an image extracted from a video is described in Patent Document 1. In Patent Document 1, key frames are selected based on the contents between frames constituting an input video (video contents such as camera movements, object movements, human faces, and acoustic contents such as audio events). Then, a new video that displays the selected key frame at an arbitrary presentation time is generated.

  An example of a method for generating a new video based on a key frame extracted from an input video is described in Patent Document 2. In Patent Document 2, a key frame selected according to face detection or user operation from an input video is displayed with a presentation time determined based on the face size, smile, age, orientation, and operation information in the key frame. Create a new video to display.

Special table 2009-537047 gazette JP 2010-213136 A

  However, according to the above prior art, the generated video is presented without considering the relevance between key frames in the source video. For this reason, in the source video, there may be cases where it is not possible to understand a change in a subject common to each other, a transition in operation, or a subject shot with a common intention by viewing a new video.

  The objective of this invention is providing the technique which solves the above-mentioned subject.

In order to achieve the above object, an apparatus according to the present invention provides:
Extraction means for extracting at least two partial videos or still images from the source video;
Determining means for determining a method of presenting the at least two partial moving images or still images extracted by the extracting means based on characteristics of the source moving images;
Generating means for generating video content including the at least two partial moving images or still images based on the presentation method determined by the determining means;
It is provided with.

In order to achieve the above object, the method according to the present invention comprises:
An extraction step of extracting at least two partial videos or still images from the source video;
A determination step of determining a method of presenting the at least two partial moving images or still images extracted in the extracting step based on characteristics of the source moving images;
Generating a video content including the at least two partial moving images or still images based on the presentation method determined in the determining step;
It is characterized by including.

In order to achieve the above object, a program according to the present invention provides:
An extraction step of extracting at least two partial videos or still images from the source video;
A determination step of determining a method of presenting the at least two partial moving images or still images extracted in the extracting step based on characteristics of the source moving images;
Generating a video content including the at least two partial moving images or still images based on the presentation method determined in the determining step;
Is executed by a computer.

  According to the present invention, it is possible to generate video content that retains the atmosphere of the source video.

It is a block diagram which shows the structure of the information processing apparatus which concerns on 1st Embodiment of this invention. It is a block diagram which shows the structure of the information processing apparatus which concerns on 2nd Embodiment of this invention. It is a figure explaining the slide show production | generation of the information processing apparatus which concerns on 2nd Embodiment of this invention. It is a flowchart which shows the flow of a process of the information processing apparatus which concerns on 2nd Embodiment of this invention. It is a figure explaining the video content production | generation of the information processing apparatus which concerns on 2nd Embodiment of this invention. It is a figure explaining the video content production | generation of the information processing apparatus which concerns on 2nd Embodiment of this invention. It is a figure explaining the video content production | generation of the information processing apparatus which concerns on 3rd Embodiment of this invention. It is a figure explaining the video content production | generation of the information processing apparatus which concerns on 3rd Embodiment of this invention. It is a figure explaining the video content production | generation of the information processing apparatus which concerns on 3rd Embodiment of this invention.

  Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the drawings. However, the components described in the following embodiments are merely examples, and are not intended to limit the technical scope of the present invention only to them.

[First Embodiment]
An information processing apparatus 100 as a first embodiment of the present invention will be described with reference to FIG. The information processing apparatus 100 is an apparatus that generates video content by editing a moving image.

  As illustrated in FIG. 1, the information processing apparatus 100 includes an extraction unit 101, a presentation method determination unit 102, and a video content generation unit 103.

  The extraction unit 101 extracts at least two partial moving images or still images from the source moving image. The presentation method determination unit 102 determines the presentation method of at least two partial moving images or still images extracted by the extraction unit 101 based on the characteristics of the source moving image. The video content generation unit 103 generates video content including at least two partial moving images or still images based on the presentation method determined by the presentation method determination unit 102. The video content includes, for example, a slide show.

  According to the present embodiment, it is possible to generate video content that leaves the atmosphere of the source video.

[Second Embodiment]
(Prerequisite technology)
Video posting sites provide a list of still images and partial videos (hereinafter referred to as key frames) extracted from the video so that viewers can quickly grasp the video content and select videos of interest efficiently. Function is used.

  In order for the viewer to properly understand the entire video, multiple key frames are usually required. However, when a large number of key frames are presented on the display at a time, the size per key frame may be reduced, and the content may not be sufficiently confirmed.

  Therefore, it is considered that a presentation method in which key frames are switched and presented in order is effective (hereinafter, videos presented in order by switching key frames are referred to as video contents). Understanding the relationship between key frames that are presented continuously in video content may be significant in understanding the content of the input video. For example, it can be understood that a continuous key frame in the video content expresses a change in a common subject or a transition of a motion, so that the flow of motion or change can be understood. In addition, it is possible to understand the relationship between the importance levels of subjects by knowing that subjects included in consecutive key frames are taken with a common intention (for example, with similar interest) in the input video. .

  In order to understand the relationship between successive key frames, the presentation method of key frame presentation time and effects inserted between key frames is important. For example, if consecutive key frames are presented in the same way, the viewer may misunderstand that there is a relationship even though there is actually no relationship between the key frames. Also, if consecutive key frames are presented in a completely different manner, the viewer may misunderstand that there is no association between the key frames. Therefore, in order to allow the viewer to correctly understand the relationship between the key frames, it can be said that it is effective to control the presentation rule according to the relationship between the contents of the key frames.

[Constitution]
The configuration of the information processing apparatus 200 according to the second embodiment of the present invention will be described with reference to FIG. FIG. 2 is a diagram for explaining a schematic configuration of the information processing apparatus 200 according to the present embodiment.

  The information processing apparatus 200 includes a key frame extraction unit 201, a presentation method determination unit 202, a video content generation unit 203, and a video input unit 204.

  The key frame extraction unit 201 extracts at least two partial moving images or still images as key frames from the source moving image 210 as input video. The presentation method determination unit 202 determines the presentation method of at least two partial moving images or still images extracted by the key frame extraction unit 201 based on the characteristics of the source moving image. The video content generation unit 203 generates video content 240 as a new video that continuously presents at least two partial moving images or still images based on the presentation method determined by the presentation method determination unit 202.

  In addition, the presentation method determination unit 202 includes an association determination unit 221 and a presentation method selection unit 222. The relevance determination unit 221 determines whether or not the objects included in at least two partial moving images or still images have commonality based on the source moving image. The relevancy determination unit 221 determines whether or not the objects included in at least two partial moving images or still images are the same based on the source moving image. When the object has commonality, the presentation method selection unit 222 selects a presentation method different from that when the object has no commonality. The presentation method determination unit 202 determines a key frame presentation time in the video content 240.

  The video input unit 204 inputs the source moving image 210 from a video camera or the like, and passes it to the key frame extraction unit 201 and the presentation method determination unit 202. The key frame extraction unit 201 sends not only the key frame extracted from the source moving image 210 but also key frame information related to the key frame to the video content generation unit 203. The key frame information is a key frame ID for identifying the key frame, a presentation order in the video content, and pixel information of the key frame.

  The video input unit 204 inputs information on the input video (video information) to the relevance determination unit 221 in response to a request from the relevance determination unit 221. The video information is a key frame ID, pixel information of a section corresponding to the key frame, and acoustic information. The section corresponding to the key frame is a unit section to which the key frame in the input video belongs, or a unit section including the same subject as the key frame. Any of the following four sections or a combination thereof may be used as the unit section. Sections separated at regular intervals. A section divided based on the control signal of the photographic equipment such as the camera turning point. A section divided based on features extracted from video such as image change points and sound change points of frames. A section that is manually delimited as a point of change in shooting content such as location, subject, or time zone.

  There may be at least one section corresponding to each key frame, and a plurality of key frames may be associated with one section. The section image information is image information of frames belonging to the section. The acoustic information of the section is sound information synchronized with the section. Further, as the section information, meta information describing the subject, the shooting location, and the shooting time shown in the section, and sensor information such as GPS may be included.

  The relevance determination unit 221 acquires video information corresponding to the key frame from the video input unit 204 based on the key frame information input from the key frame extraction unit 201, and determines the relevance between the key frames. The relevancy determination unit 221 inputs key frame relevance information to the presentation method selection unit 222. The key frame relevance information is a key frame ID and a relevance flag. In addition to the above, key frame pixel information may be input as key frame information. The relevance flag is data indicating the relevance type existing between the current key frame and the key frame to be presented after that, or any relevance type among pre-defined relevance types It is data indicating that no (no relevance). For example, as a relevance flag, flag 1 is set for all relevance types existing between a key frame and a subsequent key frame, and flag 0 is set for a non-existing relevance type. Or you may set the arbitrary numerical value which has a meaning according to a relationship classification.

  The relevancy determination unit 221 may determine whether the shooting method of the target object in the key frame is the same based on the source moving image. The relevance determination unit 221 may determine whether the objects included in the key frame have commonality based on the acoustic characteristics of the source video.

  A method for determining the relevance focusing on the identity of the subject will be described below.

(Relevance 1. Subject identity)
The relevance determination unit 221 can determine the identity of the subject between the key frames based on the continuity of shooting with the source moving image from which the key frames are extracted. The relationship thus determined is referred to as relationship 1.

  “Subjects are the same” means that a pair of consecutive key frames in the video content is a common subject. This includes the case where the subject included in the key frame pair does not change at all in appearance and the case where the subject included in the key frame pair differs from each other in the course of a series of changes and operations. For example, in a video shot of a building that changes color over time, the key frames before and after the color change appear to be different subjects as judged from the key frame alone, but refer to the source video. Thus, it can be determined that the subject is the same. Similarly, the relationship between the key frame pair extracted before and after hatching, extracted from the source video that captures the insect hatching, appears to be a different subject when judged from the key frame alone, but refers to the source video. Thus, it can be determined that the subject is the same. On the other hand, if key frames before and after the timbre change are extracted from a video of a musical instrument whose timbre changes over time, those key frames can be determined based on only the sound at the key frame extraction location. Can be determined to have taken a different subject. However, in this case as well, it can be determined that the subject is the same by referring to the entire source moving image.

  That is, by referring to the source moving image, it becomes clear that the key frame group is extracted from a moving image obtained by continuously shooting the same subject more clearly. That is, it can be estimated that the editing point of the source moving image is found and the subject is the same for the key frame group in the meantime.

  The relevance flag for relevance 1 is set to 1 when a certain key frame and the subsequent key frame are the same subject, and set to 0 when they are not the same subject. The identity determination of the subject can be realized by the following method, for example. When consecutive key frame pairs in the video content are associated with the same section and the subject area (image area of the subject) detected from the key frame is the same, it is determined that they are the same subject. May be. Alternatively, the subject area is detected from each pair of consecutive key frames in the video content, and the subject area is detected and tracked from the section corresponding to each key frame. When the subject area detected from one section and the subject area of the track process are compared with the subject area detected from the other section and the subject area of the track process, and the image feature values such as color and shape are similar May determine that the subject is the same. Alternatively, the image area of the subject and the acoustic information emitted from the subject are extracted from the section corresponding to each key frame for a pair of key frames that are continuous in the video content.

  When the subject area detected from one section and the subject area detected from the other section have similar image characteristics and acoustic information, the subject included in the key frame pair is the same You may judge. The method of detecting the subject area from the key frame is divided into a case where a specific object registered in advance is detected and a case where a general object which is not registered is detected. When detecting a specific target, the image data of each registered target is used as a template, a key frame belonging to a section is scanned with a template converted into various resolutions, and an area with a small difference in pixel values at the same position as the template is detected. It may be detected as a corresponding subject area.

  Alternatively, an image feature amount expressing color, texture, and shape is extracted from each partial region of the key frame, and a partial region having an image feature amount similar to the registered image feature amount of each target may be used as the corresponding subject region. Good. In addition, when the specific target is a person, there is a method of using information obtained from the entire face. For example, there is a method of storing an image showing various faces as a template and determining that the face is present in the input image when the difference between the key frame and the template is equal to or smaller than a threshold value. Alternatively, a model combining color information such as skin color, edge direction and density may be stored in advance, and an area similar to the model may be detected as a subject area. Furthermore, any of the following methods or a combination thereof may be used. A method of detecting the outline of the face (head) using an ellipse and a template created using the shape of an elongated eye and mouth. A technique that uses the characteristics of the luminance distribution that the cheeks and forehead are bright and the eyes and mouth are low. A technique that uses the symmetry and skin tone area and position of the face.

  A method of statistically learning feature quantity distributions obtained from a large number of human face and non-face learning samples and determining whether the feature quantity obtained from the input image belongs to a face or non-face distribution ( Neural network, support vector machine, AdaBoost method) or the like may be used. When detecting a general target, for example, Normalized Cut, Saliency Map, or Depth of Field (DoF) may be used. Normalized Cut is a technique for dividing an image into a plurality of regions. Jianbo Shi and Jitendra Malik, “Normalized Cuts and Image Segmentation”, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.22, No.8, August2000 has a detailed explanation of Normalized Cut. Of the areas divided by the Normalized Cut, an area located at the center of the key frame may be detected as a subject area. Alternatively, an area having a high importance calculated by the Saliency Map may be detected as a subject area. Saliency Map is a method of calculating an object region in an image from visual attention. For Saliency Map, see L. Itti, C. Koch and E. Niebur, “A Model of Saliency-based Visual Attention for Rapid Scene Analysis,” IEEE Trans. Pattern Analysis and Machine Intelligence, Vol. 20, No. 11, pp. .1254-1259, 1998 has a detailed description.

  Dof is a method based on the characteristic that the target edge existing within the depth of field is not blurred and the edge outside the depth of field is blurred. Details are disclosed in 3Du-Ming Tsai, Hu-Jong Wang, “Segmenting focused objects in complex visual images”, Pattern Recognition Letters, Vol. 19, pp. 929 940, 1998. The blur amount may be calculated based on the thickness of the edge, the edges with less blur may be combined, and the focused area may be detected as the target area.

  Shows the quality of the image based on the position in the still image or high visibility (lighting conditions, orientation, angle, position on the screen, hiding by other objects, blur, facial expression (in the case of a person), etc.) The target area may be detected from the key frame based on the evaluation value) or the appearance frequency in a plurality of images. A plurality of detected subject areas may be combined into one. To detect the subject area from the section corresponding to the key frame, for example, the image information of the subject area detected from the key frame is used as a template, and the subject area is detected from any frame belonging to the section corresponding to the key frame. May be.

  Tracking of the subject area detected in the section can be realized by the following method. A frame in which the subject area is detected is set as a start frame, and subject area detection processing is performed on frames adjacent in the time direction. The template used for the detection of the subject area uses the image feature quantity of the target area that has already been detected, and scans the template in the area of the specified range centering on the detection position of the target area that has already been detected. The similarity between the subject areas may be calculated based on a scale that extracts an image feature amount from each subject region and calculates a higher value as the difference in the image feature amount is smaller. The image feature amount can be calculated based on image information such as a color, an edge, and a texture detected from the subject area. Alternatively, local feature points such as SIFT are detected from the image area of each subject, the feature points are associated between the image areas, and the number of associated feature points is large, or the positional relationship of the associated feature points A scale may be used that calculates a higher value as the images are more similar between images. The similarity of the acoustic information emitted by the subject is, for example, a scale that extracts acoustic energy in a plurality of frequency bands as acoustic information from the section in which the subject region is detected, and calculates a higher value as the difference in acoustic energy is smaller. And may be calculated as As described above, by using the information of the section corresponding to the key frame, it is more robust against changes in the appearance of the subject and changes in the background that occur between the key frames than when only the key frame information is used. The identity of the subject can be determined.

  The presentation method selection unit 222 determines an effect or jingle when switching key frames. The presentation method selection unit 222 determines an effect or jingle different from the case where there is no relationship when consecutive key frames are related to each other. The presentation method selection unit 222 determines the background music of the key frame in the video content.

  Specifically, the presentation method selection unit 222 determines a key frame presentation method based on key frame relevance information input from the relevance determination unit 221 and a pre-registered presentation rule. The presentation method selection unit 222 inputs information (presentation method information) indicating a key frame presentation method to the video content generation unit 203. The presentation method information is data indicating a presentation method of each key frame, and includes a key frame ID and a presentation time. In addition to the above, effects, BGM, audio jingles, and video jingles may be held as presentation method information.

  The presentation rule is a rule that defines a method for presenting a key frame in accordance with the relevance type. As a presentation rule, a parameter defining each presentation time of consecutive key frame pairs is held. In addition to the presentation time, control parameters related to effects, BGM, and jingles (short video, music, and sound effects) inserted between key frames may be held. Moreover, you may prescribe | regulate the presentation method in case no relevance type exists in a continuous key frame pair. Examples of the presentation rules include the following.

(1) Rule regarding presentation time When at least two partial moving images or still images are related to each other, the presentation method determination unit 202 determines the other presentation time based on one presentation time. Specifically, the presentation method determination unit 202 inserts after the presentation time of the partial moving image or still image inserted before when the objects included in at least two partial moving images or still images are the same. To shorten the presentation time of the partial video or still image. On the other hand, the presentation method determination unit 202 determines the presentation time independently when at least two partial moving images or still images are not related to each other.

  In other words, the presentation time of the key frame pair is determined based on the identity of the subject included in the successive key frame pairs or the identity of the photographer's interest in the subject. For example, when the subjects included in consecutive key frame pairs are the same or the photographer's interest in the subject is the same, the presentation time of the key frame presented first is set as the initial value Ts, and the subsequent key is based on Ts. The frame presentation time may be determined. Further, among key frame groups including the same subject or subjects photographed with the same interest, the presentation time of the key frame with high visibility is set as Tp, and the presentation time of the subsequent key frame is determined based on Tp. Also good. In addition, among key frame groups including the same subject or subjects photographed with the same interest, the presentation time of the key frame next to the key frame whose presentation time is equal to or less than Tq is set as an initial value Ts, and Ts is used as a reference. The presentation time of subsequent key frames may be determined. In addition, among key frame groups including the same subject or a subject photographed with the same interest, the presentation time of the key frame presented last may be set as the initial value Ts. The values of Ts and Tp may be calculated according to the number of key frames to be presented by setting the time of the entire video content in advance. If the subjects included in the consecutive key frame pairs are not the same or are not taken with the same interest, the subsequent presentation time is determined independently of the presentation time of the previous key frame. For example, the initial value Ts may be set, or a random value within a specified range may be set.

さらに、正面を向いたキーフレーム３０３で視認性の評価値が閾値以上の場合、キーフレーム３０３の提示時間はＴｐ、後続のキーフレーム３０４、３０５の提示時間Ｔｊは以下の式（１）で表わされる。

パラメータａを０から１の間に設定すると、人物Ａを含むキーフレームのうち初めに提示されたキーフレーム３０１と人物Ａの映りがよいキーフレーム３０３は長く提示され、キーフレーム３０１、３０３から遠ざかるに従って徐々に短く提示される。 As an example, a case will be described in which five key frames 301 to 305 are extracted from a moving image shot around the person A with reference to FIG. Here, since the subjects included in the key frames 301 to 305 are the same, as described above, the presentation time of the subsequent key frame is obtained by multiplying the presentation time Ts of the first key frame by the parameter a. calculate. At this time, when the presentation time of the first key frame 301 is an initial value Ts, the presentation time Ti of the subsequent key frame 302 is expressed by the following equation (1).

Further, when the visibility evaluation value of the key frame 303 facing the front is equal to or greater than the threshold value, the presentation time of the key frame 303 is represented by Tp, and the presentation times Tj of the subsequent key frames 304 and 305 are represented by the following formula (1). It is.

When the parameter a is set between 0 and 1, among the key frames including the person A, the key frame 301 presented first and the key frame 303 with good reflection of the person A are presented long and move away from the key frames 301 and 303. Will be presented gradually and according to.

  As a result, the user can understand the moment when the object first appears and the contents of the keyframes with good reflection, and the other images can understand that the contents are almost the same as the understood contents. is there. In addition, even if the images include the same target, it is possible to generate a video in which the presentation time of successive images changes, so that it is possible to generate video content with a sense of tempo that does not bore viewers.

(2) Rules for Effects, BGM, and Jingles The presentation method selection unit 222 determines whether the key frame pair is based on the identity of the subject included in the continuous key frame pair or the identity of the photographer's interest in the subject. Determine the effect, BGM, and jingle to be inserted. For example, if the subject included in a pair of consecutive key frames is the same or the photographer's interest in the subject is the same, special effects (dissolve and fade) registered in advance as effects with little visual change when switching key frames. Etc.). If they are not the same, a special effect (DVE such as page turning or wipe) registered in advance as an effect having a large visual change at the time of switching key frames is inserted. Further, for example, when the subjects included in the continuous key frame pairs are subjects photographed with the same or the same interest, the same BGM is played during the presentation of the key frame pairs, and when they are not the same, the key frames are switched. Sometimes stop BGM or switch to a different BGM. Also, a video or audio jingle may be inserted between key frames that do not include subjects that are not the same or are photographed with the same interest. As a result, the key frame group including the same subject or the subject photographed with the same interest is smoothly connected without any image or acoustic change.

  Therefore, the viewer includes subjects with the same key frame or subjects with the same importance, and images that are in the middle of a series of changes or operations, or include subjects that are taken with the same intention. It can be easily understood. In addition, when the subject in the key frame is not the same or does not include the subject photographed with the same intention, the image and the acoustic change greatly change. Therefore, the viewer can notice that the contents of the key frame have changed greatly and can concentrate on understanding the new video.

  The video content generation unit 203 generates and outputs a new video content based on the presentation method information selected by the presentation method selection unit 222 and the key frame information input from the key frame extraction unit 201.

(Operation)
Next, the operation of the present embodiment will be described in detail with reference to the flowchart of FIG. Here, as an example, consider the case where key frames 501 to 513 shown in FIG. 5 are extracted to generate video content. This video content conveys the event of shooting a flower and a person in the greenhouse in the building. The target area detected from each key frame by the relevance determination unit 221 is indicated by a rectangle in FIG.

  Also, as a presentation rule, a key frame pair with the same target area uses a rule based on the magnitude relationship or partial relation, and a key frame pair with the same target area uses a rule based on homogeneity to control the presentation method. To do. The rules based on the magnitude relationship, the partial relationship, and the homogeneity will be described in detail in the third and subsequent embodiments.

  First, in step S401, the video input unit 204 inputs a moving image serving as a source. In step S402, the input source moving image is transferred to the key frame extraction unit 201, where key frames are extracted.

  In step S 403, the key frame information is passed from the key frame extraction unit 201 to the presentation method determination unit 202, and the video information is also passed from the video input unit 204 to the presentation method determination unit 202. The relevancy determination unit 221 determines the identity of the subject and the commonality of the shooting method with reference to the source moving image from which the key frames 501 to 513 are extracted.

  Furthermore, in step S403, the relevance determination unit 221 detects the target area from the key frames 501 to 513. Assume that in the relevance determination unit 221, buildings, flowers, and people are registered in advance, and the respective models are learned. And the location enclosed with the solid-line rectangle as a target area | region of a building is detected from the key frames 501-513, respectively.

  In step S405, image feature amounts are extracted from the pixel information of the target area 0 and the target area 1, and the identity, magnitude relationship, partial relationship, and homogeneity are determined based on the similarity between the areas. Since the target areas 0 and 1 are detected as the types of buildings, they have the same type. Also, a broken-line rectangular area on the key frame 501 is detected as a common area of the target area 1 and the target area 0, and it can be seen that the target areas 1 and 0 are in a size relationship. Further, since there is no area other than the common area on the target area 0, it is determined that there is no partial relationship. Therefore, the relationship flag between the key frame 501 and the key frame 502 is 1, −1, 0, 1 in the order of identity, magnitude relationship, partial relationship, and homogeneity.

  The presentation method selection unit 222 selects a presentation method based on the image ID and the relevance flag as the image relevance information. For example, since the target areas of the key frame 501 and the key frame 502 are the same, a rule based on a magnitude relationship or a partial relationship is applied. Since the presentation time of the key frame 501 that is the start image is the initial value Ts and the magnitude relationship between the key frames 501 and 502 is small / large, the presentation time of the key frame 502 is a * Ts. Further, since there is a magnitude relationship between the key frames 501 and 502, a dissolve with little visual change is inserted as an effect of switching between the key frames 501 and 502 (step S407).

  The video content generation unit 203 generates video content using the key frames 501 and 502 with the determined presentation time / effect (step S409).

  The target area type 601 detected from the key frame, the relevance flag 602 for each relevance type, the presentation time length 603 determined by the presentation method determining unit 202, and the effect 604 are shown in FIG.

  According to the present embodiment, it is possible to generate a new video that can easily understand the semantic relevance of the subject in the key frame in the input video by using the key frame extracted from the input video.

[Third Embodiment]
Instead of or in addition to the relevance 1 disclosed in the second embodiment, the presentation method may be changed according to any one of the following relevance changes.

(Relevance 2. Movie shooting method)
When a group of key frames relevant to the photographing method in which the photographer photographed the subject is extracted from the source video, the key frames are presented by a presentation method according to the relevance of the photographing method. For example, if multiple consecutive keyframes are extracted from the video part that was shot by follow shooting (a technique that moves the camera to follow the movement of the subject), these keyframes are related. Judge that there is sex.

  Similarly, when a plurality of consecutive key frames are all extracted from a moving image portion shot by zoom shooting, or all of them are extracted from a moving image portion shot still for a certain period of time or more, those keys are used. It is determined that the frame group has some kind of relationship and commonality. As the relevance flag for relevance 2, 1 is set when there is a relevance to the imaging method, and 0 is set when there is no relevance.

  A subject area is detected from each pair of consecutive key frames, and each subject area is detected from a section corresponding to the key frame based on information on the subject area. The motion vectors of the subject area and the background area are analyzed, and it is determined that the subject is photographed by an intentional method such as follow, zoom, or still.

  The method for photographing the subject area in the section can be determined by the following method, for example. It can be determined by the follow target determination method disclosed in Japanese Patent No. 4593314 that the subject has been photographed following. Whether the subject is zoomed in or taken still can be determined by the camera motion determination method disclosed in Japanese Patent Application Laid-Open No. 2007-19814.

[Rules based on commonality of video shooting methods]
(2-1) Rules for Presentation Time The presentation time for key frame pairs is determined based on the identity of the source video capturing method for successive key frame pairs. For example, when key frames shot with the same shooting method are consecutive, the presentation time is gradually shortened. That is, among the key frame groups photographed by the same method, the presentation time of the first key frame presented is set as the initial value Ts, and the presentation time of the subsequent key frame is determined based on Ts. Moreover, the presentation time of the key frame with high visibility among the key frame groups photographed by the same method may be Tp, and the presentation time of the subsequent key frame may be determined based on Tp. In addition, among the key frame groups photographed by the same method, the presentation time of the next key frame in which the presentation time of the key frame is equal to or less than Tq is set as the initial value Ts, and the presentation time of the subsequent key frame based on Ts. May be determined. Also, the presentation time of the image presented last among the key frame groups photographed by the same method may be set to Ts. The values of Ts and Tp may be calculated according to the number of images to be presented by setting the presentation time of the entire video content in advance. When successive key frame pairs are photographed in different ways, the subsequent presentation time is determined independently of the previous key frame presentation time. For example, the initial value Ts may be set, or a random value within a specified range may be set.

(2-2) Rules Regarding Effects, BGM, and Jingles Based on the identity of consecutive key frame pairs, the effects, BGM, and jingles to be inserted between key frame pairs are determined. For example, when consecutive key frame pairs are photographed by the same method, special effects (such as dissolves and fades) registered in advance as effects with little visual change are inserted when switching key frames. When the image is shot by a different method, a special effect (DVE such as page turning or wipe) registered in advance as an effect having a large visual change at the time of switching key frames is inserted. Also, for example, when consecutive key frame pairs are photographed in the same method, the same BGM is played during presentation of the key frame pair, and when photographed in a different method, the BGM is stopped or different when the key frame is switched. Switch to BGM. A jingle may be inserted between key frames photographed by different methods. As a result, when consecutive key frame pairs are photographed in the same manner, they are smoothly connected without any image or acoustic change. Therefore, the viewer can easily understand that the key frames have almost the same contents without any change. In addition, when the image is taken by a different method, the image and the acoustic change greatly change, so that the viewer can notice that the content is changed and can concentrate on understanding the content of the video content.

(Relevance 3. Target size relationship)
The relevancy determination unit 221 may determine the magnitude relationship between the key frames based on the presence / absence of zoom shooting in the source moving image from which the key frames are extracted and the area of the target region. The relationship thus determined is referred to as relationship 3.

  The “target size relationship” means that the targets included in consecutive key frame pairs in the video content are the same, and the area of the target region has a difference greater than a specified value. For example, there is a case where a target is introduced by generating a video content by combining an image including the periphery of the target and an image obtained by photographing only the target.

  The magnitude relationship between objects can be determined based on the area of partial areas common to the target areas determined to be the same or the distance between feature points included in the common partial areas. For example, it can be determined that the larger the distance between feature points, the larger the object is photographed. The determination may be made between target areas determined to be identical between consecutive key frame pairs in the video content. In this case, the relevance flag for relevance 3 is set to 1 when the area of the target area in the next key frame is larger than the area of the target area in a certain key frame, and to 1 when the area is smaller. -1 is set to 0 when there is no magnitude relationship. Or you may determine by comparing the area of the partial area | region common to the object area | region determined to be the same among the object area | regions detected from all the key frames contained in video content, or the distance between feature points. For example, based on the maximum area Smax and the minimum area Smin of the partial areas common to the target areas determined to be the same, the same target area smaller than (Smax + 2Smin) / 3 is smaller and larger than (Smax + 2Smin) / 3 The same target area smaller than (2Smax + Smin) / 3 is set as medium, and the same target area larger than (2Smax + Smin) / 3 is set as large. The relevance flag includes 1 if the target area in consecutive keyframes has a small-medium or medium-large relationship, 2 if a small-to-large relationship, or large-medium or medium-small If the relationship is -1, -1 is set. If the relationship is large and small, -2 is set. If there is no size relationship, 0 is set.

[Rules according to the target size]
(3-1) Rules for Presentation Time The presentation time of a key frame pair is determined based on the size relationship of objects included in successive key frame pairs. For example, the presentation time of the next key frame is determined using Ts as the initial value Ts and the presentation time of the key frame presented first among the key frame groups in the target size relationship. Moreover, the presentation time of the key frame with high visibility among the key frame groups in the size relationship of the target may be Tp, and the presentation time of the subsequent key frame may be determined based on Tp. In addition, among the key frame groups having a magnitude relationship, the presentation time of the next key frame whose presentation time of the key frame is equal to or less than Tq is set as the initial value Ts, and the presentation time of the subsequent key frame is determined based on Ts. May be. Further, the presentation time of the key frame presented last among the key frame groups having a magnitude relationship may be set to Ts. The values of Ts and Tp may be calculated according to the number of images to be presented by setting the presentation time of the entire video content in advance. Further, when there is no magnitude relationship between the objects included in the consecutive key frame pairs, the subsequent presentation time is determined independently of the presentation time of the previous key frame. For example, the initial value Ts may be set, or a random value within a specified range may be set.

  A case where key frames obtained by shooting the target B of various sizes are reproduced will be described with reference to FIG. It is assumed that the magnitude relationship between successive key frames is determined by comparing the areas between target regions determined to be the same among target regions detected from all key frames included in the video content. Also, it is assumed that the presentation time of the next key frame is calculated by multiplying the presentation time of a key frame by the relevance flag parameter a. At this time, the presentation time of the first key frame 701 is the initial value Ts, the key frames 701 and 702 are small and medium, the key frames 702 and 703 are medium and large, and the key frames 703 and 704 are large and small. It is related. At this time, since the relevance flag of the key frames 701 and 702 is 1, the presentation time of the key frame 702 is aTs (multiplication of a). Further, since the relevance flag of the key frame 703 is 1, the presentation time of the key frame 703 is a × a × Ts (multiplication of a). Since the relevance flag of the key frames 703 and 704 is −2, the presentation time of 704 is Ts (a × a division). When the parameter a is set between 0 and 1, a key frame (long shot) in which the subject B is photographed small is long and a key frame (middle shot and tight shot) in which the subject B is photographed larger is presented short. .

  As a result, the user can understand the contents of a key frame with a large amount of information in which a scene other than the target B is reflected, and can intuitively understand that the subsequent contents are a part of the previous key frame. . In addition, even if the images include the same target, it is possible to generate a video in which the presentation time of successive images changes, so that it is possible to generate video content with a sense of tempo that does not bore the viewer.

(3-2) Rules for Effect / BGM / Jingle The effect / BGM / jingle to be inserted between the key frame pairs is determined based on the size relationship of the objects included in the consecutive key frame pairs. For example, when the objects included in the consecutive key frame pairs are in a magnitude relationship, special effects (such as dissolves and fades) registered in advance as effects with little visual change when key frames are switched are inserted. If there is no size relationship, a special effect (DVE such as page turning or wipe) registered in advance as an effect having a large visual change at the time of switching the key frame is inserted. Also, for example, if the objects included in the consecutive key frame pairs are in a size relationship, the same BGM is played during the presentation of the key frame pairs, and if they are not the same, the BGM is stopped at the time of switching the key frame or is changed to a different BGM. Switch.

  Also, jingles may be inserted between images that do not have a magnitude relationship. Thereby, the key frame group which image | photographed the object of magnitude relationship is connected smoothly, without an image and an acoustic change. Therefore, the viewer can easily understand that the key frames have almost the same contents without any change. In addition, when there is no size relationship, image and acoustic changes greatly change, so that the viewer can notice that there is a change in the content and can concentrate on understanding the content of the video content.

(Relevance 4. Target partial relationship)
The relevancy determination unit 221 may determine the relevance based on the partial relationship of the target represented in the two key frames. That is, the determination may be made depending on whether the object represented in the two key frames included in the key frame pair is in a relationship between the whole and the part. The relationship thus determined is referred to as relationship 4.

  “In a partial relationship of objects” indicates a relationship in which the objects shown in consecutive key frame pairs in the target video content are the same, and are images obtained by capturing different target parts. For example, when a wide landscape, a large object, or a long object is to be photographed, this is a case in which the whole is expressed by reproducing video content by combining key frames in which a part of the object is photographed.

  The relevance flag for relevance 4 is 1 if the target area in a key frame and the target area in the next key frame are in a target partial relationship, and 1 if not in the target partial relationship. Set to 0. The target partial relationship can be determined based on a partial area (common area) common to the target areas determined to be the same in consecutive key frames in the video content. For example, one of the target areas is used as a template, and the other target area is scanned to detect a position with a small difference, and an overlapping area is set as a common area. When both regions other than the common region of each target region are larger than the specified area, it is determined that the target partial relationship. Alternatively, the determination may be made based on the relative position of the target area determined to be the same from all the key frames included in the video content.

  When the change of the object from the whole to the part continues, it is assumed that there is no change in relevance, and the same presentation method is changed.

[Rules according to the target partial relationship]
(4-1) Rules for Presentation Time Based on the partial relationship of objects included in consecutive key frame pairs, the presentation time of key frame pairs is determined. For example, in the group of key frames in the target partial relationship, the presentation time of the first key frame presented is set as the initial value Ts, and the presentation time of the subsequent key frame is determined based on Ts. Moreover, the presentation time of the key frame with high visibility among the key frame groups in the target partial relationship may be Tp, and the presentation time of the subsequent key frame may be determined based on Tp. In addition, the presentation time of the next key frame in which the presentation time of the key frame is equal to or less than Tq is set as the initial value Ts, and the presentation time of the subsequent key frame is determined based on Ts. May be. In addition, the presentation time of the image presented last among the key frame groups having a partial relationship may be set to Ts. The values of Ts and Tp may be calculated according to the number of images to be presented by setting the presentation time of the entire video content in advance. When there is no partial relationship among the objects included in the consecutive key frame pairs, the subsequent presentation time is determined independently of the presentation time of the previous key frame. For example, the initial value Ts may be set, or a random value within a specified range may be set.

  A case of reproducing a key frame obtained by reproducing a key frame obtained by photographing a landscape will be described with reference to FIG. The partial relationship between consecutive key frames has a positional relationship between a target region and a partial region that is common among target regions determined to be the same among target regions detected from all key frames included in the video content. It is assumed that Further, it is assumed that the presentation time of the next key frame is calculated by multiplying the presentation time of a certain key frame by a specified parameter.

The presentation time of the first key frame 801 is set as an initial value Ts. Key frames 801 and 802 and 802 and 803 have a partial relationship, and key frames 803 and 804 have no partial relationship. At this time, if the presentation time of the first key frame 801 is the initial value Ts, the relevance flag of the key frames 801 and 802 is 1, so the presentation time of the key frame 802 is a × Ts. Further, since the relevance flag of the key frames 802 and 803 is again 1, the presentation time of the key frame 803 is a ² Ts. Since the relevance flag of the key frames 803 and 804 is 0, the presentation time of the key frame 804 is returned to the initial value to be Ts.

  The parameter a is set to a smaller value between 0 and 1 and as the area of the partial region that matches between the key frames is larger. Then, the key frame 801 presented for the first time with respect to the landscape is presented for a long time, and the other part is presented with a presentation time corresponding to the amount of information overlapping with the previously presented image. As a result, the user can understand the contents of the key frame presented first for the landscape, and can understand that the subsequent contents are substantially equivalent to the first key frame. In addition, even if the images include the same target, it is possible to generate a video in which the presentation time of successive images changes, so that it is possible to generate video content with a sense of tempo that does not bore the viewer.

(4-2) Rules for Effect, BGM, and Jingle The effect, BGM, and jingle to be inserted between the key frame pairs are determined based on the target partial relationship included in the continuous key frame pairs. For example, if the objects included in successive key frame pairs are in a partial relationship, a special effect (such as dissolve or fade) registered in advance as an effect with little visual change when key frames are switched is inserted. When there is no partial relationship, a special effect (DVE such as page turning or wipe) registered in advance as an effect having a large visual change at the time of switching key frames is inserted. Also, for example, when consecutive key frame pairs are in the target partial relationship, the same BGM is played during presentation of the key frame pairs, and when they are not identical, the BGM is stopped or switched to a different BGM when the key frames are switched. Also, jingles may be inserted between images that do not have a magnitude relationship. Thereby, when a continuous key frame pair has a target partial relationship, there is no image or acoustic change, and the connection is smooth. Therefore, the viewer can easily understand that the key frames have almost the same contents without any change. In addition, when there is no partial relationship, image and acoustic changes greatly change, so that the viewer can notice that there is a change in the content and can concentrate on understanding the content of the video content.

(Relevance 5. Target homogeneity)
The relevance determination unit 221 may determine relevance depending on whether or not the objects represented in the two key frames are of the same type. The relationship thus determined is referred to as relationship 5.

  “The objects are of the same type” means that main objects appearing in consecutive key frame pairs in the video content are objects of the same type. The relevance flag for relevance 5 is set to 1 when the target area in a key frame and the target area in the next key frame have the same kind of relationship, and set to 0 in the case of different kinds. . The homogeneity of an object can be realized by a method based on machine learning based on image data (registered data) of an object belonging to various types for which homogeneity is to be determined. First, target image feature quantities belonging to various types are extracted from the registered data. As the image feature amount, a global feature such as a color histogram or an edge histogram may be used, or a local feature amount such as HoG or SIFT may be used. Learning may be performed using SVM, neural network, GMM, or the like using global features, or learning may be performed after converting a feature amount space such as BoW (Bag of Words) from a local feature amount. When determining the homogeneity for the target area in each key frame included in the video content, obtain the similarity between the image feature quantity of each target area and various models obtained as a result of learning, The target area is determined to be the closest model type that has obtained a similarity equal to or greater than a specified value. Target areas determined to be the same type are determined to be the same type. The homogeneity may be determined by a method other than the above.

  When three images including the same type of target are consecutive, it is assumed that there is no change in relevance, and the same presentation method is changed.

[Rules according to target homogeneity]
(5-1) Rules for Presentation Time The presentation time of a key frame pair is determined based on the homogeneity of objects included in consecutive key frame pairs. For example, when key frames that photograph the same type of object are consecutive, the presentation time is gradually shortened. That is, among the key frame groups including the same type of target, the presentation time of the key frame presented first is set as the initial value Ts, and the presentation time of the subsequent key frame is determined based on Ts. In addition, among key frame groups including the same type of target, the presentation time of a key frame with high visibility may be Tp, and the presentation time of subsequent key frames may be determined based on Tp. Also, among key frame groups including the same type of target, the presentation time of the next key frame whose key frame presentation time is Tq or less is set as the initial value Ts, and the presentation time of the subsequent key frame is determined based on Ts. May be. Moreover, you may set the presentation time of the image shown last among the key frame groups containing the same kind of object to Ts. The values of Ts and Tp may be calculated according to the number of images to be presented by setting the presentation time of the entire video content in advance. If the objects included in successive key frame pairs are not the same type, the subsequent presentation time is determined independently of the presentation time of the previous key frame. For example, the initial value Ts may be set, or a random value within a specified range may be set.

The case where the key frame which reproduced the key frame which image | photographed the flower is reproduced is demonstrated using FIG. Assume that the homogeneity between consecutive key frames is determined by a method based on machine learning. Also, the presentation time of the next key frame is calculated by multiplying the presentation time of a certain key frame by the parameter for the relevance flag. The presentation time of the first key frame 901 is an initial value Ts. The key frames 901 and 902 and the key frames 902 and 903 have the same type, and the key frames 903 and 904 have a different type. At this time, since the relevance flag of the first key frames 901 and 902 is 1, the presentation time of the key frame 902 is a × Ts. Further, since the relevance flag of the key frames 902 and 903 is 1, the presentation time of the key frame 903 is a ² Ts. Since the relevance flag of the key frames 903 and 904 is 0, the presentation time of the key frame 904 is returned to the initial value to be Ts. When the parameter a is set between 0 and 1, the key frame 901 presented for the first time among the key frames including the plant is presented for a long time, and the subsequent key frames are presented with a short presentation time as the distance from the 901 is increased. Thereby, the user can understand that the image content is a plant from the key frame presented first, and can understand that the content of the subsequent key frame is almost the same.

  In addition, even if the images include the same target, it is possible to generate a video in which the presentation time of successive images changes, so that it is possible to generate video content with a sense of tempo that does not bore the viewer. By reproducing images of multiple flowers taken in the flower field in order of the same type of subject, it can be expressed that there are many such types of subjects.

(5-2) Effect / BGM / Jingle Rules The effect / BGM / jingle to be inserted between key frame pairs is determined based on the homogeneity of objects included in consecutive key frame pairs. For example, when the targets included in successive key frame pairs are of the same type, a special effect (such as dissolve or fade) registered in advance as an effect with little visual change when key frames are switched is inserted. In the case of different types, a special effect (DVE such as page turning or wipe) registered in advance as an effect having a large visual change at the time of switching key frames is inserted. Further, for example, when consecutive key frame pairs are of the same type, the same BGM is played during presentation of the key frame pair, and when different, the BGM is stopped or switched to a different BGM when the key frame is switched. A jingle may be inserted between different types of key frames. Thereby, when the object contained in the continuous key frame pair is of the same type, there is no image or acoustic change, and the connection is smooth. Therefore, the viewer can easily understand that the key frames have almost the same contents without any change. In the case of different types, since the image and acoustic change greatly change, the viewer can notice that the content has changed and can concentrate on understanding the content of the video content.

(Relevance 6. Identity of shooting location)
The relevance determination unit 221 may determine the relevance based on the commonality of the shooting locations of the two key frames. The relationship thus determined is referred to as relationship 6.

  “The shooting location is the same” means that the location where the consecutive key frame pairs are shot in the video content is the same. The relevance flag for relevance 6 is set to 1 when a key frame and the next key frame are at the same shooting location, and set to 0 when they are at different shooting locations. The identity of the shooting location can be determined based on the similarity of an area (background area) other than the target area in the key frame. For example, when the target region and the background region are separated from the key frame and the image feature values extracted from the background region are similar, it may be determined that they are the same shooting location. The identity of the shooting location may be determined by a method other than the above. The identity of the shooting location may determine the similarity of the background between consecutive key frames in the video content. Alternatively, the determination may be made based on the identity of the background area in all key frames included in the video content. In addition to the image information, determination may be made by combining shooting location as meta information and GPS as sensor information.

  When three images taken at the same shooting location are consecutive, it is assumed that there is no change in relevance, and the same presentation method is changed.

[Rules according to the identity of the shooting location]
(6-1) Rules for Presentation Time Based on the identity of the shooting locations of consecutive key frame pairs, the presentation time of key frame pairs is determined. For example, when key frames shot at the same shooting location are consecutive, the presentation time is gradually shortened. For example, in the group of key frames shot at the same place, the presentation time of the first key frame presented is set as the initial value Ts, and the presentation time of the subsequent key frame is determined based on Ts. In addition, among the key frame groups photographed at the same place, the presentation time of a highly visible key frame may be Tp, and the presentation time of the subsequent key frame may be determined based on Tp. In addition, among the key frame groups photographed at the same place, the presentation time of the next key frame whose key frame presentation time is equal to or less than Tq is set as the initial value Ts, and the presentation time of subsequent key frames based on Ts. May be determined. In addition, the presentation time of the image presented last among the key frame groups photographed at the same place may be set to Ts. The values of Ts and Tp may be calculated according to the number of images to be presented by setting the presentation time of the entire video content in advance. When successive key frame pairs are photographed at different locations, the subsequent presentation time is determined independently of the previous key frame presentation time. For example, the initial value Ts may be set, or a random value within a specified range may be set.

(6-2) Effects, BGM, and Jingle Rules Based on the identity of the shooting locations of consecutive key frame pairs, the effect, BGM, and jingle inserted between the key frame pairs are determined. For example, when consecutive key frame pairs are photographed at the same place, special effects (such as dissolves and fades) registered in advance are inserted as effects with little visual change when switching key frames. When the image is taken at a different place, a special effect (DVE such as page turning or wipe) registered in advance is inserted as an effect having a large visual change when the key frame is switched. Also, for example, when consecutive key frame pairs are photographed at the same location, the same BGM is played during presentation of the key frame pairs, and when photographed at different locations, the BGM is stopped or different when the key frames are switched. Switch to BGM. A jingle may be inserted between key frames taken at different locations. As a result, when consecutive key frame pairs are photographed at the same place, they are smoothly connected without any image or acoustic change. Therefore, the viewer can easily understand that the key frames have almost the same contents without any change. In addition, when images are taken at different places, image and acoustic changes greatly change, so that the viewer can notice that there is a change in the content and can concentrate on understanding the content of the video content.

(Relevance 7. Same time zone)
The relevancy determination unit 221 may determine the relevance based on the commonality of the shooting time zones of two key frames included in the key frame pair. The relationship determined in this way is referred to as relationship 7.

  “The shooting time zones are the same” means that the time zones in which consecutive key frame pairs are shot in the video content are the same. The relevance flag for relevance 7 is set to 1 when a certain key frame and the next key frame are in the same shooting time zone, and set to 0 when they are in different shooting locations. The identity of the shooting time zone can be determined based on the color information of the background area in the key frame. For example, a day is divided into a plurality of time zones, the statistics of the color histogram of sunlight in each time zone are held, and the partial area close to the statistics of any time zone is included in the background area of the key frame The key frame is taken during that time period. The shooting time zone of each key frame is estimated, and when the estimated time is the same, it is determined that the shooting time zones are the same. The identity of the shooting time period may be determined by a method other than the above. The identity of the shooting time zone may determine the similarity of the shooting time zone between consecutive key frames in the video content. Alternatively, the determination may be made based on the identity of the shooting time zones in all the key frames included in the video content. In addition to the image information, the determination may be made in combination with the shooting time that is meta information.

  When three images taken in the same shooting time period are consecutive, it is assumed that there is no change in relevance, and the same presentation method is changed. For example, the presentation time is gradually shortened at the same time interval.

[Rules according to the identity of the shooting period]
(7-1) Rules Regarding Presentation Time Based on the identity of the shooting time zones of consecutive key frame pairs, the presentation time of key frame pairs is determined. For example, in the case where key frames photographed within a certain range of photographing time are consecutive, the presentation time is gradually shortened. That is, among the key frame groups photographed in the same time zone, the presentation time of the key frame presented first is set as the initial value Ts, and the presentation time of the subsequent key frame is determined based on Ts. Moreover, the presentation time of the key frame with high visibility among the key frame groups photographed in the same time zone may be Tp, and the presentation time of the subsequent key frame may be determined based on Tp. In addition, among key frame groups photographed in the same time zone, the presentation time of the next key frame whose presentation time of the key frame has become equal to or less than Tq is set as the initial value Ts, and the presentation of subsequent key frames is performed based on Ts. Time may be determined. Moreover, the presentation time of the image presented last among the key frame groups photographed in the same time zone may be set to Ts. The values of Ts and Tp may be calculated according to the number of images to be presented by setting the presentation time of the entire video content in advance. When successive key frame pairs are photographed in different time zones, the subsequent presentation time is determined independently of the presentation time of the previous key frame. For example, the initial value Ts may be set, or a random value within a specified range may be set. When the source moving image is an uncut moving image over a long time, it can be determined as a relevant key frame even if the shooting time zone of the moving image where the key frame is extracted is greatly different.

(7-2) Rules Regarding Effects, BGM, and Jingles Based on the identity of the shooting time zones of consecutive key frame pairs, the effects, BGM, and jingles to be inserted between the key frame pairs are determined. For example, when consecutive key frame pairs are photographed in the same time zone, special effects (such as dissolves and fades) registered in advance as effects with little visual change are inserted when switching key frames. When the image is taken at a different time, a special effect (DVE such as page turning or wipe) registered in advance is inserted as an effect having a large visual change when the key frame is switched. Also, for example, when consecutive key frame pairs are shot at the same time zone, the same BGM is played during presentation of the key frame pairs, and when shot at different time zones, the BGM is switched at the time of key frame switching. Stop or switch to a different BGM. A jingle may be inserted between key frames in different time zones. As a result, when consecutive key frame pairs are photographed in the same time zone, they are smoothly connected without any image or acoustic change. Therefore, the viewer can easily understand that the key frames have almost the same contents without any change. In addition, when the images are taken at different time periods, image and acoustic changes greatly change, so that the viewer can notice that the contents have changed and can concentrate on understanding the contents of the video content. As the presentation rule, any one of the above rules may be applied, or a plurality of rules may be used in combination. The video content generation unit 203 generates video content based on the presentation method information input from the presentation method determination unit 202 and the image information input from the video input unit 204.

[Other Embodiments]
As mentioned above, although embodiment of this invention was explained in full detail, the system or apparatus which combined the separate characteristic contained in each embodiment how was included in the category of this invention.

  In addition, the present invention may be applied to a system composed of a plurality of devices, or may be applied to a single device. Furthermore, the present invention can also be applied to a case where an information processing program that implements the functions of the embodiments is supplied directly or remotely to a system or apparatus. Therefore, in order to realize the functions of the present invention on a computer, a program installed in the computer, a medium storing the program, and a WWW (World Wide Web) server that downloads the program are also included in the scope of the present invention. .

Claims (15)

Extraction means for extracting at least two partial videos or still images from the source video;
Determining means for determining a method of presenting the at least two partial moving images or still images extracted by the extracting means based on characteristics of the source moving images;
Generating means for generating video content including the at least two partial moving images or still images based on the presentation method determined by the determining means;
An information processing apparatus comprising:   The information processing apparatus according to claim 1, wherein the generation unit generates the video content that continuously presents the at least two still images.   The information processing apparatus according to claim 1, wherein the determination unit determines a presentation time of the at least two partial moving images or still images in the video content.   The said determination means determines the other presentation time based on one presentation time, when the said at least 2 partial moving image or a still image has a relation with each other. Information processing device.   When the at least two partial moving images or the objects included in the still image are the same, the determining means includes the partial moving image or the still image inserted after the presentation time of the previously inserted partial moving image or the still image. The information processing apparatus according to claim 4, wherein an image presentation time is shortened.   The said determination means determines presentation time independently, when the said at least 2 partial moving image or a still image does not have a mutual relationship, The presentation time of Claim 3 thru | or 5 characterized by the above-mentioned. Information processing device.   The information processing apparatus according to claim 1, wherein the determination unit determines an effect or a jingle when switching the at least two partial moving images or still images in the video content.   The said determination means determines the effect or jingle different from the case where it does not have a relationship, when the said at least 2 partial moving image or a still image has a relationship with each other. Information processing device.   The information processing apparatus according to claim 1, wherein the determining unit determines background music of the at least two partial moving images or still images in the video content. The determining means includes
Determination means for determining whether or not the object included in the at least two partial moving images or still images is related based on the source moving image;
A selection means for selecting a presentation method different from the case where the object is not related when the object is related;
The information processing apparatus according to any one of claims 1 to 9, further comprising: The determination means includes
The information processing apparatus according to claim 10, wherein whether or not the objects included in the at least two partial moving images or still images are the same is determined based on the source moving image. The determination means includes
The information processing apparatus according to claim 10 or 11, wherein it is determined based on the source moving image whether or not the shooting method of the object in the at least two partial moving images or still images is the same. The determination means includes
13. The method according to claim 10, wherein whether or not the objects included in the at least two partial moving images or the still image have commonality is determined based on an acoustic feature of the source moving image. The information processing apparatus according to item 1. An extraction step of extracting at least two partial videos or still images from the source video;
A determination step of determining a method of presenting the at least two partial moving images or still images extracted in the extracting step based on characteristics of the source moving images;
Generating a video content including the at least two partial moving images or still images based on the presentation method determined in the determining step;
An information processing method comprising: An extraction step of extracting at least two partial videos or still images from the source video;
A determination step of determining a method of presenting the at least two partial moving images or still images extracted in the extracting step based on characteristics of the source moving images;
Generating a video content including the at least two partial moving images or still images based on the presentation method determined in the determining step;
An information processing program for causing a computer to execute.

Images