JP2003169298A - Video abbreviated playback device, video abbreviated playback method, program for this method, and recording medium storing this program - Google Patents

Abstract

(57) [Summary] [Problem] To shorten and reproduce a video, it is easy for a user to see the current scene, and it is possible to reproduce video information at high speed by reducing missing important scenes. Enable playback within the time specified by the user. SOLUTION: A sound / video separating circuit 1 separates sound information and image information from video information, a sound recognizing means 2 recognizes a human voice from the separated sound information, and an image recognizing means 3 is separated. The transition part of the scene is recognized from the image information. The video control means 4 increases or decreases the threshold value for the section power of the recognized sound information in response to the user's designation of the total playback time, and the sum of the sections above the threshold is longer than the user-specified total playback time. Select a sound information by setting a threshold so that it is the shortest and the maximum playback time, combine both the sound information and the recognized image information, and determine the image part necessary for playback in the video part I do. The sound / image synchronizing means 5 synchronizes and reproduces the sound and the image controlled by the video control means.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a video shortening / reproducing apparatus equipped with a calculation mechanism for shortening and editing the reproduction time of video information.

[0002]

2. Description of the Related Art In a conventional VCR or video playback tool, video is played back at a fixed speed during high speed playback or fast forward playback. The speed is 2 to 20 times the standard speed. At that time, the sound is often not reproduced at all, and when it is reproduced, there is a case where only a part of the sound is reproduced intermittently.

In addition, in the time saving function of the SANYO video, the image is reproduced at a high speed at a constant speed, and the sound is speeded up except for the silent section so that the unreproducible sound is reduced as much as possible.

[0004] In addition to this, research for summarizing video is also being conducted. For example, `` Michael A. Smith and Takeo Kanad
e, Video Skimming and Characteriza-tion through the
Combination of Image and Language Understanding T
echniques, Proceedings of Computer Vision and Patt
ern Recognition, pp.775-781, June 1997 ”.

[0005]

The high-speed reproduction in the conventional system attaches importance to the image, and the image is always reproduced at a high speed at a constant speed, and the audio is reproduced in a fast-moving manner or the silent part is skipped. I was doing it.

In the case of using the fast-forwarding reproduction of the voice, the fast-forwarding reproduction in which a human question can be heard is about double speed at the most, and it is often impossible to hear the original voice.

Also, in the case of the method of skipping the silent part,
The image part is always played at a high speed at a constant speed, and the sound is played without being synchronized with the image, so that the sound and the image do not match, which is a factor that hinders understanding.

Further, in the conventional video summarizing system, the main purpose is to generate the structural information of the video, and the user is asked to display and select an image or text of the representative scene. The mainstream method is to reproduce the scene. In this case, in the case of a video that the user has never seen, even if the user looks at the image, there is often no way to understand the inside. There is also the possibility that the desire to see the main part will fade away.

Furthermore, it is not possible for the user to specify the total reproduction time and to collect the images within that time.

An object of the present invention is to enable a user to easily understand a current scene, to reduce the possibility of missing an important scene, to reproduce video information at high speed, and to reproduce it within a time designated by the user. Video shortening playback device,
A method, a program, and a recording medium are provided.

[0011]

In the present invention, importance is attached to voice. First, a section containing a human voice is extracted from the sound information. This is because the section where the human voice is uttered is often important, and the image of the section where the voice is uttered is also often important.

Next, after extracting the human voice section, the power (energy) of the voice of the section is calculated this time. This applies to human voices and BGM as well, but voices and music in important parts tend to have higher power than surrounding parts, so we use them. The power of this section is played back in a section having a value exceeding a certain threshold value, and the sections below that are not played back, and the total playback time is adjusted by increasing or decreasing the value of this threshold value. That is, if the threshold value is lowered, more sections are played back and the total playback time becomes longer, and if the threshold value is raised, less sections are played back, and as a result, the total playback time becomes shorter. Therefore, by adjusting this threshold value, the reproduction time is adjusted within the time specified by the user.

When the threshold value is changed, it does not always reach the time designated by the user, but it often becomes shorter than that time. In addition, the voice section that exceeds the threshold appears in the video. Once the audio section to be played back is decided, this time the location of the scene change is checked from the image information, the relationship between the section of the sound to be played and the scene is checked, and the section of the sound to be played is actually a part of the scene, and If the total reproduction time specified by the user is not exceeded even if the entire scene is reproduced, not only the section of the sound but the entire scene is targeted for reproduction. In this way, each voice section exceeding the threshold value is examined within the range not exceeding the total reproduction time, and the section to be reproduced by the scene is corrected.

When the section to be reproduced is determined in this way, the reproduction is actually performed. At that time, the image reproduces the image corresponding to the sound reproduced in the original video. As a result, the sound and the image are reproduced without any deviation.

According to the above video shortening reproduction system, in the present invention, reproduction is mainly performed on a portion including a sound which is considered to be relatively important, and at that time, a scene such as a conversation is kept by keeping the sound and the video synchronized at all times. However, it makes it possible to reproduce the image without unnaturalness. Furthermore, by playing back the audio part at a slightly higher speed, it is possible to view the audio part at a faster time than usual. Further, even in the case of a video without sound, the possibility that the user misses an important scene is reduced by reproducing the beginning portion of the scene transition. Also, by shortening the image within the time set by the user, busy users can certainly see the image with a little free time, and it is also possible to effectively use the time. It features a method, a program and a recording medium.

(1) A video shortening / reproducing device for shortening and reproducing video information, wherein a sound / video separating means for separating the video information into sound information and image information, and a person from the separated sound information. Sound recognition means for recognizing the voice, an image recognition means for recognizing a transition part of a scene from the separated image information, and a sound reproduction of the recognized sound information in response to designation of a total reproduction time from a user. Increase or decrease the threshold for the section power, and if the sum of the sections above the threshold is shorter than the total playback time specified by the user, set the threshold so that the maximum playback time is reached and select sound information. Video control means for collecting both information and the recognized image information and determining an image portion required for reproduction in the video portion, and reproducing the sound and image controlled by the video control means in synchronization with each other. Sound / image synchronization Video shortening reproducing apparatus characterized by comprising a stage.

(2) A video shortening reproduction method for shortening and reproducing video information, wherein a sound / video separation process of separating video information into sound information and image information, and a person from the separated sound information. Of the recognized sound information in response to the sound recognition process of recognizing the voice, the image recognition process of recognizing the transition part of the scene from the separated image information, and the designation of the total reproduction time from the user. Increase or decrease the threshold for the section power, and if the sum of the sections above the threshold is shorter than the total playback time specified by the user, set the threshold so that the maximum playback time is reached and select the sound information. Both the information and the recognized image information are put together, and a video control process for determining an image part necessary for reproduction in the video part and a sound and an image controlled by the video control process are reproduced in synchronization with each other. Sound / image synchronization Video shortened playback method characterized by having a degree.

(3) A program for a video shortening reproduction method, characterized in that the processing procedure in the video shortening reproduction method is configured as a program that can be executed by a computer.

(4) A recording medium recorded with a video shortening reproducing method, characterized in that a program for causing a computer to execute the processing procedure in the video shortening reproducing method is recorded on a recording medium readable by the computer. .

[0020]

BEST MODE FOR CARRYING OUT THE INVENTION An embodiment of the present invention will be described below with reference to the drawings. FIG. 1 is a block configuration diagram of this embodiment.

In the figure, since sound and image information are mixed in ordinary video information, they are first separated in the sound / video separation circuit 1. The separation circuit 1 can use the audio / video separation system in television.

Next, with respect to the separated sounds, the sound recognizing means 2 extracts a part including a human voice. The sound recognition means 2 is realized by passing a filter having a pass band in the human voice band.

With respect to the separated images, the image recognition means 3 detects the transitional part of the scene. The image recognition unit 3 can make a determination based on whether or not the amount of change in luminance or its pattern becomes large, for example.

Next, in the video control means 4, the information of both the sound (sound) separated by the sound recognition means 2 and the image extracted by the image recognition means 3 is put together, and the image portion necessary for reproduction within the video portion is collected. To decide. The image portion necessary for reproduction is the beginning portion of the detected scene transition and the image of the portion where the extracted sound is emitted.

Further, the video control means 4 increases or decreases the threshold value for the section power of the sound with respect to the separated sound in response to the designation of the total reproduction time from the user, and sums the sections above the threshold value. Is shorter than the total playback time specified by the user, and the threshold is set to the maximum value. Specifically, the following processing is performed.

Select one from each section to be reproduced selected by the threshold setting. The scene change information obtained from the image information corresponding to the section is examined. If the section to be played back is a part of a scene, and even if all the scenes are played back, if the total playback time specified by the user is shorter, the playback part is set to the entire scene.

The above process is sequentially executed for another reproduction section, and the reproduction time is set to be the longest within the total reproduction time.

FIG. 2 shows an example of the increase / decrease in the total reproduction time by changing the threshold value. With respect to the sound power signal, the lower the threshold value, the longer the reproduction section, and the higher the threshold value, the shorter the difference section. Become.

Finally, the sound / image synchronizing means 5 synchronizes the sound and the image, and outputs the reproduced image with the time shortened. As a method of synchronizing the sound with the necessary image portion, the reproduced image is configured so that the time of the sound from the beginning of the original video information matches the time of the image portion.

FIG. 3 is a flowchart showing an algorithm which is an embodiment of the method of the present invention. The algorithm is as follows.

(S1) Sound / video separation The sound information and the image information are separated from the video information.

(S2) Sound Recognition and Image Recognition The following processing is performed on each of the separated sound information and image information.

Regarding the sound information, the voice part is detected from the sound information.

A scene change is detected for image information.

(S3) Video control Information about both the detected voice and image is put together to determine the image portion required for reproduction in the video portion. Further, the maximum threshold for the section power of the sound is set in response to the designation of the total reproduction time from the user.

(S4) Sound / image synchronization Play while synchronizing the sound and image.

The present invention can be realized by using a computer by configuring some or all of the processing functions of the method shown in FIG. 3 as a program. A computer-readable recording medium, such as an FD (floppy disk: registered trademark), stores a program for realizing the processing function of each part of the computer or a program for causing the computer to execute the processing procedure. MO, ROM, memory card, CD, DVD,
The data can be recorded on a removable disk or the like and can be saved or provided, and can be distributed via a communication network such as the Internet.

[0038]

As described above, according to the present invention, the video can be shortened within an arbitrary time designated by the user, so that the video content can be confirmed by the time before the next conference starts, and Can be used for home use, such as watching the contents of the broadcast last week again before the program starts.

Also, unlike the conventional summarization system,
Since the system automatically summarizes, it is possible to suddenly see a shortened video without looking inside the video.

Further, since the sound and the image of the image reproduced according to the present invention are always reproduced in coincidence with each other, it is possible to see the image which is easy to understand.

[Brief description of drawings]

FIG. 1 is a block diagram showing an embodiment of the present invention.

FIG. 2 is an example of increase / decrease in total reproduction time due to change in threshold value in the embodiment.

FIG. 3 is a flowchart showing an embodiment of the present invention.

[Explanation of symbols]

1 ... Sound / video separation circuit 2 ... Sound recognition means 3 ... Image recognition means 4 ... Image control means 5. Sound / image synchronization means

─────────────────────────────────────────────────── ─── Continuation of front page (51) Int.Cl. ⁷ Identification code FI theme code (reference) H04N 5/7826 H04N 5/91 C 5/91 5/782 A (72) Inventor Toshihiro Takada Chiyoda-ku, Tokyo 2-3-1, Otemachi Nippon Telegraph and Telephone Corporation (72) Inventor Shunji Sugawara 2-3-1, Otemachi, Chiyoda-ku, Tokyo Nihon Telegraph and Telephone Corporation F-term (reference) 5C018 AB03 AB13 5C052 AA01 AC01 DD04 DD06 5C053 FA14 HA21 HA27 LA06 5D015 DD05 KK02

Claims (4)

[Claims] 1. A video shortening / reproducing device for shortening and playing back video information, comprising sound / video separating means for separating the video information into sound information and image information; Sound recognition means for recognizing a voice, image recognition means for recognizing a transitional part of a scene from the separated image information, and a section of the recognized sound information in response to designation of a total reproduction time from a user. Increase or decrease the threshold for power, and if the sum of the sections above the threshold is shorter than the total playback time specified by the user, set the threshold so that the maximum playback time is reached and select sound information. And a video control unit that collects both of the recognized image information and determines an image portion required for reproduction in the video portion, and a sound that is reproduced in synchronization with the sound controlled by the video control unit and the image.・ Image synchronizer Video shortening reproducing apparatus characterized by comprising and. 2. A video shortening reproducing method for shortening and reproducing video information, comprising a sound / video separation process of separating video information into sound information and image information, and a human being selected from the separated sound information. A sound recognition step of recognizing a voice, an image recognition step of recognizing a transition part of a scene from the separated image information, and a section of the recognized sound information in response to a user's designation of a total reproduction time. Increase or decrease the threshold for power, and if the sum of the sections above the threshold is shorter than the total playback time specified by the user, set the threshold so that the maximum playback time is reached and select sound information. And a video control process that determines both of the recognized image information and the image part required for reproduction within the video part, and a sound that is reproduced by synchronizing the image and the sound controlled by the video control process.・ Image sync error Video shortened playback method characterized by comprising and. 3. A program for a video shortening reproduction method, characterized in that the processing procedure in the video shortening reproduction method according to claim 2 is configured as a program that can be executed by a computer. 4. A video shortening and reproducing method, characterized in that a program for causing a computer to execute the processing procedure in the video shortening and reproducing method according to claim 2 is recorded in a recording medium readable by the computer. Recording medium.