JP2015082691A - Video editing device, video editing method, and video editing program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To edit video in consideration of condition on a viewer who is viewing a video.SOLUTION: A video editing device 10 records recording time of a point in which a viewer takes distinctive action from a viewer video which shows condition of the viewer who is viewing a video content. Then, the video editing device 10 extracts video image data of a time zone of a predetermined range from the point corresponding to the recording time when the distinctive action is taken from the video content. And then, the video editing device 10 prepares an edited video in which the extracted video image data is connected in time series.

Description

  The present invention relates to a video editing apparatus, a video editing method, and a video editing program.

  Conventionally, a technique for creating a digest video such as video content has been proposed. Patent Document 1 discloses a technique for extracting a portion where emotion (sad, fun) such as a character is expressed in video content and creating a digest video.

JP 2009-1111938 A

  However, the digest video created by the above-described technique is merely an extracted portion where the emotion of the character in the video content is expressed. That is, the state on the viewer side of the video is not taken into consideration. SUMMARY OF THE INVENTION Accordingly, it is an object of the present invention to solve the above-described problems and to edit a video in consideration of the state on the viewer side of the video.

  In order to solve the above-described problems, the present invention provides a characteristic behavior detection unit that detects a characteristic behavior of the viewer from a viewer video obtained by capturing the viewer when viewing the video content, and the video content A video identification unit for recording a location corresponding to the shooting time of the viewer video where the characteristic behavior is detected, and a location corresponding to the shooting time of the viewer video where the characteristic behavior is detected A video extracting unit that extracts video data in a predetermined time period from the video content, and an edited video creating unit that creates an edited video by linking the extracted video data in time series. The featured video editing device.

  According to the present invention, it is possible to edit a video in consideration of the state on the viewer side.

FIG. 1 is a diagram for explaining the outline of the present embodiment. FIG. 2 is a diagram illustrating a configuration example of a system including the video editing apparatus of FIG. FIG. 3 is a flowchart showing a processing procedure of the video editing apparatus of FIG. FIG. 4 is a diagram for explaining the details of the processing of the characteristic behavior detection unit of FIG. FIG. 5 is a diagram for explaining the details of the processing of the video specifying unit in FIG. FIG. 6 is a diagram for explaining the details of the processing of the video extraction unit in FIG. FIG. 7 is a diagram for explaining the details of the processing of the edited video creation unit in FIG. FIG. 8 is a diagram illustrating a computer that executes a video editing program.

(Overview)
Hereinafter, embodiments of the present invention will be described with reference to the drawings. First, an outline of the present embodiment will be described. FIG. 1 is a diagram for explaining the outline of the present embodiment. Here, a case where the video content is a television program and the television program receiver is installed outside the video editing apparatus 10 will be described. The video editing apparatus 10 will be described with respect to a case where a viewer who views video content is photographed by an external camera (photographing apparatus). Here, a case will be described as an example where the recording of the video content and the shooting of the video including the state of the viewer of the video content are started simultaneously. Below, the outline | summary of the process of the video editing apparatus 10 is demonstrated.

As shown in FIG.
(1) First, at the same time that the viewer starts watching the television program, the video editing apparatus 10 records the video of the television program and the video of the viewer.
(2) The video editing apparatus 10 detects a viewer's characteristic behavior from the viewer's video and creates a list of times at the time of detection.
(3) Based on the list, the video editing apparatus 10 identifies a video of a television program corresponding to each time in the list, and extracts a video in a predetermined range before and after that time.
(4) The video editing apparatus 10 joins the plurality of extracted videos to create an edited video (summary content) of the recorded television video.

  By doing in this way, the video editing apparatus 10 can create an edited video of video content based on the behavior of the viewer. For example, when the viewer of the video content takes an action (action) such as laughing or moving a hand or a foot, the video editing apparatus 10 connects the partial videos when the action is taken in time series. Create a combined edited video. Thereby, for example, it is possible to create an edited video in which portions that the viewer thinks are interesting or emotions are shaken.

(Constitution)
Next, the configuration of the video editing apparatus 10 will be described with reference to FIG. FIG. 2 is a diagram illustrating a configuration example of a system including the video editing apparatus of FIG. Here, a case where the tuner 2 is used as a video content receiver will be described as an example.

  As shown in FIG. 2, the system includes a camera (photographing device) 1, a tuner 2, a display device (output device) 3, and a video editing device 10.

  The camera 1 captures the state of the viewer who is viewing the video content. The camera 1 may have a function of recording not only the viewer's video but also the viewer's voice. Then, the camera 1 outputs the captured video of the viewer to the video editing device 10.

  The tuner 2 receives video content such as a TV program and outputs it to the video editing device 10.

  The display device 3 outputs the video and audio output from the video editing device 10. The display device 3 displays, for example, video content, viewer video, edited video, and the like.

  The video editing apparatus 10 includes an input / output unit 11, a control unit 12, and a storage unit 13.

  The input / output unit 11 controls an input / output interface with the camera 1, the tuner 2, the display device 3, and the like. The input / output unit 11 may have a communication interface function for performing communication via a network such as the Internet.

  The control unit 12 controls the entire video editing apparatus 10. The control unit 12 includes a reception processing unit 120, a recording unit 121, a playback unit 122, and an editing unit 123.

  The reception processing unit 120 receives data from an external device. For example, the reception processing unit 120 receives a viewer's video from the camera 1 and video content from the tuner 2.

  The recording unit 121 stores the data received by the reception processing unit 120 in the storage unit 13. For example, the recording unit 121 stores the viewer's video and video content received by the reception processing unit 120 in a predetermined area of the storage unit 13.

  The reproduction unit 122 outputs the information stored in the storage unit 13 to the display device 3 via the input / output unit 11. For example, the playback unit 122 outputs video content in the storage unit 13, viewer video, edited video, and the like to the display device 3 based on an instruction input from the input / output unit 11.

  The editing unit 123 creates an edited video using the video content stored in the storage unit 13 and the video of the viewer who is viewing the video content. Specifically, when the editing unit 123 detects a viewer's characteristic behavior from the viewer's video, the editing unit 123 extracts a partial video of the video content at the time when the behavior is detected. Then, the editing unit 123 connects the extracted partial videos to create an edited video.

  The editing unit 123 includes a feature behavior detecting unit 124, a video specifying unit 125, a video extracting unit 126, and an edited video creating unit 127.

  The characteristic behavior detection unit 124 detects the viewer's characteristic behavior from the viewer's video (viewer video) stored in the storage unit 13. Then, the characteristic behavior detection unit 124 records the shooting time of the viewer video in which the characteristic behavior is detected. For example, the characteristic behavior detection unit 124 creates a list that shows a list of viewer video shooting times at which characteristic behavior is detected, and an elapsed time from the start of viewer video shooting. The characteristic behavior here is, for example, that the viewer moves hands or feet, or the viewer changes facial expressions. Details of detection of characteristic behavior by the characteristic behavior detection unit 124 will be described later with specific examples.

  The video specifying unit 125 specifies the location of the video content corresponding to the shooting time of the viewer video (or the elapsed time from the start of shooting the viewer video) at which the characteristic behavior is detected by the characteristic behavior detection unit 124. For example, when the time shown in the list is the shooting time of the viewer video when the viewer's characteristic behavior is detected, the video specifying unit 125 corresponds to the video shooting time of the viewer video shown in the list. The location of the content (for example, the elapsed time from the start of video content recording) is specified.

  The video extraction unit 126 extracts videos in a predetermined time range before and after each video content location specified by the video specification unit 125.

  The edited video creation unit 127 creates an edited video by linking the video extracted by the video extraction unit 126 in time series, and stores the edited video in the storage unit 13. The created edited video may be output to the display device 3 or the like via the input / output unit 11.

  The storage unit 13 stores the data output from the control unit 12. For example, the storage unit 13 stores video content output from the control unit 12, viewer video, edited video, and the like in a predetermined area.

  Note that the video editing apparatus 10 may be connected to an input device that receives inputs such as reception and recording (recording) of video content and viewer video, and instructions for creating edited video.

(Processing procedure)
Next, the processing procedure of the video editing apparatus 10 will be described with reference to FIG. FIG. 3 is a flowchart showing a processing procedure of the video editing apparatus of FIG.

  First, the reception processing unit 120 of the video editing apparatus 10 receives video content and a viewer's video (viewer video) from the tuner 2 or the like (S1). Then, the recording unit 121 starts recording the video content and viewer video received in S1 (S2). Next, the recording unit 121 stops recording the video content and the viewer video (S3). Thereafter, upon receiving an input of a video content editing instruction (edited video creation instruction) via the input / output unit 11 of the video editing apparatus 10 (S4), the editing unit 123 starts editing the video content. That is, first, the characteristic behavior detection unit 124 of the editing unit 123 detects the characteristic behavior of the viewer from the recorded viewer video (S5). Then, the characteristic behavior detection unit 124 creates a list of characteristic behavior times (S6). That is, the characteristic behavior detection unit 124 creates a list indicating the times at which the characteristic behavior is detected from the viewer video.

  After S6, the video identification unit 125 acquires video content (content viewed by the viewer) from the storage unit 13 (S7). Then, the video identification unit 125 identifies the location of the video content corresponding to each time indicated in the list created in S6 (for example, the elapsed time from the start of video content recording) (S8). Next, the video extraction unit 126 extracts partial videos before and after the location of the video content specified in S8 for a predetermined time (S9). Then, the edited video creation unit 127 joins the partial videos extracted in S9 in time series to create an edited video (S10).

  By doing in this way, the video editing apparatus 10 can create an edited video in which portions where the viewer of the video content has taken a characteristic action are collected.

  Next, details of processing performed by the control unit of the video editing apparatus 10 will be described with reference to specific examples with reference to FIGS. FIG. 4 is a diagram for explaining the details of the processing of the characteristic behavior detection unit of FIG. FIG. 5 is a diagram for explaining the details of the processing of the video specifying unit in FIG. FIG. 6 is a diagram for explaining the details of the processing of the video extraction unit in FIG. FIG. 7 is a diagram for explaining the details of the processing of the edited video creation unit in FIG.

  First, the characteristic behavior detection unit 124 of the video editing apparatus 10 extracts still images at set time intervals from the recording start time of the viewer video (S11 in FIG. 4). Then, the characteristic behavior detection unit 124 determines whether or not it is the last still image of the viewer video (S12). If it is the last still image (“last” in S12), the process proceeds to S21 in FIG. If it is not the still image (“not last” in S12), the two still images before and after the extracted image are compared, and it is determined whether or not the change in the image is equal to or greater than the set threshold (S13). Here, when the characteristic behavior detection unit 124 determines that the change in the extracted image is equal to or greater than the set threshold value (“threshold value or more” in S13), the intermediate time between the two still images is listed (S14). ), Return to S11. In S13, the characteristic behavior detection unit 124 compares the two still images before and after and determines that the change in the image is less than the predetermined threshold (“less than threshold” in S13), the process also returns to S11.

  The characteristic behavior detecting unit 124 repeats the above processing, for example, to create a list 401 indicating the time when the characteristic behavior of the viewer is detected. In the list 401 shown in FIG. 4, Action1 is detected at A1 hour A2 minute A3 second, and Action2 is detected at B1 hour B2 minute B3 second.

  Turning to the description of FIG. When the characteristic behavior detecting unit 124 determines that the still image extracted in S11 of FIG. 4 is the last still image (“last” in S12), the video specifying unit 125 reads and reads the list with each time. It is determined whether or not there are still items (S21). Here, when the video specifying unit 125 determines that there is no item that has not been read out in the list (“None” in S21), the process proceeds to S31 in FIG. 6 and when it is determined that there are still items that have not been read out in the list ( In step S21, "there is still"), one item in the list (one item in the item group that has not yet been read) is read, and the part of the video content corresponding to the time of the item is marked (S22). Then, the process returns to S21.

  The video identification unit 125 repeats the above processing, so that, for example, recording of video content indicated by reference numeral 501 at each time (Action 1 time, Action 2 time, Action 3 time) in the list 401 shown in FIG. It is marked between the time and the recording stop time, and a state indicated by reference numeral 502 is obtained.

  Turning to the description of FIG. When it is determined in S21 of FIG. 5 that there is no item that has not been read in the list (“None” in S21), the video extraction unit 126 reads the mark of the video content, and whether there is a mark that has not been read yet. Is determined (S31 in FIG. 6). Here, when the video extraction unit 126 determines that there is a mark that has not been read (“is still” in S31), one item of the mark that has not been read is read, and a partial video before and after the mark is extracted for a predetermined time. (S32). Thereafter, the process returns to S31, and if the video extraction unit 126 determines that there is no unread mark in the video content (“None” in S31), the process proceeds to S41 in FIG.

  The video extraction unit 126 repeats the above processing, for example, as shown by reference numeral 601 in FIG. 6, the partial video images before and after each of the video content marks (time of Action1, time of Action2, time of Action3) ( Partial videos 1, 2, 3) are extracted for a predetermined time and stored individually in the storage unit 13.

  Turning to the description of FIG. The edited video creation unit 127 reads the partial video extracted in S32 of FIG. 6 and determines whether there is a partial video that has not been read (S41 of FIG. 7). Here, when the edited video creation unit 127 determines that there is a partial video that has not yet been read (“is still” in S41), the subsequent partial video is joined to the previous partial video among the read partial videos. (S42). That is, the edited video creation unit 127 connects the partial videos cut out from the video content in time series. Then, the process returns to S41. On the other hand, when the edited video creation unit 127 determines that there is no partial video that has not been read (“No” in S41), the stitched video is stored in the storage unit 13 as an edited video (S43).

  When the edited video creation unit 127 repeats the above processing, for example, as indicated by reference numeral 701 in FIG. 7, partial video (partial video) corresponding to each of the time of Action1, the time of Action2, and the time of Action3 in the video content. 1 to 3) are created to create an edited video. The edited video may include metadata indicating what action each partial video has taken at what time, as exemplified by reference numeral 701.

  According to the video editing apparatus 10 described above, it is possible to create an edited video of a portion of the video content where a viewer of the video content has taken a characteristic action. For example, the video editing apparatus 10 can create a digest version of a particularly impressed part or an interesting part of the video content viewed by the viewer.

(Other embodiments)
The video content has been described as an example of a broadcast program such as a television program, but is not limited thereto. For example, the video content may be video content recorded in advance. In this case, the playback unit 122 of the video editing apparatus 10 outputs the video content stored (recorded) in advance in the storage unit 13 to the display device 3. Then, the camera 1 captures the state of the viewer who is viewing the reproduced video content and outputs it to the video editing apparatus 10.

  Moreover, although the video editing apparatus 10 decided to create the edited video of the video content after putting a mark in the video content where the viewer took a characteristic action, it is not limited to this. For example, the video editing apparatus 10 stores the video content in the storage unit 13 in a state where marks are put in places where the viewer has taken a characteristic action in the video content. That is, the video editing apparatus 10 inserts a chapter mark at a place where a characteristic action is taken in the video content. Then, when a viewer or the like instructs the video editing apparatus 10 to play back the recorded video content, the part where the mark is present may be detected and jumped to this part to be played back. . That is, when an instruction to reproduce video content specifying a chapter mark is input to the video editing device 10, the reproducing unit 122 outputs the video after the portion where the chapter mark is inserted in the video content to the display device 3. . Further, the feature behavior detecting unit 124 may add meta information indicating a summary of a feature behavior or the like at a marked portion in the video content. In this way, a user (for example, a viewer) of the video editing apparatus 10 can easily instruct what action the viewer has taken in the video content.

  In addition, the area in which the characteristic behavior detection unit 124 detects a change from the viewer video is not a whole area of the viewer video but a predetermined area (for example, a viewer's face, hand, or foot area). Also good. By doing in this way, the characteristic action detection part 124 can detect a viewer's characteristic action accurately.

  In addition, the characteristic behavior detection unit 124 detects a characteristic behavior when there is a change of a predetermined threshold or more with respect to one of the viewer's video and audio in the viewer video, or a combination thereof. May be. Also by doing in this way, the characteristic behavior detection unit 124 can accurately detect the characteristic behavior of the viewer.

  In addition, the characteristic behavior detection unit 124 stores a pattern indicating a characteristic behavior movement in the storage unit 13, and detects a characteristic behavior when the viewer video has a motion corresponding to this pattern. You may make it do.

  In addition, the characteristic behavior detection unit 124 sets the region for detecting the characteristic behavior from the viewer video, the video, the audio threshold, and the motion pattern in the storage unit 13 as the characteristic behavior setting information in the video editing device 10. In addition, a user (for example, a viewer) of the video editing apparatus 10 may be rewritable. By doing in this way, the user (for example, viewer etc.) of the video editing apparatus 10 can set the detection of characteristic behavior to the behavior desired by the user.

  In the embodiment described above, the case where the start of video content recording and the start of viewer video shooting is described as an example. However, the present invention is not limited to this. For example, when there is a difference between the recording start time of the video content and the shooting start time of the viewer video, the video specifying unit 125 takes into account the difference in the start time, and the list created by the characteristic behavior detecting unit 124 The location of the video content corresponding to each time shown in FIG.

(program)
It is also possible to create a program in which the processing executed by the video editing apparatus 10 according to the above embodiment is described in a language that can be executed by a computer. In this case, the same effect as the above-described embodiment can be obtained by the computer executing the program. Further, such a program may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be read by a computer and executed to execute the same processing as in the above embodiment. Hereinafter, an example of a computer that executes a video editing program that realizes the same function as the video editing device 10 will be described.

  FIG. 8 is a diagram illustrating a computer that executes a video editing program. As illustrated in FIG. 8, the computer 1000 includes, for example, a memory 1010, a CPU 1020, a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070. These units are connected by a bus 1080.

  The memory 1010 includes a ROM (Read Only Memory) 1011 and a RAM 1012. The ROM 1011 stores a boot program such as BIOS (Basic Input Output System). The hard disk drive interface 1030 is connected to the hard disk drive 1090. The disk drive interface 1040 is connected to the disk drive 1041. For example, a removable storage medium such as a magnetic disk or an optical disk is inserted into the disk drive 1041. For example, a mouse 1110 and a keyboard 1120 are connected to the serial port interface 1050. For example, a display 1130 is connected to the video adapter 1060.

  Here, as shown in FIG. 8, the hard disk drive 1090 stores, for example, an OS 1091, an application program 1092, a program module 1093, and program data 1094. Each table described in the above embodiment is stored in the hard disk drive 1090 or the memory 1010, for example.

  Further, the video editing program is stored in the hard disk drive 1090 as a program module in which a command executed by the computer 1000 is described, for example. Specifically, a program module describing each process executed by the video editing apparatus 10 described in the above embodiment is stored in the hard disk drive 1090.

  Data used for information processing by the video editing program is stored as program data, for example, in the hard disk drive 1090. Then, the CPU 1020 reads out the program module 1093 and the program data 1094 stored in the hard disk drive 1090 to the RAM 1012 as necessary, and executes the above-described procedures.

  The program module 1093 and the program data 1094 related to the video editing program are not limited to being stored in the hard disk drive 1090. For example, the program module 1093 and the program data 1094 are stored in a removable storage medium and read by the CPU 1020 via the disk drive 1041 or the like. May be issued. Alternatively, the program module 1093 and the program data 1094 related to the video editing program are stored in another computer connected via a network such as a LAN (Local Area Network) or a WAN (Wide Area Network), and are transmitted via the network interface 1070. May be read by the CPU 1020.

2 tuner 3 display device 10 video editing device 11 input / output unit 12 control unit 13 storage unit 120 reception processing unit 121 recording unit 122 playback unit 123 editing unit 124 characteristic behavior detection unit 125 video specifying unit 126 video extraction unit 127 edited video creation unit

Claims (4)

A characteristic behavior detection unit for detecting a characteristic behavior of the viewer from a viewer video obtained by photographing the viewer when viewing video content;
A video specifying unit for recording a portion corresponding to a shooting time of the viewer video in which the characteristic behavior is detected in the video content;
A video extraction unit that extracts video data of a predetermined time range from a location corresponding to a shooting time of the viewer video in which the characteristic behavior is detected, from the video content;
An edited video creation unit that creates an edited video by linking the extracted video data in time series;
A video editing apparatus comprising: The viewer video includes an image of the viewer and the audio of the viewer,
The characteristic behavior detection unit
The video editing apparatus according to claim 1, wherein a characteristic action is detected when a predetermined amount of change occurs in at least one of the image and the sound. Detecting a characteristic behavior of the viewer from a viewer video of the viewer when viewing the video content;
A step of recording a portion corresponding to a shooting time of the viewer video in which the characteristic behavior is detected in the video content;
Extracting video data of a predetermined time range from the video content from a location corresponding to a shooting time of the viewer video in which the characteristic behavior is detected;
Creating an edited video in which the extracted video data is connected in time series; and
A video editing method, wherein the video editing apparatus executes the above. Detecting a characteristic behavior of the viewer from a viewer video of the viewer when viewing the video content;
A step of recording a portion corresponding to a shooting time of the viewer video in which the characteristic behavior is detected in the video content;
Extracting video data of a predetermined time range from the video content from a location corresponding to a shooting time of the viewer video in which the characteristic behavior is detected;
Creating an edited video in which the extracted video data is connected in time series; and
A video editing program characterized by causing a computer to execute.

Images