JP2021164094A - Recording device and recording system using it - Google Patents

Abstract

A recording device and a recording system using the recording device are provided, in which indexes using video signals and audio signals are linked to improve usability when using the indexes. A recording device for recording an input video and an index in the input video, wherein the index is an annotation index using a video signal of the input video and an audio index using an audio signal accompanying the input video, If the generation time of the audio index and the annotation index overlap, each association is performed, and by selecting either the associated audio index or annotation index, at the time of the start point of the generation time of the selected index It is configured to record the corresponding time. [Selection drawing] Fig. 11

Description

The present invention relates to an image and a recording device that records an index in the image.

Conventionally, an index used when searching for or playing back a specific scene in a recorded video is known.

Such an index is generated based on the color, shape, movement of the subject in the video, characters, motion events, etc. using the video signal, and these are added to the video. It's being used.

Further, there is Patent Document 1 as a background technique for generating an index using an audio signal. Patent Document 1 describes a device that automatically generates an index corresponding to various feature items generated during an event using an audio signal, and includes each analysis unit that analyzes the power level and frequency of audio data. A configuration including each feature extraction unit that extracts features from each and a portion that generates and outputs an index from the obtained features is disclosed.

Japanese Unexamined Patent Publication No. 2001-143451

The indexes generated by using the video signal and the audio signal described above are assumed to be used independently, and their cooperation is not considered. In addition, usability such as operability when using the index was not considered.

The present invention has been made in view of these problems, and an object of the present invention is to improve usability when using indexes by linking indexes using video signals and audio signals.

The present invention is, for example, a recording device that records an input video and an index in the input video, and the index includes an annotation index using the video signal of the input video and an audio signal accompanying the input video. If the voice index used and the generation time of the voice index and the annotation index overlap, the selected index is generated by associating them with each other and selecting either the associated voice index or the annotation index. The configuration is such that the time corresponding to the time of the start point of the time is recorded.

According to the present invention, it is possible to provide a recording device capable of improving usability when using an index by linking indexes using video signals and audio signals, and a recording system using the index.

It is a block diagram of the recording system using the recording apparatus in Example 1. FIG. It is a functional block block diagram of the recording apparatus in Example 1. FIG. It is a figure explaining the determination of the annotation index in Example 1. FIG. It is a figure explaining the display format of the annotation index in Example 1. FIG. It is a figure which shows the display layout of the annotation index and the voice index in Example 1. FIG. It is a figure which shows the other display layout of the annotation index and the voice index in Example 1. FIG. It is a figure explaining the display format of the voice index to be displayed in Example 1. FIG. It is a flowchart of the voice index creation processing in Example 1. FIG. It is explanatory drawing about the link between the voice index and the annotation index in Example 1. FIG. It is a figure explaining the display format of the annotation index in Example 1. FIG. It is a figure explaining the display format of the link of the voice index and the annotation index in Example 1. FIG. It is a figure explaining the display format of the link when the difference between the generation time of the voice index and the annotation index in Example 1 is small. It is a figure explaining the display format of the link of the voice index and the annotation index in Example 1. FIG. It is a conceptual diagram explaining the recording of the input video and the storage structure of the annotation index and the audio index in Example 1. It is a data structure diagram of the voice index in Example 1. FIG. It is a conceptual diagram explaining the reproduction of the input video in the recording apparatus using the audio index in Example 2. FIG. It is a conceptual diagram explaining the integrated index in Example 3. FIG.

Hereinafter, examples will be described with reference to the drawings.

FIG. 1 is a configuration diagram of a recording system using the recording device in this embodiment. The recording system in FIG. 1 assumes, for example, a lecture recording system in which a video of a lecture at a school can be recorded and the student can review the recorded lecture content later.

In FIG. 1, reference numeral 10 denotes a recording device according to the present embodiment, which is an image acquisition device such as a projector 20 or a camera 30 that is an input source of a video to be recorded, and a microphone 40 that is an input source of audio accompanying the video to be recorded. Etc. are connected. The projector 20 has a function of adding handwritten characters, lines, circles, and other additional information to the display image by a touch pen or the like, and these additional information can be important scenes in the lecture. Hereinafter referred to as annotation information in the sense of supplementary information. Instead of the projector having the above function, a so-called electronic blackboard having the same function may be used. Reference numeral 50 denotes a data input PC (Personal Computer) for inputting teaching material data into the recording device 10, and may be a document camera or other device. Further, 60 is a server, and 70 is a viewing PC which is a viewing device for students to view the video recorded by the recording device 10 via the server 60.

FIG. 2 is a functional block configuration diagram of the recording device in this embodiment. In FIG. 2, the recording device 10 includes a video input unit 101, a channel allocation unit 102, an information input unit 103, an annotation processing unit 104, an audio input unit 105, an audio processing unit 106, an image processing unit 107, a recording unit 108, and a video output. It is composed of a part 109. Further, the video processing unit 107 is composed of a video compositing unit 110 and an index generation unit 111. Each of the above processing units is controlled by a control unit (not shown).

The video input unit 101 is an input interface unit for a projector, a PC, a camera, etc., and includes HMDI (High-Definition Multimedia Interface), VGA (Video Graphics Array), LAN (Local Area Network), SDI (Serial Digital Interface), and the like. Capture video from.

The channel allocation unit 102 allocates the video captured from the video input unit 101 to each channel for each of the plurality of video inputs.

The information input unit 103 is an input unit for annotation information for video, and is connected to a projector, an electronic blackboard, or the like via USB (Universal Serial Bus), and for example, input information by a touch pen is input.

The annotation processing unit 104 is a process of drawing the captured annotation information, for example, converting the input information by the touch pen into image information.

The voice input unit 105 is an interface unit for inputting voice data from a microphone or the like, and is connected to the microphone or the like by Audio I / O, Bluetooth (registered trademark) or the like.

The voice processing unit 106 processes the captured voice information as digital data, and further converts the voice into text by voice recognition. If the load of the voice recognition process is large, the text conversion may be performed by an external server connected by a LAN or the like via an external connection terminal.

The video processing unit 107 includes a video compositing unit 110 that synthesizes the video captured from the channel allocation unit 102 and the drawing information of the annotation information processed by the annotation processing unit 104, and an index generation unit 111. Further, it has a reproduction processing function of capturing the recorded video in the recording unit 108, which will be described later, and reproducing the recorded video.

The index generation unit 111 generates an annotation index and a voice index, which will be described later.

The recording unit 108 is an HDD or the like, and records the video synthesized by the video processing unit 107 and the index generated by the index generation unit 111.

The video output unit 109 is an interface for outputting the reproduced video signal of the video recorded in the recording unit 108 and the video signal from the video processing unit 107, and is a LAN for connecting to a server or the like or an external display device. Output in HMDI or VGA to a projector or electronic blackboard.

The hardware configuration of the recording device shown in FIG. 2 is realized by a device having a processing device (CPU), a storage device (memory, HDD, etc.), and an input / output interface (I / F). That is, the processing of each processing unit in FIG. 2 is executed by the CPU processing those processing programs stored in the memory by software. It should be noted that processing with a large load such as video processing may be performed by a dedicated signal processing processor.

Next, the annotation index generated by the index generation unit 111 will be described.

First, the annotation index is a snapshot of the moment of the input video, and a plurality of snapshots including the annotation information are stored together with the time information. Then, by selecting a predetermined annotation index, the video recorded from the time corresponding to the time of the selected snapshot can be played back or searched. Hereinafter, video reproduction using an index is referred to as index reproduction.

As a procedure for using the annotation index, first, the storage device 10 records the input video. This recording start instruction is given by selecting a button provided on the main body of the storage device 10 or selecting an icon on the screen of the output video of the storage device 10. Next, take a plurality of snapshots including the annotation information. This shooting is to acquire an image including annotation information from the video input to the storage device 10. For example, a teacher giving a lecture uses a touch pen or the like to display handwritten characters, lines, or circles on the displayed image. When annotation information such as a figure such as is drawn on the screen, a snapshot shooting instruction is given by selecting an icon on the screen of the output image of the storage device 10. The storage device 10 receives an instruction to take a snapshot, acquires an image at that time from the input video, and records the time together with the recording unit 108. This will generate an annotation index.

Next, when performing reproduction using the annotation index, the captured snapshot is reproduced in the storage device 10, and the snapshot including the annotation information to be reproduced is selected to correspond to the creation time of the selected snapshot. The recorded video can be played back from the time recorded.

Further, the audio index is an index created for the audio accompanying the input video from the start of recording of the input video in the storage device 10 to the end of recording, and the recorded video can be played back or searched from a specific scene. Use when doing.

Here, the annotation index has a problem that a plurality of snapshots including the annotation information must be taken and that only the annotation index creation time by taking the snapshot can be used as an index. Furthermore, since oral content such as explanations and explanations may be played from the middle of the video, the index playback using the annotation index is based on the annotation index generation time, so the necessary index playback and search from the middle of the video. There is a problem that the created annotation index cannot be modified.

Therefore, in this embodiment, these problems are solved by associating the annotation index with the voice index using the voice index. In addition, the index can be modified, and the visibility of the index and the operability are improved. The details of this embodiment will be described below.

FIG. 3 is a diagram for explaining the determination of the annotation index in this embodiment. FIG. 3 shows the annotation generation times A, B, and C including the annotation information A, B, and C, respectively, when the time when the snapshot including the annotation information is generated is called the annotation generation time.

In FIG. 3, in order to prevent a large number of annotation indexes from being generated, those whose annotation generation time intervals are less than a predetermined threshold are set to be the same annotation index. As an example, when the predetermined threshold is 5 seconds, the interval between the annotation generation times A and B is less than 5 seconds, and when the interval between the annotation generation times B and C is 5 seconds or more, the annotation information A and B are combined. The same annotation index 200 is used, and the annotation information C is a single annotation index 201.

One annotation index 200 that summarizes the annotation information A and B has a time width, and the annotation index 200 has a start point and an end point. That is, in FIG. 3, if annotation is generated without annotation generation above a predetermined threshold value, it becomes the start point (1) of the first annotation index 200, the annotation generation state continues within the predetermined threshold value, and annotation generation above the predetermined threshold value is performed again. If no annotation is generated, that is the starting point (3) of the next annotation index 201. The last annotation generation time in which the annotation generation state within the predetermined threshold value continues becomes the end point (2) of the first annotation index 200. Index reproduction by the annotation index 200, which is a collection of a plurality of annotation information, is based on the time of the start point.

FIG. 4 is a diagram illustrating a display format of the annotation index under the conditions of FIG. As shown in FIG. 4, the annotation index is displayed on the snapshot with the annotation information surrounded by a dotted line frame. Here, the annotation information A and B are collectively displayed as the annotation index 200 surrounded by one dotted frame, and the annotation information C is displayed as the annotation index 201 surrounded by a single dotted frame.

FIG. 5 is a diagram showing a display layout of the annotation index and the voice index in this embodiment. When the snapshot including the annotation information is index-searched, the dotted line frame surrounding the annotation information on the snapshot shown in FIG. 4 is displayed in the video display area 301 of the display screen 300 as shown in FIG.

The voice index is displayed in the voice index display area 302. Further, the text obtained by converting the voice of the selected voice index into characters is displayed in the text display area 303.

FIG. 6 is a diagram showing other display layouts of the annotation index and the voice index in this embodiment. In FIG. 6, in consideration of the MIX screen display of the image display and the camera image display, the image display area 301 and the camera image display areas 304 and 305 are provided, and the image display area 301 and the camera image are similar to those in FIG. A voice index display area 302 and a text display area 303 may be provided above and below the display areas 304 and 305. The voice index display area 302 and the text display area 303 may be customized. For example, if the text is fixed at the bottom, it may be difficult to see, so the voice index display area 302 may be at the bottom and the text display area 303 may be at the top. When there is no camera image, the camera image display area 304 or 305 may be used as the text display area 303.

FIG. 7 is a diagram illustrating a display format of the voice index displayed in the voice index display area 302 of FIGS. 5 and 6 in this embodiment. As shown in FIG. 7, the voice index is displayed for each microphone channel 405 as a display for each voice block 400. For example, channel CH1 is a teacher, channel CH2 is a student, a listener, etc., and an audio index for each channel is automatically created and displayed according to the number of connected microphones.

The voice block 400 has the same voice index when the speech time interval is less than a predetermined threshold value. For example, the predetermined threshold value is 5 seconds, the speech interval of less than 5 seconds is the same voice index as the content of a series of speeches, and the speech index of 5 seconds or more is displayed as another voice block.

Also, inside the voice block 400, for example, when the speech time interval is less than 2 seconds, it is highlighted as a period point (displayed with a dotted line, displayed in a different color for each period point in the same block, or mottled. It is displayed by changing the color alternately), and it is managed and used as a divisible position by the editing function. It can be used when editing so-called sentences.

Since the audio index is an index for the audio accompanying the input video from the start of recording to the end of recording of the input video in the storage device 10, the default display of the audio index is the entire period from the start of recording to the end of recording. It is a display, and the display interval 401 such as 5 minutes and 10 minutes of the recording display time is fixed. The display interval of the recording display time can be changed, and the length of the audio block changes by changing the display interval.

Further, when adjusting the display interval of the recording display time, if it does not fit in the audio index display area, the scroll display 402 is performed.

Select a voice index to display the text, and select it again to play the index using that voice index. In the case of index playback using an audio index, an earlier time in the minus direction is added as a margin to the time at the start point of the audio index generation time, and the input video recorded from the time corresponding to that time is reproduced. do.

Reference numeral 403 is a display of the playback position, which automatically scrolls during playback. It is also possible to scroll manually during automatic scrolling. If another audio index is not selected within a certain period of time, the position of the audio index being played is restored and automatic scrolling is restarted.

The audio block 400 may be color-coded for each channel, or the display height or color may be changed in response to the volume.

Further, a volume level threshold value may be set for each channel so that sounds below the threshold value are not converted into text.

Further, in the voice block 400, important parts such as frequently-used words and keyword parts may be highlighted by changing the color, blinking, adding a frame, or the like. In addition, the above highlighting may be performed in the case of a link with an annotation index, which will be described later, or when the characters of the teaching material and the frequently-used words match.

Further, the voice index displayed in the voice index display area 302 may be displayed in a list format instead of the above-mentioned voice block format. At that time, the main components of the data structure of the voice index, which will be described later, are listed.

FIG. 8 is a flowchart of the voice index creation process in this embodiment. In FIG. 8, first, in step S81, recording of the input video in the storage device 10 is started. Next, in step S82, it is detected by voice level detection or the like whether or not there is a statement, and if there is a statement, the process proceeds to step S83, and if there is no statement, the process proceeds to step S87. In step S83, text conversion processing from voice to character is performed by voice recognition. Next, in step S84, a period detection process is performed to detect, for example, whether the speech time interval is less than 2 seconds. Next, in step S85, the continuity of the speech is determined. For example, if the speech time interval is less than 5 seconds, the process returns to step S83, and if it is 5 seconds or more, it is determined that the speech is finished. Time information and text data are stored as one voice index. Then, in step S87, it is determined in the storage device 10 whether or not the recording stop of the input video is selected, and if it is not stopped, the process returns to step S82 and the next audio index creation process is performed. If recording stop is selected, the process proceeds to step S88 to end recording.

FIG. 9 is an explanatory diagram of the link between the voice index and the annotation index in this embodiment. In FIG. 9, the horizontal axis is time, and the annotation indexes 1 and 2 show a case where a plurality of annotation informations have a time width. When the generation time of the voice index and the annotation index overlaps, it is defined that they are linked, and they are associated with each other. In FIG. 9, the annotation index 1 and the voice indexes 1 to 5 are linked, and the annotation index 2 and the voice index 5 are linked. In addition, annotation indexes 1 and 2 are also indirectly linked via the voice index 5. A plurality of annotation indexes linked to a common voice index in this way are collectively referred to as a unit index.

FIG. 10 is a diagram illustrating a display format of the annotation index under the conditions of FIG. In FIG. 10, 200 and 201 in which the annotation information on the snapshot displayed in the video display area 301 is surrounded by a dotted line frame are annotation indexes, and 250 surrounded by a dotted line frame offset from the annotation index is a unit index. Is displayed as.

The color of the dotted line frame of the annotation indexes 200 and 201 and the dotted line frame of the unit index 250 may be changed.

Also, hide the dotted frame of the annotation index when the unit index is defined. When the unit index is selected, the dotted frame of the annotation index inside is displayed. The specifications of the displayed annotation index are as described above.

FIG. 11 is a diagram illustrating a display format of a link between the voice index and the annotation index in this embodiment.

In FIG. 11, when the voice index and the annotation index have the relationship as shown in the figure, the annotation indexes 1 to 3 constitute the unit index 250. The voice index linked to the annotation index displays an icon 410 such as a microphone mark indicating the voice index near the annotation index.

The display position of the linked voice index icon 410 is set to the left side of the left side of the dotted frame of the uniform annotation index (or unit index) for the voice index 1 which is the pattern ahead in time. Further, the voice index 2 which is a later pattern in time is set to the right side of the left side of the dotted line frame of the annotation index (or unit index).

In addition, the generation time of the annotation index (or unit index) is regarded as the width of the dotted line frame, and the link line 420 connecting the icon 410 and the annotation index (or unit index) according to the difference between the voice index and the generation start time of the annotation index. It is the connection position with the annotation index (or unit index) of. At this time, the margin is not included in the generation start time of the voice index.

In FIG. 11, what is displayed in the video display area 301 of the display screen 300 is an icon 410 indicating an annotation index or a unit index and an audio index. When the icon 410 or the unit index indicating the audio index displayed in the video display area 301 is selected, the text corresponding to the audio index is displayed. Also, the dotted frame of the unit index in the unit index is displayed.

Further, in the case of reproduction by the index of the input video in the recording device, when the unit index is selected, the video is reproduced from the time corresponding to the index corresponding to the earlier index of the generation time of the annotation index and the audio index. In the case of an audio index, the audio index is reproduced including a margin ((1) in FIG. 11). Further, when the icon 410 indicating the audio index is selected, the video is reproduced from the time corresponding to the audio index ((2) in FIG. 11). When the annotation index is selected, the video is played back from the time corresponding to the annotation index ((3) in FIG. 11).

FIG. 12 is a diagram illustrating a link display format when the difference between the generation time of the voice index and the annotation index in this embodiment is small.

In FIG. 12, similarly to FIG. 9, the annotation indexes 1 and 2 show a case where a plurality of annotation informations have a time width, and a case where the voice index and the annotation index generation time overlap, respectively. Is defined as a link, but if the difference between their generation times is small, for example, less than 2 seconds, it is defined as a sublink. A link with an annotation index separated by 5 seconds or more is called a main link.

In FIG. 12, the annotation index 1 and the voice index 7 are mainly linked, and the annotation index 2 and the voice index 10 are mainly linked. Further, the annotation index 1 and the voice indexes 6, 8, 9 and 10 are sub-linked, and the annotation index 2 and the voice indexes 7 and 9 are sub-linked.

FIG. 13 is a diagram illustrating a display format of a link between the voice index and the annotation index under the conditions of FIG. FIG. 13 shows an icon 410 indicating an annotation index or a unit index and an audio index displayed in the video display area 301.

When the icon 410 has a voice index 6, the voice index 6 is sublinked only to the annotation index 1, so that the icon 410 is connected to the dotted frame surrounding the annotation index 1 by the link line 420.

When the icon 410 has a voice index 7, the voice index 7 has a main link with the annotation index 1 and a sublink with the annotation index 2. Therefore, the annotation indexes 1 and 2 have a unit index 250, and the icon 410 has a unit index 250. It is connected to the surrounding dotted frame by a link line 420.

When the icon 410 has a voice index 8, the voice index 8 is sublinked only to the annotation index 1, so that the icon 410 is connected to the dotted frame surrounding the annotation index 1 by the link line 420.

When the icon 410 has a voice index 9, the voice index 9 is sublinked by the annotation indexes 1 and 2, so that the annotation indexes 1 and 2 become the unit index 250, and the icon 410 is linked to the dotted frame surrounding the unit index 250. It is connected by line 420. If there is only a sublink, the inclination of the link line 420 is determined as the link for the one with the earlier generation time.

When the icon 410 has a voice index 10, the voice index 10 has a sublink with the annotation index 1 and a main link with the annotation index 2. Therefore, the annotation indexes 1 and 2 have a unit index 250, and the icon 410 has a unit index 250. It is connected to the surrounding dotted frame by a link line 420. When the main link and the sub link are mixed, the main link is used as the reference. When there are a plurality of main links, the slope of the link line 420 is determined as the link for the one with the earlier generation time.

The degree of association may be distinguished between the sublink and the main link, and the color and shape of the link line 420 may be changed.

FIG. 14 is a conceptual diagram illustrating the recording of the input video and the storage configuration of the annotation index and the audio index in this embodiment. In FIG. 14, the upper part is a storage configuration of a main layer which is a conventional input video and an annotation layer having annotation information. That is, conventionally, since the video and the annotation information are combined into one video at the time of saving, the annotation information cannot be changed on the recorded video. Also, if you forget to take a snapshot, you cannot define the annotation index.

On the other hand, the lower part is the storage configuration of the main layer and the annotation layer in this embodiment. That is, in this embodiment, the voice is configured to store the voice index data and the text data separately from the voice.
Indexing can be done with voice index data without a snapshot at the time of recording. Further, since the correction by the editing function of the voice index data and the text data can be reflected in each data and saved, the index can be corrected.

FIG. 15 is a data structure diagram of the voice index in this embodiment. In FIG. 15, 450 is a file name and indicates a date and a recording start time. Reference numeral 451 is an index, which indicates a number indicating the order in which the voice index is generated and a branch number indicating that the voice index is divided by a period point. Further, 452 is an attribute, and indicates a microphone channel, identification of a person, a type of index group, and the like. Further, 453 is a start time, and indicates a voice start time or a period point start time. Further, 454 is an end time, and indicates a voice end time or a period point end time. The start time 453 and the end time 454 are elapsed times from the recording start time, and jump from the end time to the next index start time during continuous playback of the index group. Reference numeral 455 is a coordinate, which indicates the connection coordinate between the display position (center coordinate) of the icon 410 indicating the voice index and the annotation index. Further, 456 is an annotation index name to be linked.

As described above, according to the present embodiment, by associating the annotation index with the voice index using the voice index, recording is performed using the voice index even if there is no snapshot for the annotation index. It is possible to search and play back a specific scene in the recorded video. In addition, the index can be modified, and the visibility of the index can be improved and the operability can be improved.

In this embodiment, reproduction of an input video in a recording device using an audio index will be described.

FIG. 16 is a conceptual diagram illustrating reproduction of an input video in a recording device using an audio index in this embodiment.

In FIG. 16, similarly to FIG. 7, the voice index is displayed for each microphone channel as a display for each voice block 400.

Here, words that frequently appear are extracted from the textual content of the voice index, and these important words are used as important words to construct an index group of these important words. Then, as shown by 430, the input video is reproduced in the recording device using the index in units of period points including the important words. If there is no period point, play from the beginning of the audio block.

In the above, the words that frequently appear are regarded as important words, but the important words may be determined by using a dedicated system for the important words.

In addition, a summary index group is constructed by comprehensively judging important words, volume, context, etc. Then, as shown by 440, playback is performed from the first index of the audio block, or when the distance between the beginning of the audio block and the summary portion is long, playback is performed from the index of the period point.

Using such an index group, the index group may be individually selected for index playback, or as continuous playback of the index group, when playback in period point units or audio block units is completed, the index is moved to the next index. You may make an automatic jump and play an index.

Although important words and summaries are automatically determined and created by the recording device, an index group may be constructed with keywords determined by the user.

As described above, according to the present embodiment, it is possible to improve the operability of index reproduction using the voice index.

In this embodiment, an index in which a voice index and an annotation index are integrated will be described.

FIG. 17 is a conceptual diagram illustrating an integrated index in this embodiment. In FIG. 17, the annotation index, the audio index, and the index information from the presentation tool separate from the recording device are displayed with the horizontal axis as the time axis. The audio index is displayed for each microphone channel as a display for each audio block 400.

In this embodiment, an integrated index as shown by 450, which integrates these various index information, is constructed. Here, among a plurality of index information, indexes that differ at approximately the same time or a few seconds are set as approximate indexes 451 and adjusted by automatically deleting or turning off the approximate indexes, and an integrated index group is constructed using only useful indexes. ..

In addition, by constructing an index group such as a digest composed of all indexes including index groups such as important words, keywords, and summaries described in the second embodiment, a pseudo automatic video editing function becomes possible. In addition, in order to construct this index group, it is necessary to understand the contents even what is drawn by the annotation index, so AI may be used.

In addition, using a stereo sensing camera that can calculate distance information from parallax and capture the imaging range three-dimensionally, it outputs motion information of human behavior from that camera, and uses a recording device or a dedicated AI-equipped machine to output it. The information may be analyzed, and the analysis result of capturing the motion characteristics such as the gesture of the subject (for example, a teacher) may be reflected in the index generation.

As described above, according to this embodiment, it is possible to improve the function of index reproduction by using the integrated index.

Although it has been described in FIG. 5 that the annotation index indicated by the dotted line frame surrounding the annotation information is displayed on the snapshot in the video display area 301 of the display screen 300, the one to be displayed on the snapshot is described. It may be selectable. For example, it may have a display selection function such as full display of both annotation index and voice index, full display of both linked items, full display of only annotation index, and full display of only voice index.

Further, in addition to the above, the voice index display area 302 may be provided with a display selection function for displaying an integrated index, important words, keywords, summaries, digest index groups, and the like.

Although the examples have been described above, the present invention is not limited to the above-mentioned examples, and various modifications are included. For example, the above-described embodiment has been described in detail in order to explain the present invention in an easy-to-understand manner, and is not necessarily limited to the one including all the configurations described. Further, it is possible to replace a part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment. Further, it is possible to add / delete / replace a part of the configuration of each embodiment with another configuration.

10: Recording device, 20: Projector, 30: Camera, 40: Microphone, 50: Data input PC, 60: Server, 70: Viewing PC, 101: Video input unit, 102: Channel allocation unit, 103: Information input Unit, 104: Annotation processing unit, 105: Audio input unit, 106: Audio processing unit, 107: Video processing unit, 108: Recording unit, 109: Video output unit, 110: Video synthesis unit, 111: Index generation unit, 200 , 201: Annotation index, 250: Unit index, 300: Display screen, 301: Video display area, 302: Audio index display area, 303: Text display area, 304, 305: Camera image display area, 400: Audio block, 410 : Icon indicating voice index, 420: Link line

Claims (13)

A recording device that records an input video and an index in the input video.
The index is an annotation index using the video signal of the input video and an audio index using the audio signal accompanying the input video.
If the generation time of the voice index and the annotation index overlap, they are associated with each other.
A recording device characterized in that by selecting either the associated voice index or the annotation index, a time corresponding to the time of the start point of the generation time of the selected index is recorded. The recording device according to claim 1.
The annotation index is a snapshot obtained by cutting out the moment of the input video, and is obtained by storing a plurality of snapshots including annotation information which is handwritten information on the display image together with time information.
The audio index is a recording device created for audio accompanying the input video from the start of recording of the input video to the end of recording. The recording device according to claim 2.
As the voice index, the same voice index is used when the speech time interval in the voice is less than the predetermined threshold value, and another voice index is used when the speech time interval is equal to or more than the predetermined threshold value.
A recording device characterized in that, inside the voice index, when the speech time interval is less than a second threshold value smaller than the predetermined threshold value, it is managed as a divisible position. The recording device according to claim 1.
When the recorded input video is reproduced by selecting the audio index, a time in the earlier direction in the negative direction is added as a margin to the time of the start point of the generation time of the audio index, and the time is set to that time. A recording device characterized in that the recorded input video is reproduced from a corresponding time. The recording device according to claim 1.
A recording device characterized in that the voice of the voice index is converted into text from voice to characters. The recording device according to claim 2.
When an index search is performed on a snapshot containing the annotation information, a dotted line frame surrounding the annotation information on the snapshot is displayed in the image display area of the display screen of the external display device, and is above or below the image display area in the display screen. A recording device characterized in that a video signal is output so that the audio index is displayed in a side area. The recording device according to claim 3.
The audio index is displayed as an audio block for each audio input channel in the area above or below the image display area of the display screen of the external display device, and the video signal is highlighted so that the divisible position is highlighted. A recording device characterized by outputting. The recording device according to claim 1.
A plurality of annotation indexes linked to the voice index are collectively referred to as a unit index.
The annotation index is a snapshot obtained by cutting out the moment of the input video, and is obtained by storing a plurality of snapshots including annotation information which is handwritten information on the display image together with time information.
When an index search is performed on a snapshot containing the annotation information, a dotted frame surrounding the annotation information on the snapshot is displayed in the video display area of the display screen of the external display device, and further, when the annotation index is the unit index. Is a recording device characterized in that a video signal is output so that a second dotted frame offset from the dotted frame surrounding the annotation information is displayed in the video display area of the display screen of the external display device. The recording device according to claim 8.
The audio index linked to the annotation index is characterized by outputting a video signal so as to display an icon indicating the audio index near the annotation index in the video display area of the display screen of the external display device. Recording device. The recording device according to claim 9.
The display position of the icon of the linked voice index is the left side of the left side of the dotted line frame of the annotation index or the second dotted line frame of the unit index for the first voice index ahead in time, and the time. A recording device characterized in that a video signal is output so that the second audio index later is on the right side of the left side of the dotted line frame of the annotation index or the second dotted line frame of the unit index. The recording device according to claim 10.
The generation time of the annotation index or the unit index is regarded as the width of the dotted line frame, and a link line connecting the icon and the annotation index or the unit index according to the difference between the voice index and the generation start time of the annotation index. A recording device characterized in that a video signal is output so as to be a connection position with the annotation index or the unit index of the above. The recording device according to claim 10.
When the unit index is selected, the input video is played back from the time corresponding to the earlier index of the annotation index and the audio index generation time.
When the icon indicating the audio index is selected, the input video is played from the time corresponding to the audio index.
A recording device characterized in that when the annotation index is selected, the input video is reproduced from the time corresponding to the annotation index. A recording system consisting of a video acquisition device, an audio acquisition device, a recording device that captures and records video and audio information from the video acquisition device and audio acquisition device, and a viewing device that views playback information from the recording device via a server. And
The recording device is
A video input unit that inputs video from the video acquisition device and captures the input video,
An information input unit for inputting annotation information for the input video, and
Annotation processing unit that draws the captured annotation information, and
A voice input unit that captures voice information from the voice acquisition device,
A voice processing unit that processes the captured voice information as digital data,
A video synthesis unit that synthesizes drawing information of the input video and annotation information processed by the annotation processing unit, a video processing unit having an index generation unit that generates an index in the input video, and a video processing unit.
A recording unit that records the video synthesized by the video compositing unit and the index generated by the index generation unit,
A video output unit that outputs the video recorded in the recording unit to an external display device, and
Has a control unit
The index is an annotation index using the video signal of the input video and an audio index using the audio signal accompanying the input video.
The control unit associates the voice index and the annotation index with each other assuming that the generation times of the voice index and the annotation index overlap with each other, and any one of the linked voice index and the annotation index. When is selected, the recording system is characterized in that the input video recorded in the recording unit is reproduced from the time corresponding to the time of the start point of the generation time of the selected index.

Images