JP2007012013A - Video data management apparatus, method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To search by adding metadata analyzed for a feature amount to metadata (program performers, genres, etc.) given to a video.
The present invention monitors video data, specifies video data to be indexed, outputs related information as initial metadata, performs indexing on the specified video data, and performs indexing. Based on the metadata, a representative image is selected, a list of video data is generated, additional metadata is generated based on the initial metadata and indexing metadata, and each generated data is stored in the metadata DB. The metadata DB is searched based on the stored search key that is stored and input, information on the Internet is searched, and the search result is displayed in one window that is newly generated on the display means.
[Selection] Figure 1

Description

  The present invention relates to a video data management apparatus, method, and program, and more particularly to an index related to video data in any apparatus capable of recording video data, such as a personal computer, a hard disk recorder, or a set top box. The present invention relates to a video data management apparatus, method, and program for automatically generating information and managing, searching, and browsing video data.

  There is a demand for searching for desired video data in a short time from video data owned by an individual. For example, in a state where a lot of video data is stored on the disk by means of recording, copying, or downloading, the video scene is not displayed by a method of sequentially reproducing and checking the content and searching for the content, but by a list of still images. This is a method of searching by grasping or a request to search for a video file to be obtained by text search.

  In response to this requirement, the following functions are provided by the prior art.

(1) A function for displaying a list of video data by a function of a file management application (Internet Explorer (registered trademark), etc.);
(2) Function to search video data for file names;

  However, the functions provided by the conventional technique have the following problems.

  -In the function of the file management application currently provided generally, only the head scene of the video is displayed when the video data list is displayed. For this reason, it is difficult to grasp the contents included in the video data, and as a result, it is difficult to reach the desired video data.

  -The video data search function currently available to the public only targets file names. The name of the video data does not necessarily represent the content of the video, and if the user remembers part of the content instead of the file name of the video data, the appropriate search key cannot be entered As a result, it is difficult to reach the desired video data.

  -In the video data search function that is generally provided at present, the search result screen only shows a list of video data. For this reason, when the user once searches and then enters another search key and wants to search again, the user needs to devise a new search key by himself. For this reason, it may be difficult to reach the desired video data as a result.

  As described above, in the conventional technique, it is difficult to reach desired video data.

  The present invention has been made in view of the above points, and an object of the present invention is to provide a video data management apparatus, method, and program that make it easier to reach desired video data than before.

  FIG. 1 is a principle configuration diagram of the present invention.

The present invention (Claim 1) is a video data management device for generating index information related to video data and managing, searching, and browsing the video data,
The video data in the storage means 20 and the video data input / output to / from the recording medium are monitored, the video data to be indexed is specified, and the related information given in advance when the video data is acquired by the user Video data monitoring means 31 for outputting as initial metadata,
Indexing means 32 for performing indexing on the video data designated by the video data monitoring means 31 and outputting indexing metadata in which feature information and time information of the video data are associated;
Based on the indexing metadata obtained by the indexing means 32, a representative image is selected, a list of video data is generated, and the video data list generating means 33 stored in the storage means 20,
An additional metadata generating unit 34 for generating additional metadata which is related information to be added to video data based on the initial metadata acquired from the video data monitoring unit 31 and the indexing metadata acquired from the indexing unit 32; ,
Metadata DB generation means 35 for storing initial metadata, indexing metadata, and additional metadata in the metadata DB 50;
Based on the input search key, the metadata DB 50 is searched, information on the Internet is searched using an Internet search engine, and the search results are collectively displayed in a new window generated on the display means. Search means 36;
And a video data list update unit 37 that updates the video data list based on the progress of the indexing process of the indexing unit 32.

Further, according to the present invention (Claim 2), in the additional metadata generation means 34,
A means for performing an Internet search using the initial metadata as a search key, removing the tag symbol from the obtained result page, extracting the text, and adding it as additional metadata;
Means for acquiring a character string from the indexing metadata by telop recognition and adding it as additional metadata;
When multiple keywords in the electronic program guide are available as initial metadata, the number of keywords used as keys is reduced step by step, the tag symbol is removed from the result of the Internet search, and the text is extracted. Adding as additional metadata.

Further, according to the present invention (Claim 3), the search means 36
There are two types of results of searching the metadata DB 50: video data in which words that match the initial metadata field or indexing metadata field exist, and video data in which words that match the additional metadata field exist. Means for displaying it in the window of the display device together with the representative image obtained by the video data list generation means 33;
Means for collectively displaying results obtained using an Internet search engine in the same window.

The present invention (Claim 4) is the video data list generation means 33,
It includes means for generating a list arranged for each type of event included in the indexing metadata including cut points and camera work, and displaying the list in the same window as the initial metadata.

Further, the present invention (Claim 5) is provided in the video data list generation unit 33.
The output image obtained by the indexing means 32 includes means for selecting, from the plurality of images indicating the cut points, ones whose time points in the video indicated by the cut images are equally close based on the indexing metadata.

The present invention (Claim 6) is a video data management apparatus for generating index information related to video data and managing the video data,
The video data in the storage means and the video data input / output to / from the recording medium are monitored, the video data to be indexed is specified, and related information given in advance at the time when the user acquires the video data is displayed. Video data monitoring means for outputting as initial metadata;
A voice enhancement section extracting means for extracting a section in which the audio of the video data is emphasized as a voice enhancement section;
Indexing means for performing indexing on the video data designated by the video data monitoring means, and outputting indexing metadata in which information of the characteristic portion and time information of the video data are associated;
An indexing progress management means for monitoring the progress of the indexing process in the indexing means and outputting the progress rate;
A video data list is created based on the indexing metadata output from the indexing means, the progress rate obtained from the indexing progress management means, and the voice enhancement section obtained from the voice enhancement section extraction means, and the video data in the storage means A video data list updating unit for updating the list and displaying the video data list on the display unit;
Additional metadata generation means for generating additional metadata, which is related information to be added to video data, based on the initial metadata acquired from the video data monitoring means and the indexing metadata acquired from the indexing means; ,
Metadata DB generation means for storing the initial metadata, the indexing metadata, and the additional metadata in a metadata database (DB);
Based on the input search key, the metadata DB is searched, information on the Internet is searched using an Internet search engine, and search results are newly displayed in a single window that is newly generated on the display means. Search means.

The present invention (Claim 7) is the data management apparatus according to Claim 6,
The video data list update means
The progress rate acquired from the indexing progress management means is compared with the progress rate acquired last time and stored in the storage means. If the progress rate is higher than the previous time, the video data list is updated and output to the display means. Including means.

  FIG. 2 is a diagram for explaining the principle of the present invention.

The present invention (Claim 8) is a video data management method for generating index information related to video data and managing, searching and browsing the video data,
In a video data management apparatus having video data monitoring means, indexing means, video data list generation means, additional metadata generation means, metadata DB generation means, search means,
The video data monitoring means monitors the video data in the storage means and the video data input / output to / from the recording medium, designates the video data to be indexed, and gives it in advance when the user acquires the video data A video data monitoring step (step 1) for outputting the related information as initial metadata;
An indexing step (step 1) in which the indexing means performs indexing on the video data designated in the video data monitoring step (step 1), and outputs indexing metadata in which the feature information and the time information of the video data are associated with each other. 2) and
A video data list generating unit selects a representative image based on the indexing metadata obtained in the indexing step (step 2), generates a list of video data, and stores it in the storage unit (step 3) and
An additional metadata generation step in which the additional metadata generation means generates additional metadata based on the initial metadata acquired in the video data monitoring step (step 1) and the indexing metadata acquired in the indexing step (step 2). (Step 4),
A metadata DB generating means for storing the initial metadata, the indexing metadata, and the additional metadata in the metadata DB (step 5);
The search means searches the metadata DB based on the input search key, searches information on the Internet using an Internet search engine, and newly generates a search result on the display means. A search step (step 6) for displaying in an integrated manner,
The video data list update means performs a video data list update step of selecting a representative image and updating the video data list based on the progress of the indexing process in the indexing step (step 2).

The present invention (Claim 9) is a video data management method for generating index information related to video data and managing the video data,
In a video data management apparatus having video data monitoring means, indexing means, voice enhancement section extraction processing means, progress management means, video data list update means, additional metadata generation means, metadata DB generation means, search means,
The video data monitoring means monitors the video data in the storage means and the video data input / output to / from the recording medium, designates the video data to be indexed, and gives it in advance when the user acquires the video data Video data monitoring step for outputting the related information as initial metadata,
A voice enhancement section extraction step in which the voice enhancement section extraction processing means extracts a section in which the audio of the video data is emphasized as a voice enhancement section;
An indexing step for indexing the video data specified in the video data monitoring step, and outputting indexing metadata in which the feature information and the time information of the video data are associated;
An indexing progress management step for monitoring the progress of the indexing process in the indexing step and outputting the progress rate;
The video data list updating means creates a video data list based on the indexing metadata acquired in the indexing step, the progress rate acquired in the indexing progress management step, and the voice enhancement section obtained in the voice enhancement section extraction step. Updating the video data list of the storage means and displaying the video data list on the display means; and
Additional metadata generating means generates additional metadata that is related information to be added to video data based on the initial metadata acquired in the video data monitoring step and the indexing metadata acquired in the indexing step. A data generation step;
A metadata DB generating means for storing initial metadata, indexing metadata, and additional metadata in the metadata DB;
The search means searches the metadata DB based on the input search key, searches information on the Internet using an Internet search engine, and newly generates a search result on the display means. And a search step for collecting and displaying.

  The present invention (Claim 10) is a video data management program that causes a computer to execute each step of the video data management method according to Claim 8 or 9.

  The present invention provides a list for discriminating video contents, automatically adds words related to the video contents, and provides a character string that can be a new search word when a search result is acquired. Therefore, desired video data can be reached in a shorter time than the prior art.

  Furthermore, according to the present invention, even if a long-time video is input, it is immediately reflected in the list, so that it is easier to reach the desired video data as compared with the method provided by the prior art.

  In particular, assuming that video data is 1 hour, a conventional video data list that takes about 40 minutes can be acquired in about 5 minutes. For this reason, a list of videos can be acquired in about 1/8 of the time compared to the prior art.

  Before describing this embodiment, terms used in this specification will be defined.

  Any device capable of storing video data (computer disk, memory, etc., or hard disk recorder, set top box, etc.) is referred to as a “storage medium (storage unit)”.

  Content including moving images such as programs stored in a storage medium via a broadcast receiving device or a network is referred to as “video data”.

  The process of automatically extracting cut points, camera work sections, audio sections, music sections, telop display sections, etc. from video data is called “indexing”. Information on feature parts (still images, feature parameters, etc.) and video obtained by indexing Data associated with time information of data is referred to as “indexing metadata”.

  Related information given in advance when the user acquires the video data, such as the file name of the video data and the electronic program guide, is referred to as “initial metadata”.

  The related information added to the video data using the method described in the present invention is referred to as “additional metadata”.

  The initial metadata, indexing metadata, and additional metadata are collectively referred to as “metadata”.

  Search functions provided on portal sites such as Google (registered trademark), Google (registered trademark), and Yahoo (registered trademark) are referred to as “Internet search functions”.

  A part that is temporarily occupied on a screen of a display device such as a display by software operating on a computer and is generally divided by a rectangle is referred to as a “window”.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[First Embodiment]
FIG. 3 shows the configuration of the video data management apparatus according to the first embodiment of the present invention.

  The video data management apparatus shown in the figure includes a video data input / output unit 10, a storage unit 20, a processing unit 30, a display unit 40, and a metadata DB 50. In the present embodiment, the storage unit 20 uses a hard disk, and stores video data, metadata, log data, and the like.

  The processing unit 30 includes a video data monitoring unit 31, an indexing unit 32, a video data list generation unit 33, an additional metadata generation unit 34, a metadata database generation unit 35, and a search unit 36.

  The video data monitoring unit 31 monitors video data existing in the storage unit 20 and video data input / output from the video data input / output unit 10, determines a video to be indexed, and generates initial metadata as a video data list. To the unit 33.

  The indexing unit 32 receives the video data designated by the video data monitoring unit 31 and inputs the documents “Yukinobu Taniguchi, Kenichi Minami, Takashi Sato, Hideo Kuwano, Haruhiko Kojima, Yoshinobu Tonomura,“ Scene Cabinet: Video Analysis Technology ”. Video indexing system that integrates "The Journal of the Institute of Electronics, Information and Communication Engineers, Vol. J84-D-II, No.6, pp.1112-1121, 2001" and references "Mori Kaoru, Masaharu Kurakake, Toshiaki Sugimura, Akio Shio, Akira Suzuki, “Telop Character Recognition in Video Using Background / Character Shape Features and Dynamic Correction Discrimination Function”, IEICE Transactions, D-II, Vol. J83-D-II ” Indexing is performed based on the method described in 316378 “Audio processing method, apparatus using the method and program thereof”, and the indexing metadata is output to the video data list generation unit 33.

  The video data list generation unit 33 receives the indexing metadata and video data output from the indexing unit 32, creates a list of video data, and outputs the video data list to the storage unit 20.

  The additional metadata generation unit 34 receives the initial metadata output from the video data monitoring unit 31 and the indexing metadata output from the indexing unit 32, generates additional metadata, and outputs the additional metadata to the storage unit 20.

  The metadata database generation unit 35 receives initial metadata, indexing metadata, additional metadata, and video data as inputs, and outputs them to the metadata database (DB) 50.

  The data search unit 36 searches the metadata DB 50 using any text input by the user as a key, and simultaneously searches information on the Internet using an Internet search engine (not shown), and displays the results of both. The information is displayed in one window of the unit 40 in a consolidated manner.

  Hereinafter, the above operation will be described in detail.

  FIG. 4 is a flowchart of an overall outline operation in the first embodiment of the present invention, FIG. 5 is a flowchart of a video monitoring step in the first embodiment of the present invention, and FIG. 6 is a flowchart of the present invention. FIG. 7 is a general flowchart of the search step in the first embodiment of the present invention, and FIG. 7 is a general flowchart of the list and DB generation step in the first embodiment.

  Prior to the following processing, it is assumed that at least video data is stored in the storage unit 20 from the video data input / output unit 10.

Step 11) Video data monitoring step:
Step 111) File search step:
The video data monitoring unit 31 of the processing unit 30 monitors a video file in the storage unit 20 (disk) and acquires a file path.

Step 112) Indexing log file search step:
The indexing unit 32 searches for the indexing log file with the file path obtained in step 111.

Step 113) Indexing target video designation step:
Of the video data obtained in step 111, the video file whose log file in the storage unit 20 was not found in step 112 is designated and output as an indexing target moving image.

Step 114) Initial metadata output step:
The video data monitoring unit 31 outputs the initial metadata of the video data output in step 113 to the video data list generation unit 33 and the additional metadata generation unit 34.

Step 12) List and DB generation step:
Step 121) Initial metadata input step:
The video data list generation unit 33 receives the initial metadata output in step 114.

Step 122) Indexing step:
The indexing unit 32 inputs the video data output in step 113, and documents “Taniguchi Yukinobu, Minami Kenichi, Sato Takashi, Kuwano Hideo, Kojima Haruhiko, Tonomura Yoshinobu,“ Scene Cabinet: Integrated Video Analysis Technology ”. A feature extraction process is performed by the method of the video indexing system “Journal of the Institute of Electronics, Information and Communication Engineers, Vol. J84-D-II, No.6, pp.1112-1121, 2001” (hereinafter referred to as Reference 1). The data is output to the video list generation unit 32 and the meta database generation unit 35, and the indexing log file is output to the storage unit 20.

Step 123) List generation step:
The video data list generation unit 33 generates a list of video data based on the indexing metadata output at step 122 and the initial metadata output at step 114.

Step 124) Additional metadata generation step:
The additional metadata generation unit 34 performs an Internet search using the initial metadata input in step 121, and generates additional metadata from the obtained result.

Step 125) Metadata DB generation step:
The metadata database generation unit 35 uses the initial metadata input in step 121, the indexing metadata obtained in step 122, and the additional metadata generated in step 124, and has metadata for each video data field. DB50 is generated.

Step 13) Search step:
Step 131) Text acquisition step:
The search unit 36 acquires the text input in the text input field to be displayed on the program.

Step 132) Metadata DB search step:
The search unit 36 searches the metadata DB 50 generated in step 125 using the text character string obtained in step 131 as a key. The search results are obtained in two types: a path of video data that matches in the initial metadata or indexing metadata, and a path of video data that matches in the additional metadata.

Step 133) Internet search step:
The search unit 36 performs a search using an Internet search engine using the text character string obtained in step 131 as a key, and acquires a search result.

Step 134) Search result page generation step:
The search unit 36 summarizes the results obtained in step 132 and step 133, and in particular, selects a representative image with reference to indexing metadata from the video data path obtained in step 132, and generates a result page. To do.

  Next, the indexing step (step 122) in the indexing unit 32 will be described in detail.

  FIG. 8 is a detailed flowchart of the indexing step in the first embodiment of the present invention.

Step 1221) Video data input step:
The indexing unit 32 inputs the video data output in step 113.

Step 1222) Feature extraction step:
The feature amount is extracted from the video file input in step 1221 by the method of the above-mentioned document 1, and the success or failure of the event is determined from the extracted result and recorded.

Step 1223) Indexing metadata output step:
The event recorded in step 1222 is output together with file information of video data (including information generally described in properties such as path information and file size) and playback time information of the video data.

Step 1224) Indexing log output step:
The indexing execution result is output to a log file in the storage unit 20. The log file includes path information of video data, error information during indexing, date and time, and the like.

  FIG. 9 is a detailed flowchart of the list creation step in the first embodiment of the present invention.

Step 1231): Indexing metadata input step:
The video data list generation unit 33 receives the indexing metadata output at step 122 and the initial metadata output at step 114.

Step 1232) Image selection step:
Based on the information of the indexing metadata, a cut point is selected from the cut point information so as to be equally close to the video length, and the associated still image is output by referring to the indexing metadata. .

Step 1233) Playlist generation step:
A list file in which the playlist generated in step 1233 is linked to the still image output in step 1232 by a link method such as HMTL is output to the storage unit 20.

  Next, the operation of the additional metadata generation step (step 124) in the video data monitoring unit 31 will be described in detail.

  FIG. 10 is a detailed flowchart of the additional metadata generation step according to the first embodiment of the present invention.

Step 1241) Initial metadata input step:
The additional metadata generation unit 34 inputs the initial metadata obtained in step 121.

Step 1242) Indexing metadata input step:
In the indexing unit 32, the indexing metadata obtained by performing the process of step 122 is input, and a part other than the text string is filtered.

Step 1243) Internet search step:
Using the indexing metadata obtained by filtering except the initial metadata and the text character string input in step 1241 and step 1242, the Internet search is performed to obtain the URL. When N unique words that are not duplicated are included in the initial metadata and the filtered indexing metadata, as a result of an AND search with n words, URLs that are not duplicated that are included in the top p ranks. A set of URLs U, where U (n, p) is a set.
U = Σ (U (i, p)), where (i = 1, 2,..., N)
To get.

Step 1244) Text extraction step:
A page included in the set of URLs obtained in step 1243 is analyzed, and a text character string is extracted.

Step 1245) Additional metadata output step:
The text character string obtained in step 1244 is added to the additional metadata field of the metadata DB 50.

  Next, the text extraction step of the above step 1244 in the additional metadata generation unit 34 will be described in detail.

  FIG. 11 is a detailed flowchart of the text extraction step according to the first embodiment of the present invention.

Step 12441) Page acquisition step:
A page is acquired for the URL obtained from step 1243.

Step 12442) Page analysis step:
From the page obtained in step 12441, the character string in the page title tag, keyword
Extract the character string in the tag and the character string of the Body text. Remove the link from the string in the Body tag.

Step 12443) Filtering step:
Of the character strings obtained in step 12442, the character string derived from Body is divided by morphological analysis. After the division, other than nouns are filtered, and the process proceeds to step 1245.
Among the character strings obtained in step 12442, the character strings in the page title tag and the keyword tag are excluded from duplication, and the process proceeds to step 1245.

[Second Embodiment]
FIG. 12 shows the configuration of the video data management apparatus according to the second embodiment of the present invention.

  The video data management apparatus shown in the figure has a system monitoring unit 37 in the processing unit 30 in the configuration shown in FIG.

  FIG. 13 is a flowchart of the overall outline operation in the second embodiment of the present invention.

  In the present embodiment, the system monitoring step (step 14) is performed by the system monitoring unit 37 in the preceding stage of step 11 in the flowchart of FIG. 4 in the first embodiment described above. This is the same as in the first embodiment.

  In the system monitoring step (step 14), the system monitoring unit 37 monitors other tasks on the system. Due to the presence of this step, especially when executed on a personal computer, the start and end of the list and DB creation step (step 12) are synchronized with the start and end of other software (eg screen saver) on the system. Can be implemented. When synchronized with the screen saver, the indexing step can be executed at a time when the use of computer resources such as memory and CPU is low.

  In addition, as the information to be output to the display device 40 such as a display during the execution of the screen saver, the image during the execution of the indexing step and initial metadata are output.

  The operation of other steps is the same as that of the first embodiment described above.

[Third Embodiment]
FIG. 14 is a flowchart of the overall outline operation in the third embodiment of the present invention.

  In the present embodiment, the search step (step 13) is performed after the list and DB generation step (step 12) in the first embodiment, whereas the list and DB generation step (step 12). And the search step (step 13) are performed in parallel, and other operations are the same as those in the first embodiment.

[Fourth Embodiment]
FIG. 15 shows the configuration of a video data management apparatus according to the fourth embodiment of the present invention.

  The video data management apparatus shown in the figure has a configuration in which a task setting unit 38 is added to the configuration shown in FIG.

  FIG. 16 is a flowchart of the overall outline operation in the fourth embodiment of the present invention.

  In this embodiment, a task setting step (step 15) is performed prior to the video data monitoring step (step 11) of the first embodiment.

  In step 15, the task setting unit 38 sets an execution task of the additional metadata generation step (step 124) (eg, repeatedly executes the process of step 124 every hour). Thereby, even when information on the Internet is updated, the additional metadata can reflect the updated information.

  Other operations are the same as those in the first embodiment.

[Fifth Embodiment]
In this embodiment, the outline operation is the same as that of the first embodiment, but the operation of the indexing step (step 122) is different.

  FIG. 17 is a flowchart of detailed operation of the indexing step in the fifth embodiment of the present invention.

  In the present embodiment, a content genre determination step (step 12211) is performed in the indexing step (step 122).

  Step 12211) In the video data list generation unit 33, the initial metadata is input, and the content genre and default indexing parameters are output.

  Step 12212) Indexing is performed based on the parameters of step 12211. When the default indexing parameter is described in the initial metadata, the value is output. In addition, the user can change the value to an arbitrary value using a default indexing parameter, GUI, or the like. All other operations are the same as in the first embodiment.

  It should be noted that the present invention can also be implemented by equipment (for example, a hard disk recorder) other than those described in the first to fifth embodiments.

  Moreover, it is also possible to perform on the video distribution server instead of the personal terminal.

  Further, the user can freely add / delete the contents of the metadata DB 50 using an editor or the like.

[Sixth Embodiment]
In the above-described embodiment, the video data list is not displayed unless indexing for certain video data is completed. Since the indexing process has a cost of about 40 minutes for one hour of video data in terms of implementation, there is a time lag from the input of the video data to the reflection in the list.

  In this embodiment, the progress of indexing is managed, and the video data list is updated according to the progress of the indexing, so that the progress of the indexing process is reflected in the list, which is automatically updated according to the progress of the indexing. An example will be described.

  FIG. 18 is a configuration diagram of a video data management apparatus according to the sixth embodiment of the present invention.

  In the figure, the same parts as those in FIG. 3 are denoted by the same reference numerals, and the description thereof is omitted.

  In the configuration shown in FIG. 18, a progress management unit 321 and a voice enhancement section extraction processing unit 322 are added to the configuration of FIG. 3 in the first embodiment, and instead of the video data list generation unit 33 of FIG. A data list update unit 323 is provided. The storage unit 20 in the present embodiment stores video data, indexing metadata, and a video data list.

  The progress management unit 321 outputs the degree of progress of indexing performed by the indexing unit 32.

  The voice enhancement interval extraction processing unit 322 uses, for example, a method disclosed in Japanese Patent Application Laid-Open No. 2003-316378 “Voice processing method and apparatus and program using the method” to enhance the voice in the video. Is output. This method does not depend on the speaker, and does not depend on the simultaneous utterance of multiple speakers, without setting the conditions to be extracted in advance, and in the natural spoken language and conversation without using the manuscript. This is a voice processing technology that can determine whether a voice is in an emphasized state or a calm state stably even in an environment. Thereby, for example, in a sport such as baseball or soccer, it is possible to reflect on the list of video data the location where the sound has risen.

  The video data list update unit 323 receives the speech enhancement section information output from the speech enhancement section extraction processing unit 322, the indexing metadata and video data from the indexing unit 32, and updates and updates the list of video data. The video data list is output to the display unit 40.

  Next, the operation in the above configuration will be described.

  In the present embodiment, processing until a video data list is generated will be described, but other search processing and the like are the same as those in the above-described embodiment.

  FIG. 19 is a flowchart of an outline operation of processing up to video data list generation in the sixth embodiment of the present invention, and FIG. 20 shows a video monitoring step in the sixth embodiment of the present invention. FIG. 21 is a flowchart of the list generation step in the sixth embodiment of the present invention.

Step 31) Video data monitoring step:
Step 311) Video file search step:
The video data monitoring unit 31 monitors a video file in the storage unit 20 (disk) and acquires a file path.

Step 312) Log file search step:
The video data monitoring unit 31 searches the storage unit 20 for the indexing log file of the file obtained in step 311.

Step 313) Indexing target video designation step:
The video data monitoring unit 31 designates and outputs the video data obtained in step 311 for which the log file in the storage unit 20 is not found in step 312 as the indexing target moving image.

Step 314) Initial metadata output step:
The video data monitoring unit 31 outputs initial metadata of the video data output in step 313.

Step 32) List generation step:
Step 321) Initial metadata input step:
The indexing unit 32 receives the initial metadata output from the video data monitoring unit 31 in step 314.

Step 322) Indexing step:
The indexing unit 32 receives the video data output from the video data monitoring unit 31 in step 313, performs feature extraction processing according to the method described in Document 1, outputs indexing metadata to the progress management unit 321, and also creates an indexing log file. Is output to the storage unit 20.

Step 323) Speech enhancement section extraction step:
The voice enhancement section extraction processing unit 322 receives the video data output from the video data monitoring unit 31 in step 313, determines the section where the voice is emphasized using the method of Document 2, and starts the start time of the section. The still image at the end time and the start time is output to the video data list update unit 323 as audio enhancement section information.

Step 324) Indexing progress management step:
The progress management unit 321 receives the indexing metadata output from the indexing unit 32 in step 322, and outputs the indexing metadata and indexing progress information (progress rate) in response to an inquiry from the video data list update unit 323. .

Step 325) Video data list creation step:
In the video data list update unit 323, the speech enhancement section information output from the speech enhancement section extraction processing unit 322 in step 323, the indexing metadata output from the indexing unit 32 in step 322, and the progress management in step 324 Based on the indexing progress information (progress rate) output from the unit 321, a video data list is created and stored in the storage unit 20 and displayed on the display unit 40.

  Next, the above-described indexing step (step 322) in FIG. 21 will be described in detail.

  FIG. 22 is a detailed flowchart of the indexing step in the sixth embodiment of the present invention.

Step 3221) Video data input step:
The indexing unit 32 inputs the video data output from the video data monitoring unit 31 in step 313.

Step 3222) Feature extraction step:
The indexing unit 32 extracts a feature amount from the video data input in Step 3221 by the method of Document 1, determines the success or failure of the event from the extracted result, and records it.

Step 3223) Indexing metadata output step:
The indexing unit 32 records the event recorded in step 3222 together with video data file information (including information generally described in properties such as file path information and file size) and video data playback time information. Output.

Step 3224) Indexing output step:
The indexing unit 32 outputs the execution result of the indexing to a log file in the storage unit 20. The log file includes path information of video data, error information during indexing, date and time, and the like.

  Next, the speech enhancement section extraction step (step 323) in FIG. 21 will be described in detail.

  FIG. 23 is a detailed flowchart of the speech enhancement section extraction step according to the sixth embodiment of the present invention.

Step 3231) Video Data Input Step:
In the audio enhancement section extraction processing unit 322, the video data output from the video data monitoring unit 31 in step 113 is input.

Step 3232) Emphasis section discrimination processing step:
The voice enhancement section extraction processing unit 322 determines a part where the voice is emphasized from the video based on the technique of Document 2.

Step 3233) Speech enhancement section information output step:
The voice enhancement section extraction processing unit 322 outputs the time information at the location determined in step 3232 and the still image at the start point to the video data list update unit 323.

  Next, the video data list creation step (step 325) in FIG. 21 will be described in detail.

  FIG. 24 is a detailed flowchart of the video data list creation step in the sixth embodiment of the present invention.

Step 3251) Speech enhancement section information input step:
The video data list update unit 323 receives the speech enhancement section information output from the speech enhancement section extraction processing unit 322 in Step 3233.

Step 3252) Indexing metadata input step:
The video data list update unit 323 receives the indexing metadata output from the indexing unit 32 in step 322 and the flag obtained in step 3256 described later.

Step 3253) Playlist generation step:
The video data list update unit 323 generates a link character string that can start reproduction based on the voice enhancement section information, indexing metadata, and flag obtained in Steps 3251 and 3152.

Step 3254) List file update step:
The video data list updating unit 323 overwrites the playlist information (link character string obtained in step 3253) on the video data list in the storage unit 20 and updates it.

Step 3255) List file output step:
The video data list update unit 323 displays the video list file updated in step 3254 on the display unit 40.

Step 3256) Indexing progress inquiry step:
The video data list update unit 323 inquires the progress management unit 321 about the indexing progress, and obtains the indexing progress. If the acquired progress (progress rate) is ahead of the previous progress (progress rate) recorded in a memory (not shown), a flag “updated” is output.

  In the present embodiment, a video data list update unit 323 is provided instead of the video data list generation unit 33, but the video data list update unit 323 is similar to the video data list generation unit 33 in the video data list generation unit 33. Therefore, as in the first to fifth embodiments, the additional metadata generation process of the subsequent additional metadata generation unit 34, the generation of the metadata database of the metadata database generation unit 35, the data Search / display processing of the search unit 36 can be performed.

  According to the present embodiment, even if a long-time video is input, it can be reflected in the video data list, so that it is easier to reach the desired video data as compared with the conventional technique.

  The operation in each of the above embodiments can be constructed as a program, installed in a computer used as a video data management apparatus and executed, or distributed via a network.

  The present invention is not limited to the above-described embodiment, and various modifications and applications can be made within the scope of the claims.

  INDUSTRIAL APPLICABILITY The present invention can be applied to a system that automatically generates index information related to video data and manages, searches, and browses in an apparatus that records video data.

It is a principle block diagram of this invention. It is a figure for demonstrating the principle of this invention. It is a block diagram of the video data management apparatus in the 1st Embodiment of this invention. It is a flowchart of the whole outline | summary operation | movement in the 1st Embodiment of this invention. It is a flowchart of the image | video monitoring step (s11) in the 1st Embodiment of this invention. It is a flowchart of the list | wrist and DB production | generation step (s12) in the 1st Embodiment of this invention. It is a general flowchart of the search step (s13) in the 1st Embodiment of this invention. It is a detailed flowchart of the indexing step (s122) in the 1st Embodiment of this invention. It is a detailed flowchart of the list preparation step (s123) in the 1st Embodiment of this invention. It is a detailed flowchart of the additional metadata production | generation step (s124) in the 1st Embodiment of this invention. It is a detailed flowchart of the text extraction step (s1244) in the 1st Embodiment of this invention. It is a block diagram of the video data management apparatus in the 2nd Embodiment of this invention. It is a flowchart of the whole outline | summary operation | movement in the 2nd Embodiment of this invention. It is a flowchart of the whole outline | summary operation | movement in the 3rd Embodiment of this invention. It is a block diagram of the video data management apparatus in the 4th Embodiment of this invention. It is a flowchart of the whole outline | summary operation | movement in the 4th Embodiment of this invention. It is a flowchart of the detailed operation | movement of the indexing step in the 5th Embodiment of this invention. It is a block diagram of the video data management apparatus in the 6th Embodiment of this invention. It is a flowchart of the outline | summary operation | movement of a process until the video data list output in the 6th Embodiment of this invention. It is a flowchart of the image | video monitoring step (S31) in the 6th Embodiment of this invention. It is a flowchart of the list production | generation step (S32) in the 6th Embodiment of this invention. It is a flowchart of the indexing step (S322) in the 6th Embodiment of this invention. It is a detailed flowchart of the audio | voice emphasis area extraction step (S323) in the 6th Embodiment of this invention. It is a detailed flowchart of the video data list creation step (S325) in the sixth embodiment of the present invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Video data input / output part 20 Storage means, Storage part 31 Video data monitoring means, Video data monitoring part 32 Indexing means, Indexing part 33 Video data list generation means, Video data list generation part 34 Additional metadata generation means, Additional metadata Generation unit 35 Metadata DB generation unit, metadata database generation unit 36 search unit, search unit 37 system management unit 38 task setting unit 40 display unit 50 metadata DB
321 Progress management unit 322 Voice enhancement section extraction processing unit 323 Video data list update unit

Claims (10)

A video data management device for generating index information related to video data and managing, searching, and browsing video data,
The video data in the storage means and the video data input / output to / from the recording medium are monitored, the video data to be indexed is specified, and related information given in advance at the time when the user acquires the video data is displayed. Video data monitoring means for outputting as initial metadata;
Indexing means for performing indexing on the video data designated by the video data monitoring means, and outputting indexing metadata in which feature information and time information of the video data are associated;
Additional metadata generation means for generating additional metadata, which is related information to be added to video data, based on the initial metadata acquired from the video data monitoring means and the indexing metadata acquired from the indexing means; ,
Metadata DB generation means for storing the initial metadata, the indexing metadata, and the additional metadata in a metadata database (DB);
Based on the input search key, the metadata DB is searched, information on the Internet is searched using an Internet search engine, and search results are newly displayed in a single window that is newly generated on the display means. Search means to
A video data management apparatus comprising: The additional metadata generation means includes
A means for performing an internet search using the initial metadata as a search key, removing a tag symbol from the obtained result page, extracting text, and adding it as additional metadata;
Means for acquiring a character string by telop recognition from the indexing metadata and giving it as additional metadata;
When multiple keywords in the electronic program guide are available as initial metadata, the number of keywords used as keys is reduced step by step, the tag symbol is removed from the result of the Internet search, and the text is extracted. Means for adding as additional metadata;
The video data management apparatus according to claim 1 including: The search means includes
As a result of searching the metadata DB, video data in which a word matching the field of the initial metadata or the indexing metadata field exists, and video data in which a word matching the field of the additional metadata exists And a means for displaying this in the window of the display device together with the representative image obtained by the video data list generation means;
Means for collectively displaying results obtained using the Internet search engine in the same window;
The video data management apparatus according to claim 1 including: The video data list generation means includes
2. The video data management according to claim 1, further comprising means for generating a list arranged for each type of event included in the indexing metadata including cut points and camera work, and displaying the list in the same window as the initial metadata. apparatus. The video data list generation means includes
The output image obtained by the indexing means includes means for selecting, from among a plurality of images indicating cut points, an image whose time in the video indicated by the cut image is equally close based on the indexing metadata. The video data management apparatus according to 1. A video data management device for generating index information related to video data and managing, searching, and browsing video data,
The video data in the storage means and the video data input / output to / from the recording medium are monitored, the video data to be indexed is specified, and related information given in advance at the time when the user acquires the video data is displayed. Video data monitoring means for outputting as initial metadata;
A voice enhancement section extracting means for extracting a section in which the audio of the video data is emphasized as a voice enhancement section;
Indexing means for performing indexing on the video data designated by the video data monitoring means, and outputting indexing metadata in which feature information and time information of the video data are associated;
An indexing progress management means for monitoring the progress of the indexing process in the indexing means and outputting the progress rate;
A video data list is created based on the indexing metadata output from the indexing means, the progress rate obtained from the indexing progress management means, and the voice enhancement section obtained from the voice enhancement section extraction means. Updating the video data list of the storage means and displaying the video data list on the display means;
Additional metadata generation means for generating additional metadata, which is related information to be added to video data, based on the initial metadata acquired from the video data monitoring means and the indexing metadata acquired from the indexing means; ,
Metadata DB generation means for storing the initial metadata, the indexing metadata, and the additional metadata in a metadata database (DB);
Based on the input search key, the metadata DB is searched, information on the Internet is searched using an Internet search engine, and search results are newly displayed in a single window that is newly generated on the display means. Search means to
A video data management apparatus comprising: The video data list update means includes:
The progress rate acquired from the indexing progress management means is compared with the progress rate acquired last time and stored in the storage means. If the progress rate is higher than the previous time, the video data list is updated, and the display means Including means for outputting to
The video data management apparatus according to claim 6. A video data management method for generating index information related to video data and managing, searching, and browsing video data,
In a video data management apparatus having video data monitoring means, indexing means, video data list generation means, additional metadata generation means, metadata DB generation means, search means,
The video data monitoring means monitors the video data in the storage means and the video data input / output to / from the recording medium, designates the video data to be indexed, and the user acquires the video data in advance. A video data monitoring step for outputting the given related information as initial metadata;
An indexing step in which the indexing means performs indexing on the video data designated in the video data monitoring step, and outputs indexing metadata in which feature information and time information of the video data are associated;
The video data list generation unit selects a representative image based on the indexing metadata obtained in the indexing step, generates a list of video data, and stores the video data list in a storage unit;
Based on the initial metadata acquired in the video data monitoring step and the indexing metadata acquired in the indexing step, the additional metadata generation means adds additional metadata that is related information to be added to video data. Additional metadata generation steps to generate,
A metadata DB generating means for storing the initial metadata, the indexing metadata, and the additional metadata in a metadata DB;
The search unit searches the metadata DB based on the input search key, searches information on the Internet using an Internet search engine, and newly generates a search result on the display unit. A search step that aggregates in the window;
A video data management method characterized by: A video data management method for generating index information related to video data and managing, searching, and browsing video data,
In a video data management apparatus having video data monitoring means, indexing means, voice enhancement section extraction processing means, progress management means, video data list update means, additional metadata generation means, metadata DB generation means, search means,
The video data monitoring means monitors the video data in the storage means and the video data input / output to / from the recording medium, designates the video data to be indexed, and the user acquires the video data in advance. A video data monitoring step for outputting the given related information as initial metadata;
A voice enhancement section extraction step in which the voice enhancement section extraction processing means extracts a section in which the voice of the video data is emphasized as a voice enhancement section;
An indexing step in which the indexing means performs indexing on the video data designated in the video data monitoring step, and outputs indexing metadata in which feature information and time information of the video data are associated;
The progress management means monitors the progress of the indexing process in the indexing step, and outputs an progress rate indexing progress management step;
The video data list updating means is based on the indexing metadata acquired in the indexing step, the progress rate acquired in the indexing progress management step, and the voice enhancement section obtained in the voice enhancement section extraction step. A video data list update step of creating a video data list, updating the video data list of the storage means, and displaying the video data list on the display means;
Based on the initial metadata acquired in the video data monitoring step and the indexing metadata acquired in the indexing step, the additional metadata generation means adds additional metadata that is related information to be added to video data. Additional metadata generation steps to generate,
A metadata DB generation step in which the metadata DB generation means stores the initial metadata, the indexing metadata, and the additional metadata in a metadata DB;
The search unit searches the metadata DB based on the input search key, searches information on the Internet using an Internet search engine, and newly generates a search result on the display unit. A search step that aggregates in the window;
A video data management method characterized by:   10. A video data management program that causes a computer to execute each step of the video data management method according to claim 8 or 9.

Images