JP2009181306A - VIDEO INDEXING DEVICE, VIDEO INDEXING METHOD, VIDEO INDEXING PROGRAM, AND ITS RECORDING MEDIUM - Google Patents

Abstract

High accuracy video indexing is realized even when the relationship between the definition label and the image content changes without taking much cost and time to collect dictionary data.
A definition label, a one-class identification function for determining whether or not a processing target image calculated from a normal case image by a one-class identification function calculating unit is an image representing a definition label, and a two-class identification function calculation The dictionary storage means 6 stores a two-class identification function for identifying the positive case image and the negative case image calculated by the means 5 from the positive case image and the negative case image. The labeling candidate detection means 7 receives the indexing target video, obtains a frame image as a candidate for giving a definition label using the one-class identification function stored in the dictionary storage means 6, and the indexing means 8 The image corresponding to the negative case image is excluded from the frame image thus obtained using the two-class discriminant function stored in the dictionary storage means 6, and the image to be assigned the definition label is determined.
[Selection] Figure 1

Description

  The present invention relates to a video indexing technique for automatically assigning an index required for searching and browsing for efficiently accessing a scene to be viewed in a video.

  Based on the relationship between predefined labels and image content, when a scene corresponding to a defined label appears in a video, a large number of conventional video indexing devices that assign a defined label to a scene Some dictionary data are indexed with high accuracy by collecting and learning in advance. For example, a universal definition label such as “person face” can collect a large amount of dictionary images in advance. A dictionary of human face images learned from dictionary data is prepared, and when it is determined that a face image is included from the similarity between each frame image of the indexing target video and the dictionary, the definition label of “person face” (See Non-Patent Document 1).

When a user defines a semantic label by specifying a sample image, a scene similar to the sample image is detected by a method such as template matching (see Non-Patent Document 2), and a semantic label is detected in the detected scene. Can be given.
Takamasa Takamasa, Tanaka Masayuki, Okutomi Masatoshi, “Proposal of Face Detection Using Evaluation Value Distribution of Facialness”, Transactions of Information Processing Society of Japan, Vol.48, No.SIG16, pp.51-54, 2007. Mikio Takagi and Yoshihisa Shimoda “New Image Analysis Handbook”, pp.1669-1675, 2004, University of Tokyo Press.

  The conventional technique using a large amount of dictionary data is problematic in that it takes a lot of cost and time to collect dictionary data, and it cannot be applied when the relationship between the definition label and the image content changes. . That is, each time a user search request occurs, a large amount of images representing the definition label must be collected. In addition, when the shooting method is changed in a scene that represents an object or event that has recently become a topic or in a broadcast video, the relationship between the definition label and the image content changes. It is necessary to collect and learn a large amount of dictionary data. As described above, there is a problem that the application area is limited because learning costs and time.

  In the conventional technique using template matching, it is difficult to obtain an optimum solution of the threshold value of similarity for determining whether to provide a definition label. In addition, when it is difficult to express definition labels with a small number of templates, the problem is that indexing accuracy is low.

  The present invention solves the above-described problems, and does not impose a burden on the collection of dictionary data, and the similarity threshold for determining whether to add a label such as a semantic label is not the optimum value. An object of the present invention is to provide a device capable of indexing with high accuracy.

  A principle configuration diagram of the present invention is shown in FIG. The video indexing apparatus of the present invention includes means as shown in FIG.

  The reference image selection means 1 is a means for selecting a reference image representing a definition label from a given dictionary video. The dictionary data collecting means 2 is a means for detecting an image similar to the reference image from the dictionary video. The positive case / negative case selection means 3 is a means for selecting a correct image as a positive case image and an incorrect image as negative case images in the order of similarity, as an image representing a definition label among the collected similar images.

  The one class discriminant function calculating unit 4 is a unit that calculates a one class discriminant function for discriminating whether or not the image represents a definition label from the positive case image. The 2-class discriminant function calculating means 5 is a means for calculating a 2-class discriminant function for discriminating both from the positive case image and the negative case image. The dictionary storage means 6 is a means for accumulating and storing the 1-class discrimination function calculated by the 1-class discrimination function calculation means 4 and the 2-class discrimination function calculated by the 2-class discrimination function calculation means 5 together with the definition label.

  The candidate for label addition detection means 7 is a candidate for giving a definition label to the frame image selected from the video using the one class identification function obtained by the one class identification function calculation means 4 when the index target video is given. Means for obtaining a frame image. The indexing unit 8 determines whether or not the candidate frame image obtained by the label addition candidate detection unit 7 is an image to which a definition label is to be assigned using the 2-class discrimination function obtained by the 2-class discrimination function calculation unit 5. It is a means to determine.

  According to the present invention, an object similar to a positive case image is extracted by a one-class identification function for extracting a positive case image, and then a negative case image using a two-class identification function for distinguishing a positive case image from a negative case image is used. By performing the exclusion, it is possible to appropriately determine whether or not the image is to be provided with a definition label.

  Appearance time (media time) and image feature quantity in the video of the frame image can be used as the feature quantity in the first class discrimination function calculation means 4 and the second class discrimination function calculation means 5.

  In addition to the above-described means, the present invention includes an event setting means that rules events according to the order relationship and time intervals in which a plurality of definition labels appear, and sets the rules and labels of events to be detected, and the indexing means 8 It is also possible to provide event label assigning means for assigning event labels based on the similarity between the order relationship and the time interval of the definition labels given in step 1 and the rules set by the event setting means.

  In the present invention, the one-class discriminant function calculating unit 4 can calculate the one-class discriminant function based on the similarity criterion in the dictionary data collecting unit 2. For example, as a feature value for discriminating whether or not the image represents a definition label from the positive case image in the one-class discrimination function calculation unit 4, a feature value used as a reference for image similarity in the dictionary data collection unit 2 Can be used.

  According to the present invention, it is possible to perform indexing with high accuracy by performing the determination by two-stage determination means, that is, determination by a one-class identification function and determination by a two-class identification function. Further, it is possible to eliminate the trouble of adjusting the threshold value of the one class identification function calculating means.

  In addition, the present invention provides an event label assigning means for assigning an event label by comparing the order relationship and time interval in which the definition labels appear with a preset rule, thereby further accurately adding a desired video section. Indexing can be performed.

  Furthermore, according to the present invention, dictionary data can be collected easily and efficiently by providing means for selecting a reference image, dictionary data collecting means, and means for selecting positive case images and negative case images.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. The video indexing apparatus according to the present invention is roughly composed of a dictionary generation unit that generates a dictionary and an indexing unit that assigns a label to the indexed video.

  FIG. 2 illustrates the configuration of the dictionary generation unit of the video indexing apparatus according to the first embodiment of the present invention, and FIG. 3 illustrates the configuration of the indexing unit of the video indexing apparatus according to the first embodiment of the present invention. FIG.

  2 includes a dictionary video storage unit 11, a reference image selection processing unit 12, an image feature extraction unit 13, a dictionary data collection processing unit 14, a positive case / negative case selection processing unit 15, and a feature amount extraction unit 16. , 1 class discrimination function calculation unit 17, 2 class discrimination function calculation unit 18, and definition label dictionary storage unit 19.

  The indexing unit in FIG. 3 includes an indexing video acquisition unit 20, a frame image acquisition unit 21, a feature amount extraction unit 16, a definition label dictionary storage unit 19, a candidate detection unit 22 based on a one class identification function, and an indexing unit 23 based on a two class identification function. Consists of.

  The dictionary video storage unit 11 stores and manages the dictionary video acquired in advance, and sends the dictionary video to the reference image selection processing unit 12 in response to a request from the reference image selection processing unit 12. Output.

  When the definition label is input, the reference image selection processing unit 12 issues a read request to the dictionary image storage unit 11 and represents a definition label from the dictionary image received from the dictionary image storage unit 11. Select a suitable frame image. The selected image is output as a reference image to the image feature extraction unit 13 together with the dictionary video. For example, an interface has been implemented to divide shots where the scene of the dictionary video changes greatly, display a list of the top images of each shot, and specify the reference image from among them by using a mouse as a reference image selection method. However, the method is not limited to this method, and the selection of the reference image can be easily performed by various methods.

  The image feature extraction unit 13 extracts image feature amounts related to color, texture, and the like from each frame image of the reference image and dictionary video received from the reference image selection processing unit 12, and each frame image in the extracted feature space is extracted. The feature vector is expressed as a feature vector, and the feature vector is output to the dictionary data collection processing unit 14.

  When the dictionary data collection processing unit 14 receives the feature vector of each image from the image feature extraction unit 13, the dictionary data collection processing unit 14 collects images similar to the reference image from the dictionary video, and the collected image group is a positive case / negative case. The data is output to the selection processing unit 15.

  The positive case / negative case selection processing unit 15 selects a predetermined number of positive case images representing the image contents of the definition label and negative case images not representing from the image group received from the dictionary data collection processing unit 14. Then, the positive case image and the negative case image are output to the feature amount extraction unit 16.

  The feature quantity extraction unit 16 extracts a feature quantity for calculating the discriminant function based on the image characteristics regarding the color and texture of the positive case image or the negative case image received from the positive case / negative case selection processing unit 15. , Each image in the extracted feature space is represented by a feature vector, the feature vector of the positive case image is sent to the one-class discriminant function calculating unit 17, and the feature vector of the positive case image and the negative case image is going to the two-class discriminant function calculating unit 18. Output each. As a specific example of the feature amount, a method of adding the appearance time in the image of the image to the image feature extracted by the image feature extraction unit 13 as a feature amount, or an image feature for all images of the positive case image and the negative case image. It is effective to perform a principal component analysis using the image features extracted by the extraction unit 13 and use a dimensionally compressed feature as a feature amount.

  Further, when receiving an image from the frame image acquisition unit 21 shown in FIG. 3, the feature amount extraction unit 16 performs similar feature extraction, and outputs the extracted feature vector to the candidate detection unit 22 using a one-class identification function. To do.

  The one-class identification function calculation unit 17 calculates a one-class identification function for determining whether the image represents a definition label in the feature space from the feature vector of the positive case image received from the feature amount extraction unit 16. Then, the calculated one-class identification function is output to the definition label dictionary storage unit 19.

  As a one-class discriminant function, for example, a method in which a spherical surface in which all positive case images are included in a spherical surface centered on an average vector of positive case images in a feature space is used as a discriminant function, or a positive case image in a feature space The probability distribution of a positive case image is expressed by a mixed Gaussian model obtained by modeling the distribution of multiple Gaussian distributions, and the probability of being a positive case image based on this probability distribution is greater than a preset threshold. For example, a method using an identification function to determine a positive case may be used. If the threshold setting at this time is set as low as 60%, for example, the detection omission due to the one-class identification function is reduced. Of course, the number of false detections increases, but this can be eliminated by the processing of the two-class discriminant function calculation unit 18, so that there is no problem. As described above, since it is not necessary to set the threshold value of the one-class discriminant function to the optimum value, it is possible to perform the indexing with high accuracy without costing the adjustment of the threshold value.

  The two-class discriminant function calculation unit 18 uses a feature case of the positive case image and the negative case image received from the feature amount extraction unit 16 to discriminate between an image representing a definition label and an image not represented in the feature space. The class identification function is calculated, and the calculated two-class identification function is output to the definition label dictionary storage unit 19. As the two-class discriminant function, for example, a Fisher discriminant function or SVM (Support Vector Machine) may be used.

  The definition label dictionary storage unit 19 stores the one class identification function received from the one class identification function calculation unit 17 and the two class identification function received from the two class identification function calculation unit 18 together with the definition label, and FIG. 1 class discriminant function or 2 class discriminant function is output in response to a request from the candidate detector 22 using the one class discriminant function and the indexing unit 23 using the two class discriminant function.

  Next, the indexed video acquisition unit 20 in FIG. 3 acquires a video to be indexed and outputs the acquired video to the frame image acquisition unit 21.

  The frame image acquisition unit 21 acquires a frame image by sampling the indexing target video received from the indexing video acquisition unit 20 at a predetermined interval, and outputs the acquired frame image to the feature amount extraction unit 16.

  Upon receiving the feature vector of each frame image from the feature quantity extraction unit 16, the candidate detection unit 22 based on the one class identification function outputs a definition label and a request signal for the one class identification function to the definition label dictionary storage unit 19. The definition label and the one class identification function are received from the dictionary storage unit 19. Using the received one-class discriminant function, it is determined whether each frame image is an image representing a definition label, an image determined to be an image representing a definition label is set as a candidate image, and a feature vector of the candidate image Is output to the indexing unit 23 using a two-class discriminant function.

  When receiving the feature vector of the candidate image from the candidate detection unit 22 based on the one class identification function, the indexing unit 23 based on the two class identification function outputs a definition label and a request signal for the two class identification function to the definition label dictionary storage unit 19. A definition label and a two-class identification function are received from the definition label dictionary storage unit 19. The indexing unit 23 based on the two-class discriminant function uses the received two-class discriminant function to determine an image representing the definition label in the candidate image, and indicates that the image has the definition label appearing in the video Is output as an indexing result.

  Next, a processing procedure in the above configuration will be described using an example of a dictionary video shown in FIG. The dictionary video is Vd, and the reference image of the definition label is Fd. In FIG. 4, sections 1, 2, and 3 are sections in which images representing definition labels appear.

  FIG. 5 is a flowchart showing the processing of the dictionary generation unit in the first embodiment of the present invention.

[Step S501]
The reference image selection processing unit 12 reads each frame image of the reference image and the dictionary video managed by the dictionary video storage unit 11. Assume that Fd shown in FIG. 4 is selected as the reference image.

  Next, the image feature extraction unit 13 extracts image feature amounts in the steps S502, S503, S504, and S505. Hereinafter, a case where a color moment is used as the color feature amount and a density gradient histogram is used as the texture feature amount will be described as an example.

[Step S502]
Since each frame image is represented by the three primary colors RGB, it is converted to the Lab color space.

[Step S503]
The image is divided into M × N blocks. FIG. 6 shows an example of division. In the example of FIG. 6, the frame image is divided into 4 × 4 = 16 block areas.

[Step S504]
The color moment is calculated from the color information of the pixels belonging to each block. For each component L, a, and b in the Lab color space, first, second, and third moments are calculated for all the pixels in the block. Since a 9-dimensional color moment is obtained from each block, the color feature from each frame image is represented by an M × N × 9-dimensional feature vector in this example.

[Step S505]
A density gradient histogram that is a texture feature is calculated only for the L component of the Lab color space.

  First, the edge direction and the edge strength are obtained for each pixel of the image representing the L component. When the value of the L component of the pixel (x, y) is L (x, y), the edge direction and the edge strength are obtained by the following equations.

Edge strength: sqrt (ΔX * ΔX + ΔY * ΔY)
Edge direction: arctan (ΔY / ΔX)
here,
ΔX = L (x + 1, y) −L (x, y)
ΔY = L (x, y + 1) −L (x, y)
Next, for each pixel in the block, the frequency distribution in the edge direction weighted by the edge strength is calculated. For the edge direction, for example, when the frequency is totaled by dividing 9 degrees into 0 degrees to 180 degrees at 20 degree intervals, the density gradient histogram of each block becomes 9 dimensions. In this case, the texture feature from each frame image is represented by a feature vector of M × N × 9 dimensions.

  Next, the dictionary data collection processing unit 14 collects dictionary data in steps S506 and S507.

[Step S506]
A distance r (i) between each frame image Fi (i = 1, 2,..., I) of the dictionary video Vd and the reference image Fd is calculated.

  The Euclidean distance between the M × N × 9-dimensional color feature vectors obtained from the reference image Fd and each frame image Fi is r_col, and the M × N × 9-dimensional texture feature obtained from the reference image Fd and each frame image Fi. Assuming that the Euclidean distance between vectors is r_tex, r (i) is obtained by the following equation as the weight sum thereof.

r (i) = w1 · r_col + w2 · r_tex
Here, w1 and w2 are constants set in advance.

[Step S507]
Frame images similar to the reference image are collected from the distance r (i) obtained in step S506 for each frame image of the dictionary video and the reference image. Using a preset threshold TH_r,
r (i) <TH_r
What is necessary is just to collect frame image Fi. In the case of FIG. 4, there are three sections in which a frame image representing a definition label appears, and there is one section that does not represent a definition label but is similar to the reference image, so frames in a total of four sections Images are collected as dictionary data.

  Next, the positive case / negative case selection processing unit 15 selects a positive case and a negative case in steps S508, S509, and S510.

[Step S508]
The collected images are sorted in order of decreasing distance r (i) from the reference image.

[Step S509]
A screen for checking the correct case images is presented. FIG. 7 shows an example of a GUI (Graphical User Interface) for checking a correct case image. For example, as shown in FIG. 7, it is only necessary to provide a GUI that displays a list of images on a display and allows the user to click an image that is regarded as a normal case image with a mouse.

[Step S510]
Using the GUI as shown in FIG. 7, an image checked as a positive case image is set as a positive case image, and a positive case is selected from the images not checked as a positive case image in ascending order of the distance r (i) from the reference image. Select as many negative images as there are images.

[Step S511]
The feature quantity extraction unit 16 calculates a feature vector from the positive case image and the negative case image. As an example of the feature vector, the image feature amount of the M × N × 9 dimensional color feature vector and the M × N × 9 dimensional texture feature vector is extracted by the same method as the procedure of steps S502, S503, S504, and S505. The method using only the image feature amount as the feature vector, the method of adding the appearance time (media time) in the dictionary video Vd to which the selected positive case image and negative case image belong to the image feature amount are effective. .

[Step S512]
The one class identification function calculation unit 17 calculates a one class identification function for identifying a positive case image. As the feature quantity used in the calculation of the one-class identification function, the feature quantity based on the image similarity used when the dictionary data collection processing unit 14 collects the dictionary data can be used. For example, a function that identifies whether the processing target image is an image representing a definition label based on the probability distribution of the positive case image in the feature space or the distance from the average feature vector of the positive case image is used as the one-class identification function. Can do.

[Step S513]
The 2-class discriminant function calculation unit 18 calculates a 2-class discriminant function for discriminating both from the positive case image and the negative case image.

[Step S514]
The one-class discriminant function and the two-class discriminant function of the definition label are stored in the definition label dictionary storage unit 19.

  With the above processing, a dictionary for the definition label can be generated.

  Next, the processing procedure of the indexing processing unit for determining whether or not to assign a definition label to each frame of the indexing target video will be described with reference to FIG.

[Step S801]
The indexing video acquisition unit 20 reads the indexing target video, and the frame image acquisition unit 21 captures the frame image from the indexing target video. In the following description, it is assumed that P frame images have been captured.

[Step S802]
In order to sequentially process P frame images, p = 1 is set as an initial value.

[Step S803]
The feature quantity extraction unit 16 calculates a feature quantity from the p-th (p = 1, 2,..., P) frame image.

[Step S804]
The candidate detection unit 22 based on the one-class identification function determines whether the p-th frame image is a candidate for adding a definition label using the one-class identification function stored in the definition label dictionary storage unit 19. When it becomes a candidate, it progresses to step S805, and when it becomes not a candidate, it progresses to step S807.

  As a determination method, for example, an average vector of positive case images is calculated for the feature vector obtained in step S511. If the Euclidean distance from the unknown image and the average vector is equal to or less than a preset threshold TH_pos1, the unknown image A method for determining that a definition label is to be assigned to the threshold value, and a distribution of the positive case image in the feature vector obtained in step S511 is estimated using a plurality of Gaussian distributions, and the likelihood of an unknown image with respect to the mixed Gaussian distribution is set in advance. If TH_pos2 or less, a method of determining that a definition label is to be given to the unknown image may be used.

[Step S805]
In the indexing unit 23 using the two-class discriminant function, it is determined whether a definition label is represented in the p-th frame image. If it is determined that the definition label is represented, the process proceeds to step S806. If it is determined that the definition label is not represented, the process proceeds to step S807.

[Step S806]
The media time of the p-th frame is stored as an indexing result of the definition label. That is, the media time of the frame image to which the definition label is assigned is stored.

[Step S807]
It is determined whether or not processing has been completed for all P frame images. If the process has not been completed, the process proceeds to step S808. If the process has been completed, the process proceeds to step S809.

[Step S808]
p = p + 1 is set, and the processing after step S803 is repeated.

[Step S809]
Among the P frame images, the media times of the frame images to which the definition label is added are collected as the indexing result, and the indexing result is output.

  With the above processing, a definition label can be assigned to the indexing target video.

  Next, a second embodiment of the present invention will be described. FIG. 9 is a diagram for explaining the configuration of the indexing unit of the video indexing apparatus according to the second embodiment of the present invention.

  In the second embodiment, the indexing processing unit of FIG. 9 includes a feature amount extraction unit 16, a definition label dictionary storage unit 19, an indexing video acquisition unit 20, a frame image acquisition unit 21, a candidate detection unit 22 based on one class identification function, It comprises an indexing unit 23 based on a two-class identification function, a frame image indexing result management unit 30, an event rule storage unit 31, and an event label assigning unit 32.

  Feature quantity extraction unit 16, definition label dictionary storage unit 19, indexing video acquisition unit 20, frame image acquisition unit 21, definition label dictionary storage unit 19, candidate detection unit 22 based on one class identification function, indexing unit 23 based on two class identification function Performs the same processing as in the first embodiment.

  The frame image indexing result management unit 30 manages the media time, which is output from the indexing unit 23 based on the two-class identification function, to which the definition label for the indexing target video is given.

  The event rule storage unit 31 sets an order relation and a time interval in which a plurality of definition labels appear as event label rules. An example of the event rule is shown in FIG. In the same figure, event rule 1 is a rule for assigning event label 1, a section to which definition label C is assigned appears after a section to which definition label A is assigned, and then a definition label is added. A rule of an event in which a section to which B is given appears and a definition label A appears at the end is shown. The event rule 2 is a rule for assigning the event label 2, and the definition label A appears again within 10 seconds of the section to which the definition label A is assigned, and again within 10 seconds. The rule of the event where the section of A appears is shown.

  When the event label assigning unit 32 receives the media time to which the definition label is assigned from the frame image indexing result managing unit 30, it outputs an acquisition request signal to the event rule storing unit 31 to read the event rule.

  Among the definition labels assigned to the indexing target video, the similarity with the read event rule is calculated, and if the similarity is equal to or higher than a preset threshold, the event label is assigned.

  Next, a processing procedure in the above configuration will be described. The definition label dictionary storage unit 19 manages dictionaries for the definition label A, definition label B, and definition label C, and the frame image indexing result management unit 30 performs indexing according to the procedure of the first embodiment of the present invention. An example will be described in which definition label A, definition label B, and definition label C are assigned to a video, and event label 1 and event label 2 in FIG.

  The event label assigning unit 32 calculates the similarity between the definition label assigned to the indexing target video managed by the frame image indexing result management unit 30 and the event rule stored in the event rule storage unit 31.

  As an example of the similarity calculation method, there is a method of obtaining a ratio of the number of sections having the same appearance order with respect to the number of defined sections of the event rule (4 for event rule 1 in FIG. 10 and 5 for event rule 2). For example, as shown in FIG. 11, if the definition label given to the indexed video has sections of definition label A, definition label C, definition label B, and definition label C in the order of the video media time, the event rule The similarity with 1 is 3/4 = 0.75. If it is set to add an event label if the degree of similarity is 0.75 or more, event label 1 is assigned to this section.

  By performing such processing for the event rule 1 and the event rule 2, an index can be assigned to a section in which a preset event appears.

  A specific example of event label assignment will be described.

[Example 1] For example, the following scene is set as a definition label in a soccer video.
・ Grand scene: The scene where the whole ground is reflected ・ Goal post scene: The scene where the goal post is reflected greatly ・ The human face scene: The scene where the person face is reflected greatly The event called the goal is the goal post scene after the grand scene Appears, and a human face scene appears within T seconds. If such an event rule is set, an event label for the goal can be automatically assigned to a video section where there is a high possibility that the goal is reflected in the soccer video.

[Example 2] For example, the following scene is set as a definition label in a news video.
-Announcer scene: Scene where the announcer is shown-Telop scene: Scene where the telop is displayed Topic transition events are defined as the telop scene appears immediately after the announcer scene, and the announcer scene does not appear for more than T seconds. . If such an event rule is set, it is possible to detect a topic transition event in a news video and automatically assign the event label.

  The above video indexing processing can be realized by a computer and a software program, and the program can be provided by being recorded on a computer-readable recording medium or provided through a network.

It is a principle block diagram of this invention. It is a block diagram of the dictionary production | generation part of the image | video indexing apparatus in 1st Example of this invention. It is a block diagram of the indexing part of the image | video indexing apparatus in 1st Example of this invention. It is a figure which shows the example of the image | video for dictionary. It is a flowchart of the dictionary production | generation process in 1st Example of this invention. It is a figure which shows the example of the block division of a frame image. It is a figure which shows the example of GUI which checks a positive example image. It is a flowchart of the indexing process in 1st Example of this invention. It is a block diagram of the indexing part of the image | video indexing apparatus in 2nd Example of this invention. It is a figure which shows the example of an event rule. It is a figure which shows the example of a definition label provision.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Reference image selection means 2 Dictionary data collection means 3 Positive case / negative case selection means 4 1 class discrimination function calculation means 5 2 class discrimination function calculation means 6 Dictionary storage means 7 Label addition candidate detection means 8 Indexing means

Claims (12)

In a video indexing device that predefines a definition label that represents image content and assigns a definition label to the scene when a scene corresponding to the definition label appears in a video to be indexed,
It is determined whether or not the processing target image is an image representing the definition label, which is calculated from the definition label and the feature amount extracted from the correct positive case image representing the definition label given as the learning image. A positive case image and a negative case image calculated by a feature amount extracted from a class identification function and a correct positive case image and an incorrect negative case image representing the definition label given as a learning image. Dictionary storage means for storing, as dictionary data, a two-class identification function for identification;
When an image to be indexed is given, the image represents a definition label stored in the dictionary storage unit using a one-class identification function stored in the dictionary storage unit for a frame image selected from the video. Labeling candidate detection means for determining whether or not to determine a frame image as a candidate for giving the definition label;
For the frame image that is a candidate by the labeling candidate detection means, by determining whether it corresponds to a positive case image or a negative case image using a two-class identification function stored in the dictionary storage means, An indexing unit that eliminates images corresponding to negative case images from candidate frame images, determines an image to which the definition label corresponding to a positive case image is to be assigned, and assigns a definition label. A video indexing device. The video indexing device according to claim 1,
The video indexing device, wherein the 1-class discriminant function and the 2-class discriminant function are functions that identify an appearance time and an image feature amount in a video of a frame image as a feature amount. The video indexing device according to claim 1 or 2,
An event rule storage means for storing an event label representing an event in the video and an event rule defined by an order relation or a time interval in which a plurality of definition labels for detecting the event in the video appear;
The order relationship or time interval of the definition labels given by the indexing means is collated with the event rules stored in the event rule storage means, and the event label is displayed on the video portion whose similarity is greater than a predetermined threshold. An image labeling device, further comprising: In the video indexing device according to claim 1, claim 2 or claim 3,
A reference image selection means for selecting a reference image representing a definition label from the dictionary image when the dictionary image is given;
Dictionary data collection means for detecting an image similar to the reference image from the dictionary video;
A positive case / negative case selection means for selecting a correct image as a positive case image and an incorrect image as a negative case image in the order of similarity to the reference image among the collected similar images representing the definition label;
One class identification function calculating means for calculating one class identification function for determining whether or not the image represents a definition label based on the feature amount extracted from the positive case image;
2-class discriminant function calculating means for calculating a 2-class discriminant function for discriminating both based on the feature values extracted from the positive case image and the negative case image,
The one-class identification function calculated by the one-class identification function calculating means, the two-class identification function calculated by the two-class identification function calculating means, and the definition label are stored as dictionary data in the dictionary storage means. A video indexing device characterized by that. The video indexing device according to claim 4,
The video indexing apparatus according to claim 1, wherein the one-class discriminant function calculating unit calculates the one-class discriminant function based on a similarity criterion in the dictionary data collecting unit. In a video indexing method executed by a video indexing device that predefines a definition label that represents image content and assigns a definition label to the scene when a scene corresponding to the definition label appears in the video to be indexed ,
It is determined whether or not the processing target image is an image representing the definition label, which is calculated from the definition label and the feature amount extracted from the correct positive case image representing the definition label given as the learning image. A positive case image and a negative case image calculated by a feature amount extracted from a class identification function and a correct positive case image and an incorrect negative case image representing the definition label given as a learning image. Refer to the dictionary storage means for storing the two-class identification function for identification as dictionary data,
When an image to be indexed is given, the image represents a definition label stored in the dictionary storage unit using a one-class identification function stored in the dictionary storage unit for a frame image selected from the video. A labeling candidate detection process for determining whether or not a frame image that is a candidate for giving the definition label,
For the frame image that is a candidate by the labeling candidate detection process, by determining whether it corresponds to a positive case image or a negative case image using a two-class identification function stored in the dictionary storage means, An image corresponding to the negative case image is excluded from the candidate frame images, an image to which the definition label corresponding to the positive case image is to be assigned is determined, and an indexing process for assigning the definition label is executed. Characteristic video indexing method. The video indexing method according to claim 6,
The video indexing method, wherein the 1-class discriminant function and the 2-class discriminant function are functions for discriminating an appearance time and an image feature quantity in a video of a frame image as a feature quantity. The video indexing method according to claim 6 or 7,
Referring to an event rule storage means for storing an event label representing an event in the video and an event rule defined by an order relation or a time interval in which a plurality of definition labels for detecting the event in the video appear,
The event label stored in the event rule storage means is collated with the order relationship or time interval of the definition labels given in the indexing process, and the event label is displayed on the video portion whose similarity is greater than a predetermined threshold. And an event label assigning process for assigning a video indexing method. In the video indexing method according to claim 6, claim 7 or claim 8,
A reference image selection process for selecting a reference image representing a definition label from a dictionary image when a dictionary image is given;
Dictionary data collection processing for detecting an image similar to the reference image from the dictionary video;
A positive case / negative case selection process for selecting a correct image as a positive case image and an incorrect image as a negative case image in order of similarity to the reference image among the collected similar images,
A one-class identification function calculating process for calculating a one-class identification function for determining whether or not the image represents a definition label based on the feature amount extracted from the positive case image;
Performing a two-class identification function calculation process for calculating a two-class identification function for identifying both based on the feature values extracted from the positive case image and the negative case image,
The one class identification function calculated by the one class identification function calculation process, the two class identification function calculated by the two class identification function calculation process, and the definition label are stored as dictionary data in the dictionary storage means. A video indexing method characterized by the above. The video indexing method according to claim 9,
In the one-class identification function calculation process, the one-class identification function is calculated based on a similarity criterion in the dictionary data collection process.   A video indexing program for causing a computer to execute the video indexing method according to any one of claims 6 to 10.   A computer-readable recording medium on which the video indexing program according to claim 11 is recorded.

Images