WO2015122162A1 - Video search device - Google Patents

Abstract

A video search device is provided with: a metadata extraction unit that extracts a predetermined type of metadata from video data that is input from the outside; a metadata storage unit that associates the metadata that is extracted by the metadata extraction unit with the video data of the extraction source and stores the result; an image data input unit that inputs image data; a search metadata extraction unit that extracts a predetermined type of search metadata from the image data that is input by the image data input unit; a search processing unit that searches for metadata that is associated with the search metadata on the basis of the search metadata and the metadata; and a search metadata verification unit that displays the search metadata and a choice relating to search advisability as search precondition information.

Description

Video search device

The present invention relates to a metadata confirmation device, a video search device, a video search method, and a program.

In recent years, many surveillance cameras have been installed in various places such as urban areas and stores. The purpose of installing the surveillance camera includes a purpose as a crime deterrent by making the person think that he / she has taken a picture, and a purpose as a record for on-site verification when a crime occurs.

Also, by installing surveillance cameras, for example, when violent crimes occur, there are many cases in which the criminal images shown on these surveillance cameras lead to criminal clearance. Various effects are exhibited by installing the surveillance camera in this way. For this reason, surveillance cameras are expected to be used more frequently as a tool for solving problems.

On the other hand, a vast amount of video data will be transmitted from surveillance cameras installed in various places. For this reason, a technique for quickly and accurately searching for target video data from an enormous amount of video data is required.

For example, Patent Document 1 discloses a technique for quickly and accurately searching for target video data from a huge amount of video data. Patent Document 1 describes a technique for generating metadata from a combined image obtained by combining a plurality of images, and collating an object included in the image based on the generated metadata.

Also, as a technique for extracting metadata from image data, for example, there is Patent Document 2. In Patent Literature 2, metadata generation means for generating metadata from image data, metadata display control means for displaying metadata on a display unit, metadata correction means for accepting correction of displayed metadata, A scanner device is described. The scanner apparatus described in Patent Document 2 has the above-described configuration, so that the user can check the processing result of the scanned image data without scanning again.

JP 2009-0554444 A JP 2009-021861 A

By the way, when searching for target video data quickly and accurately from an enormous amount of video data, it is highly convenient if the search conditions can be input as sample images instead of text. However, when a search is performed using a sample image, there may be a difference in recognition between the user and the video search device that searches for video data. Specifically, for example, in a sample image showing yellow clothes illuminated by the sunset, a person (user) recognizes that they are yellow clothes, while the video search device that performs the search contains many red components. It may be judged that it is red clothes. The reverse is also true. As described above, when a search condition is input as image data, a search condition different from the search condition intended by the user may be input.

In such a case, for example, in the technique described in Patent Document 1, the video search device may search for a user's unexpected one. In such a case, the user can recognize that the search condition is different from the intended one only after confirming the search result. Therefore, after confirming the search result, it is necessary to replace the image data serving as a search condition and perform another search. As a result, there has been a problem that it takes time to perform a search under an appropriate search condition.

Further, the technique described in Patent Document 2 is a technique related to a scanner, and cannot be applied to an apparatus that simply searches for image data. In particular, Patent Document 2 does not describe anything about shortening the time taken to perform a search under desired conditions.

As described above, since there is a difference in recognition between a person and an apparatus, there has been a problem that a search may not be performed under a search condition intended by the user. In addition, there has been a problem that it may take time until a search is performed under a search condition intended by the user.

Therefore, the object of the present invention is the above-described problem, where there are cases where the search cannot be performed with the search condition intended by the user, and it takes time until the search is performed with the search condition intended by the user. It is an object of the present invention to provide a video search apparatus that solves the problem that there is a case where the video search ends.

In order to achieve such an object, a metadata checking apparatus according to one aspect of the present invention is provided.
An image data input unit for inputting image data;
A search metadata extraction unit for extracting a predetermined type of search metadata from the image data input by the image data input unit;
A search metadata confirmation unit for displaying the search metadata and search availability options as pre-search condition information;
Is provided.

In addition, a video search apparatus according to another embodiment of the present invention
A metadata extraction unit that extracts predetermined types of metadata from video data input from the outside;
A metadata storage unit that stores the metadata extracted by the metadata extraction unit in association with the video data of the extraction source;
An image data input unit for inputting image data;
A search metadata extraction unit for extracting a predetermined type of search metadata from the image data input by the image data input unit;
A search processing unit for searching for the metadata related to the search metadata based on the search metadata and the metadata;
A search metadata confirmation unit for displaying the search metadata and search availability options as pre-search condition information;
Is provided.

In addition, a video search method according to another aspect of the present invention includes:
Extract metadata of a predetermined type from video data input from the outside, store the metadata in association with the video data of the extraction source,
Image data is input, search metadata of a predetermined type is extracted from the input image data, and the search metadata and search availability options are displayed as pre-search condition information.

Moreover, the program which is the other form of this invention is:
Video search device
A metadata extraction unit that extracts predetermined types of metadata from video data input from the outside;
A metadata storage unit that stores the metadata extracted by the metadata extraction unit in association with the video data of the extraction source;
An image data input unit for inputting image data;
A search metadata extraction unit for extracting a predetermined type of search metadata from the image data input by the image data input unit;
A search processing unit for searching for the metadata related to the search metadata based on the search metadata and the metadata;
This is a program for functioning as a search metadata confirmation unit that displays the search metadata and search availability options as pre-search condition information.

The present invention is configured as described above, and thus can provide a video search device that can perform a search with a search condition intended by the user in a short time.

It is a figure explaining the outline of the video search device concerning a 1st embodiment of the present invention. 1 is a block diagram illustrating a configuration of a video search device according to a first embodiment of the present invention. It is a flowchart which shows an example of the operation | movement at the time of the video search device which concerns on the 1st Embodiment of this invention memorize | stores metadata and video data. It is a flowchart which shows an example of the operation | movement at the time of the video search apparatus concerning the 1st Embodiment of this invention searching video data. It is a flowchart which shows an example of the operation | movement at the time of the video search apparatus relevant to this invention searching video data. It is a block diagram which shows the structure of the video search apparatus which concerns on the 2nd Embodiment of this invention. It is a block diagram which shows the structure of the metadata extraction part with which the video search device shown in FIG. 6 is provided. It is a block diagram which shows the structure of the search condition input item part with which the video search device shown in FIG. 6 is provided. It is a block diagram which shows the structure of the metadata production | generation part for collation with which the video search device shown in FIG. 6 is provided. It is a figure for demonstrating the display of a search condition display part. It is a figure for demonstrating the display of a search condition display part. It is a figure for demonstrating the display of a search condition display part. It is a block diagram which shows the structure of the search process part with which the video search device shown in FIG. 6 is provided. 7 is a flowchart illustrating an operation when the metadata extraction unit illustrated in FIG. 6 extracts metadata from video data. It is a flowchart which shows an example of the operation | movement at the time of the video search apparatus concerning the 2nd Embodiment of this invention searching video data. It is a schematic block diagram which shows the outline of a structure of the video search device concerning the 3rd Embodiment of this invention. It is a schematic block diagram which shows the outline of a structure of the metadata confirmation apparatus which concerns on the 4th Embodiment of this invention.

Next, embodiments of the present invention will be described in detail with reference to the drawings.
[First embodiment]
Referring to FIG. 1, a video search device 1 according to a first embodiment of the present invention is a device that stores video data acquired by an external device such as a surveillance camera. As will be described later, the video search device 1 according to the present embodiment is configured to be able to search video data stored therein by inputting image data to the video search device 1. In addition, as described later, the video search device 1 in the present embodiment displays the search metadata on an output device such as a monitor before searching for video data using the search metadata extracted from the image data. Is configured to do.

Referring to FIG. 2, the video search apparatus 1 according to the first embodiment of the present invention includes a video data receiving unit 11, a metadata extracting unit 12, a metadata storage unit 13, a video data storage unit 14, The image data input unit 15, the search metadata extraction unit 16, the search metadata confirmation unit 17, the search processing unit 18 (search processing unit), and a search result display unit 19 are included. Note that the video search device 1 is an information processing device including an arithmetic device and a storage device. The video search device 1 realizes each of the above functions when the arithmetic device executes a program stored in the storage device.

The video data receiving unit 11 has a function of receiving video data from an external device such as a surveillance camera. Specifically, the video data receiving unit 11 is connected to an external device such as a monitoring camera via an external network, for example, and receives video data from the external device via the external network. Alternatively, the video data receiving unit 11 acquires video data from an external device such as a monitoring camera directly connected to the video data receiving unit 11. Then, the video data receiving unit 11 performs processing necessary to process the received video data, and then transmits the video data to the metadata extraction unit 12 and the video data storage unit 14.

The metadata extraction unit 12 has a function of extracting a predetermined type of metadata from the video data. The metadata extraction unit 12 receives the video data transmitted by the video data reception unit 11. Subsequently, the metadata extraction unit 12 extracts a predetermined type of metadata from the received video data.

Here, the predetermined type of metadata is metadata for identifying video data or a person in the video data. As metadata extracted by the metadata extraction unit 12, for example, the facial features, age, gender, clothing color of the upper body and the lower body of a person can be considered. In addition, the metadata extraction unit 12 can be configured to extract, for example, the image shooting date and time, the shooting location, the title and comment set by the user, and the name of the person as metadata. In addition to the above description, the metadata extraction unit 12 can extract various types of metadata for identifying video data and persons.

In this manner, the metadata extraction unit 12 extracts a predetermined type of metadata from the video data. Then, the metadata extraction unit 12 transmits the extracted metadata to the metadata storage unit 13.

Note that the metadata extraction unit 12 can retrieve the source video data from a predetermined type of metadata extracted by the metadata extraction unit 12 (extraction of the metadata and the metadata). Extract metadata (so that it can be associated with the original video data). For example, the metadata extraction unit 12 adds information for identifying video data from which the metadata is extracted to metadata extracted from the video data (for example, a shooting date and a shooting location). Further, at that time, the metadata extraction unit 12 can include information (for example, information indicating the time when the person is shown) necessary for reproducing the video data from the portion where the person shown by the image data is shown. By including such information in the metadata, as described later, it becomes possible to output video data corresponding to the retrieved metadata.

The metadata storage unit 13 includes a storage device such as a hard disk or a RAM (Random Access Memory). The metadata storage unit 13 receives the metadata transmitted by the metadata extraction unit 12. The metadata storage unit 13 stores the received metadata. Thereafter, the metadata storage unit 13 transmits the metadata stored therein to the search processing unit 18 in response to an instruction from the search processing unit 18.

As described above, the metadata stored in the metadata storage unit 13 includes information for identifying the video data from which the metadata is extracted. That is, the metadata storage unit 13 stores the metadata in association with the source video data.

The video data storage unit 14 includes a storage device such as a hard disk or a RAM (Random Access Memory). The video data storage unit 14 receives the video data transmitted by the video data receiving unit 11. Then, the video data storage unit 14 stores the received video data. Thereafter, the video data storage unit 14 transmits the video data stored therein to the search result display unit (or the search processing unit 18) in response to an instruction from the search result display unit 19 (or the search processing unit 18). .

As described above, the metadata storage unit 13 stores metadata associated with video data. Therefore, when metadata is searched for in the search processing unit 18 described later, the video data from which the metadata is extracted is searched from the video data storage unit 14 using information for identifying the video data included in the metadata. It becomes possible to do.

The image data input unit 15 has a function of inputting image data. The image data input unit 15 is configured to acquire image data from an external network or to be connected to an external device such as a digital camera, for example. Then, one or a plurality of image data is input to the image data input unit 15 through the external device such as the external network or the digital camera. In this way, the image data input unit 15 can input image data using various methods for inputting image data. When image data is input to the image data input unit 15 using such various methods, the image data input unit 15 transmits the input image data to the search metadata extraction unit 16.

The search metadata extraction unit 16 has a function of extracting a predetermined type of search metadata from the image data. The search metadata extraction unit 16 receives the image data transmitted from the image data input unit 15. Then, the search metadata extraction unit 16 extracts a predetermined type of search metadata from the received image data.

Here, the predetermined type of search metadata refers to the same type of metadata as the metadata extracted by the metadata extraction unit 12 described above. In other words, the metadata extraction unit 12 and the search metadata extraction unit 16 include, for example, the same metadata extraction engine, and are configured to be able to extract the same type of metadata. The metadata extracted by the search metadata extraction unit 16 is the same as the metadata extracted by the metadata extraction unit 12, and therefore, detailed description thereof is omitted.

In this manner, the search metadata extraction unit 16 extracts the same type of search metadata as the metadata extracted by the metadata extraction unit 12. Then, the search metadata extraction unit 16 transmits the extracted search metadata to the search metadata confirmation unit 17.

The search metadata confirmation unit 17 has a function of displaying the search metadata as pre-search condition information before performing a search on the metadata storage unit 13 using the search metadata. The search metadata confirmation unit 17 receives the search metadata transmitted by the search metadata extraction unit 16. Then, the search metadata confirmation unit 17 displays the received search metadata on an output device such as a monitor connected to the video search device 1, for example.

Further, the search metadata confirmation unit 17 selects whether to search the metadata storage unit 13 by the search processing unit 18 to be described later using the search metadata according to the displayed pre-search condition information. Configured to be able to. That is, when the pre-search condition information displayed by the search metadata confirmation unit 17 is different from what the user intended, the user performs a search using the search metadata indicated by the different pre-search condition information. It is possible to instruct the search metadata confirmation unit 17 to stop performing. On the other hand, when the pre-search condition information displayed by the search metadata confirmation unit 17 is what the user intended, the user performs a search using the search metadata indicated by the pre-search condition information. The search metadata confirmation unit 17 can be instructed. Such a function, for example, when displaying the search metadata on an output device such as a monitor, displays a button for selecting whether or not to perform a search (displays a search possibility option). This can be realized by configuring the data confirmation unit 17.

Further, the search metadata confirmation unit 17 can be configured so that the search metadata displayed as the pre-search condition information can be corrected according to the input from the user. For example, it is conceivable that the search metadata check unit 17 is configured to correct the search metadata by reinserting the image data into the image data input unit 15. Further, the search metadata confirmation unit 17 may be configured so that the search metadata displayed on the output device can be directly corrected by using a keyboard or the like connected to the video search device 1. . In addition to the case described above, the search metadata confirmation unit 17 can have various modifications. Note that when the search metadata check unit 17 corrects the search metadata, the corrected search metadata is displayed on an output device such as a monitor. Therefore, as described later, the search by the search processing unit 18 is performed using the corrected search metadata.

Thus, the search metadata confirmation unit 17 displays the search metadata as pre-search condition information on an output device such as a monitor. Further, the search metadata confirmation unit 17 accepts correction of the displayed search metadata. Thereafter, when execution of the search is selected by the user, the search metadata confirmation unit 17 searches the search metadata (search metadata displayed on the output device or the like) instructed to execute the search. The data is transmitted to the processing unit 18. On the other hand, when search cancellation is selected by the user, the search metadata confirmation unit 17 discards, for example, the search metadata for which search cancellation has been selected. Then, it waits for input of new image data (reception of search metadata).

The search processing unit 18 has a function of searching the metadata storage unit 13 using the search metadata displayed as the pre-search condition information. The search processing unit 18 receives the search metadata transmitted by the search metadata confirmation unit 17. Subsequently, the search processing unit 18 searches the metadata storage unit 13 for metadata corresponding to the search metadata using the received search metadata.

Here, for example, the search processing unit 18 may be configured to search the metadata storage unit 13 for metadata that matches the search metadata. Alternatively, for example, the search processing unit 18 may be configured to search the metadata storage unit 13 for metadata that is the same as or similar to the search metadata.

It should be noted that the search processing unit 18 can be configured to determine the similarity between the search metadata and the metadata stored in the metadata storage unit 13 using various general methods. Specifically, for example, a search is performed so that the distance between the search metadata and the metadata is determined by a numerical value as a distance, and the search metadata and the metadata are determined to be similar when the distance is equal to or less than a threshold value. It is conceivable to configure the processing unit 18. In addition to adopting the method as described above, the search processing unit 18 may be configured to determine whether or not they are similar using various general criteria for determining similarity. I can do it.

In this way, the search processing unit 18 searches the metadata storage unit 13 for metadata corresponding to the search metadata. Then, the search processing unit 18 transmits the searched metadata to the search result display unit 19.

The search result display unit 19 is connected to an external device such as a monitor. The search result display unit 19 has a function of acquiring video data related to the metadata searched by the search processing unit 18 from the video data storage unit 14 and displaying the acquired video data on an external device such as the monitor. The search result display unit 19 receives the metadata transmitted by the search processing unit 18. Subsequently, the search result display unit 19 acquires video data related to the received metadata from the video data storage unit 14 using information included in the received metadata. Thereafter, the search result display unit 19 displays the acquired video data on an output device such as a monitor.

Note that the output device such as a monitor on which the search result display unit 19 displays video data may be the same as the output device such as a monitor on which the search metadata confirmation unit 17 displays pre-search condition information. It does not matter if they are different.

In the present embodiment, the search result display unit 19 acquires video data related to metadata from the video data storage unit 14. However, the search processing unit 18 may be configured to acquire video data related to metadata. In this case, the search result display unit 19 can be configured to receive only video data, or can be configured to receive video data and metadata.

Further, in the present embodiment, the case has been described where the storage of video data received from an external device such as a surveillance camera and the search of the stored video data are performed by one video search device 1. However, the present invention is not limited to a case having such a configuration. For example, the present invention includes a device that receives video data from an external device such as a monitoring camera, extracts and stores metadata, a device that inputs image data and checks search metadata before performing a search, It can also be implemented by providing the two information processing apparatuses. Thus, the present invention may be configured from a plurality of information processing apparatuses.

The above is the configuration of the video search device 1. By providing the video search apparatus 1 with the above configuration, the video search apparatus 1 displays the search metadata as pre-search condition information before performing a search on the metadata storage unit 13 using the search metadata. I can do it. As a result, the user confirms the displayed pre-search condition information, so that it is possible to confirm whether or not the search condition is intended by the user before the actual search is performed.

In the present embodiment, the video search device 1 includes the metadata storage unit 13 and the video data storage unit 14 separately. However, the video search device 1 may be configured to include one storage device that stores metadata and video data.

Next, the operation of the video search device 1 according to this embodiment will be described.

First, an operation for acquiring a predetermined type of metadata from video data input from an external device such as a monitoring camera and storing the acquired metadata will be described.

Referring to FIG. 3, the video data is input to the video data receiving unit 11 of the video search device 1 (S101). Then, the video data receiving unit 11 to which the video data is input performs processing necessary for processing the video data, and then transmits the video data to the video data storage unit 14 and the metadata extraction unit 12. . The video data transmitted to the video data storage unit 14 is stored in the video data storage unit 14.

The video data transmitted by the video data receiving unit 11 is received by the metadata extracting unit 12. Then, the metadata extraction unit 12 extracts a predetermined type of metadata from the received video data (S102). Thereafter, the metadata extraction unit 12 transmits the extracted metadata to the metadata storage unit 13.

Note that the predetermined type of metadata is metadata for identifying video data and a person in the video data. Details of the metadata extracted by the metadata extraction unit have already been described, and will not be described.

In addition, the metadata extraction unit 12 can retrieve the source video data from a predetermined type of metadata extracted by the metadata extraction unit 12 (extraction of metadata and the metadata). Extract metadata (so that it can be associated with the original video data). For example, the metadata extraction unit 12 adds information for identifying video data from which the metadata is extracted to metadata extracted from the video data (for example, a shooting date and a shooting location). Further, the metadata extraction unit 12 gives information (for example, information indicating the time when the person is shown) necessary for reproducing the video data from the portion where the person shown by the image data is shown. The metadata extraction unit 12 transmits the metadata generated in this way to the metadata storage unit 13.

Thereafter, the metadata storage unit 13 receives the metadata transmitted by the metadata extraction unit 12. Then, the metadata storage unit 13 stores the received metadata. Here, as described above, the metadata stored in the metadata storage unit 13 includes information for identifying video data from which the metadata is extracted. That is, the metadata storage unit 13 stores the metadata in association with the source video data (S103).

By such an operation, the video search device 1 can acquire a predetermined type of metadata from video data input from an external device such as a surveillance camera, and store the acquired metadata. Further, the metadata stored in the video search device 1 is associated with the video data from which the metadata is extracted. Next, an operation when searching for video data stored in the video search device 1 will be described.

Referring to FIG. 4, image data is input to the image data input unit 15 of the video search device 1 (S201). Then, the image data input unit 15 to which the image data is input transmits the input image data to the search metadata extraction unit 16.

Next, the search metadata extraction unit 16 receives the image data transmitted by the image data input unit 15. Then, the search metadata extraction unit 16 extracts a predetermined type of search metadata from the received image data (S202). Thereafter, the search metadata extraction unit 16 transmits the extracted search metadata to the search metadata confirmation unit 17.

The predetermined type of search metadata refers to the same type of metadata as the metadata extracted by the metadata extraction unit 12 (stored in the metadata storage unit 13). That is, the metadata extraction unit 12 and the search metadata extraction unit 16 include, for example, the same metadata extraction engine, and extract the same type of metadata.

Subsequently, the search metadata confirmation unit 17 receives the search metadata transmitted by the search metadata extraction unit 16. Then, the search metadata confirmation unit 17 displays the search metadata as pre-search condition information on an output device such as a monitor before performing a search on the metadata storage unit 13 using the search metadata. (S203). As a result, it becomes possible for the user who has seen the pre-search condition information displayed on the output device to determine whether to perform a search using the search metadata displayed as the pre-search condition information. (S204).

When it is determined that the user does not perform the search using the pre-search condition information displayed on the output device (S204, NO), the user does not perform the search using the pre-search condition information displayed on the output device. To the search metadata confirmation unit 17. In this case, for example, the user replaces the image data and starts again from the input of the image data. The operation here is not limited to the case of replacing image data. For example, the search metadata may be corrected using a keyboard connected to the video search device 1.

On the other hand, when it is determined that the user performs a search using the pre-search information displayed on the output device (S204, YES), the search metadata confirmation unit 17 displays the pre-search condition information displayed on the output device. The search metadata shown is transmitted to the search processing unit 18. Then, the search processing unit 18 that has received the search metadata transmitted by the search metadata confirmation unit 17 searches the metadata stored in the metadata storage unit 13 using the search metadata. (S205).

Thereafter, the search processing unit 18 transmits the metadata searched from the metadata storage unit 13 to the search result display unit 19 using the search metadata. Then, the search result display unit 19 that has received the metadata transmitted by the search processing unit 18 acquires video data related to the received metadata from the video data storage unit 14 and displays the acquired video data ( S206).

By such an operation, the video search device 1 can search the video data stored in itself using the input image data.

As described above, the video search apparatus 1 according to this embodiment includes the search metadata check unit 17. With such a configuration, the video search apparatus 1 can display the search metadata as pre-search condition information before performing a search on the metadata storage unit 13 using the search metadata. As a result, it is possible to check the search metadata as pre-search condition information before searching for metadata using search metadata that requires time, and the search intended by the user in a short time. It becomes possible to search by condition. In addition, since the search is performed after the search metadata is confirmed, the suspicion (for the device) with respect to the search result can be solved.

On the other hand, referring to FIG. 5, when the search metadata confirmation unit 17 is not provided, the result of the metadata search using the search metadata that takes a long time is associated with the searched metadata. Only when the video data is displayed, it can be determined whether or not the search result is intended. In other words, if the search metadata confirmation unit 17 is not provided, it is possible to determine whether or not the search can be performed with the search condition intended by the user if the search that is necessary for a long time is not performed. Absent. As a result, it takes a long time to perform a search under a search condition intended by the user.

[Second Embodiment]
Next, a second embodiment of the present invention will be described in detail with reference to the drawings. In the second embodiment, a video search device that searches video data stored by itself using image data and keywords will be described. In the present embodiment, a video search device capable of searching video data showing a desired person will be described. In addition, the video search apparatus according to the present embodiment has a part of the same configuration as the video search apparatus described in the first embodiment. Therefore, in the present embodiment, portions different from the first embodiment will be mainly described.

Referring to FIG. 6, the video search device 2 according to the second embodiment of the present invention includes a video data receiving unit 21, a video data processing unit 22, a metadata extraction unit 23, a metadata storage unit 24, The video data storage unit 25, the search condition input unit 26, the search processing unit 27, and the search result display unit 28 are included. The search condition input unit 26 includes a search condition input item unit 31 (part of which is an image data input unit), a search metadata extraction unit 32, and a search condition comparison unit 33 (search metadata check unit). And a search condition display unit 34 (a part of the search metadata confirmation unit).

The video search device 2 in the present embodiment is an information processing device including an arithmetic device and a storage device. The video search device 2 realizes the above functions by causing the arithmetic device to execute a program stored in the storage device.

The video data receiving unit 21 has a function of receiving video data from an external device such as an IP camera. The video data receiving unit 21 receives video data from one external device or a plurality of external devices, for example, via an external network. Then, the video data receiving unit 21 transmits the received video data to the video data processing unit 22.

The video data processing unit 22 has a function of performing processing necessary for extracting metadata from video data received from the video data receiving unit 21 from the video data. For example, the video data processing unit 22 performs AD conversion (Analog / Digital) on the received video data. In addition, the video data processing unit 22 can perform various other processes necessary for processing video data. Then, the video data processing unit 22 transmits the video data subjected to necessary processing to the metadata extraction unit 23 and the video data storage unit 25.

The metadata extraction unit 23 has a function of extracting a predetermined type of metadata from the video data. The metadata extraction unit 23 receives metadata transmitted by the video data processing unit 22. Subsequently, the metadata extraction unit 23 extracts a predetermined type of metadata from the received video data. Thereafter, the metadata extraction unit 23 transmits the extracted metadata to the metadata storage unit 24.

Here, an example of a specific configuration for extracting metadata from video data will be described. Referring to FIG. 7, the metadata extraction unit 23 has a function including an analysis engine common base 41, an action detection ENG (ENGINE) 42, a face identification ENG 43, an age and gender identification ENG 44, and a clothing color identification ENG 45. provide.

The analysis engine common base 41 has a function as a base common to an action detection ENG 42, a face identification ENG 43, an age and gender identification ENG 44, and a clothing color identification ENG 45, which will be described later. The analysis engine common base 41 receives video data from the video data processing unit 22. The analysis engine common base 41 transmits the metadata extracted by each engine to the metadata storage unit 24.

The behavior detection ENG 42 has a function of detecting a moving person from the video data. Specifically, the behavior detection ENG 42 extracts image data one by one from the video data and takes a difference between successive image data. Subsequently, the behavior detection ENG 42 extracts a person who is a moving object from the difference between the image data. Then, the behavior detection ENG 42 detects the extracted face image and clothes image of the person using techniques such as pattern recognition and area division. Thereafter, the behavior detection ENG 42 passes the detected face image to the face identification ENG 43 and the age / sex identification ENG 44. In addition, the behavior detection ENG 42 passes the detected clothing image to the clothing color identification ENG 45.

The face identification ENG 43 has a function of using the face image passed from the behavior detection ENG 42 to extract the feature of the person's face reflected in the face image as metadata. The facial feature extraction by the face identification ENG 43 may be configured to extract a prominent feature from the facial image, or may be configured to extract the entire facial feature using the facial image itself as one pattern. Absent. Further, the face identification ENG 43 may be configured to identify an individual from the extracted facial features and extract, for example, the name of the identified person stored in a person information storage unit (not shown) as metadata. I can do it.

Age gender identification ENG44 has a function of using the face image passed from action detection ENG42 to extract the estimated age and gender data of the person reflected in the face image. It is conceivable that the age and sex data are extracted from, for example, facial shape features such as eyebrows and facial features that show wrinkles.

The clothing color identification ENG45 has a function of extracting the clothing color data of the upper body and the clothing color data of the lower body, which are reflected by the clothing image, using the clothing image passed from the behavior detection ENG42. For example, the clothing color identification ENG 45 acquires the clothing color data of the upper body and the clothing color data of the lower body as specific numerical values indicating the clothing color of the upper body and the clothing color of the lower body.

In this way, the movement detection ENG 42 detects a moving person from the video data. Then, the metadata of the person detected by the behavior detection ENG42 is extracted by the face identification ENG43, the age / sex identification ENG44, and the clothing color identification ENG45. With this configuration, the metadata extraction unit 23 can extract predetermined types of metadata, which is metadata for identifying video data and a person in the video data, from the video data. Note that the metadata extracted by the metadata extraction unit 23 includes information for identifying the moving image from which the metadata is extracted. Details of the predetermined metadata types have already been described and will not be described.

The metadata storage unit 24 is configured by a storage device such as a hard disk or a RAM (Random Access Memory), and receives and stores metadata transmitted by the metadata extraction unit 23.

The video data storage unit 25 is configured by a storage device such as a hard disk or a RAM (Random Access Memory), and receives and stores video data transmitted by the video data processing unit 22.

As described above, the metadata stored in the metadata storage unit 24 includes information for identifying the video data from which the metadata is extracted (stored in the video data storage unit 25). Therefore, metadata associated with video data stored in the video data storage unit 25 is stored in the metadata storage unit 24.

The search condition input unit 26 has a function of inputting image data and keywords for searching video data stored in the video search device 2. In addition, the search condition input unit 26 compares the search metadata extracted from the input image data with the search metadata by the keyword, and uses the search metadata as pre-search condition information before executing the search. It has a function to display. The search condition input unit 26 may be configured to be able to modify the search metadata displayed as pre-search condition information in accordance with an input from the user. As described above, the search condition input unit 26 includes the search condition input item unit 31, the search metadata extraction unit 32, the search condition comparison unit 33, and the search condition display unit 34.

The search condition input item unit 31 has a function of inputting a search condition for searching for video data stored in the video search device 2 (the video data storage unit 25). Referring to FIG. 8, the search condition input item unit 31 includes an image data input unit 51 and a keyword input unit 52.

The image data input unit 51 has a function of inputting image data as a search condition. The image data input to the image data input unit 51 is transmitted to the search metadata extraction unit 32. For example, a plurality of images may be input to the image data input unit 51, such as inputting three pieces of image data of the face, upper body, and lower body. In addition, the image data input unit 51 may input, for example, a photograph of the entire person showing the face, the upper body, and the lower body. In this case, the face, upper body, and lower body are identified by the search metadata extraction unit 32 described later. Note that the detailed description of the configuration of the image data input unit 51 is the same as in the first embodiment, and is omitted.

The keyword input unit 52 has a function of inputting keywords. The keyword input unit 52 includes an input device such as a keyboard and a touch panel, and is configured to be able to input a keyword by operating the input device. As will be described later, the keyword input to the keyword input unit 52 is to be compared with metadata extracted from image data as search metadata by the keyword. The keyword input unit 52 may input a keyword for the purpose of comparison with the search metadata extracted from the image data. For example, a search to be performed regardless of the search metadata extracted from the image data. Metadata may be input as a keyword. The keyword input unit 52 transmits the keyword input to the keyword input unit 52 to the search condition comparison unit 33.

The search metadata extraction unit 32 has a function of extracting search metadata from image data. The search metadata extraction unit 32 receives the image data transmitted from the image data input unit 51. Then, the search metadata extraction unit 32 extracts a predetermined type of search metadata from the received image data. Thereafter, the search metadata extraction unit 32 transmits the extracted search metadata to the search condition comparison unit 33.

Here, an example of a specific configuration for extracting search metadata from image data will be described. Referring to FIG. 9, the search metadata extraction unit 32 provides a function including an analysis engine common base 41, an action detection ENG 42, a face identification ENG 43, an age / sex identification ENG 44, and a clothing color identification ENG 45. . That is, the search metadata extraction unit 32 has the same configuration as the metadata extraction unit 23. Thus, the metadata extraction unit 23 and the search metadata extraction unit 32 have the same configuration, so that the metadata extraction unit 23 and the search metadata extraction unit 32 extract the same type of metadata. It becomes possible to do. As a result, the metadata extracted by the metadata extraction unit 23 can be searched using the search metadata extracted by the search metadata extraction unit 32. Note that the search metadata extraction unit 32 may not necessarily have the same configuration as long as it can extract the same type of search metadata as the metadata extracted by the metadata extraction unit 23. .

The search condition comparison unit 33 has a function of comparing search metadata extracted from image data with search metadata based on keywords. As described above, a predetermined type of search metadata is extracted from the image data. Therefore, the search condition comparison unit 33 compares the search metadata extracted from the image data and the search metadata using keywords for each predetermined type. Then, the search condition comparison unit 33 transmits the comparison result to the search condition display unit 34.

The search condition display unit 34 has a function of displaying the comparison result transmitted from the search condition comparison unit 33 on an external device such as a monitor. The display of the comparison result by the search condition display unit 34 is performed before the search for the metadata storage unit 24 using the search metadata.

Here, a specific example of the display of the comparison result displayed by the search condition display unit 34 will be described. Referring to FIG. 10, the search condition display unit 34 displays, for example, an image illustration display, image metadata information, and keyword metadata information. Here, the metadata information of the image is search metadata extracted by the search metadata extraction unit 32 from the image data input to the image data input unit 51. In the image metadata information, for example, the presence / absence of a facial feature indicated by the image data, age, gender, upper body color, and lower body color are displayed. The image illustration display is a simple display corresponding to the image data input to the image data input unit 51. The facial features, the upper body (torso and arm) clothing color, and the lower body clothing color are displayed. This is a simple illustration. The keyword metadata information is search metadata based on keywords input to the keyword input unit 52. In the keyword metadata information, keywords input to the keyword input unit 52, for example, age, sex, clothes color, and the like are displayed. In the search condition display section 34, such an image illustration display, image metadata information, and keyword metadata information can be compared horizontally, for example, in three categories of face, upper body, and lower body. They will be displayed side by side.

Referring to FIG. 11, the search condition display unit 34 displays a display of comparison information in addition to, for example, an image illustration display, image metadata information, and keyword metadata information. It can be configured as follows. Here, in the display of the comparison information, information that can be obtained by comparing the metadata information of the image and the metadata information of the keyword is displayed. The display of the comparison information includes, for example, the presence / absence of facial features and the difference between the metadata information of the image and the metadata information of the keyword. Further, the search condition comparison unit 34 may be configured to display the matching point between the image metadata information and the keyword metadata information for displaying the comparison information. As described above, the search condition display unit 34 can be configured to compare the metadata information of the image and the metadata information of the keyword by the same type and display the result of the comparison.

Referring to FIG. 12, the search condition display unit 34 displays a reference illustration display in addition to, for example, an image illustration display, image metadata information, and keyword metadata information. It can be configured as follows. Here, in the reference illustration display, an illustration showing image data necessary for extracting the search metadata by the keyword from the image data is displayed for reference. As described above, the search condition display unit 34 can be configured so that the image data necessary for inputting the metadata information of the keyword from the image data is represented by an illustration for reference.

Also, the search condition display unit 34 displays the image metadata information and the keyword metadata information so that the matching points and the difference points are emphasized when the same type is compared. I can do it. Specifically, it may be possible to display different portions of the search metadata indicated by the metadata information of the image and the search metadata indicated by the keyword metadata information in red and bold.

Thus, the search condition display unit 34 displays the result of the comparison by various methods. The search condition display unit 34 is not limited to the above case, and can adopt an expression method in which the user can easily recognize the comparison result.

Further, the search condition display unit 34 is configured to be able to select whether or not to search the metadata storage unit 24 using the search metadata according to the displayed pre-search condition information. . Further, the search condition display unit 34 can be configured to be able to correct the search metadata displayed as the pre-search condition information in accordance with the input from the user. A detailed description of this configuration of the search condition display unit 34 is the same as that already described, and will be omitted.

Thus, the search condition display unit 34 displays search metadata as pre-search condition information on an output device such as a monitor. In addition, the search condition display unit 34 accepts correction of the displayed search metadata. Thereafter, when execution of search is selected by the user, the search condition display unit 34 searches the search metadata (search metadata displayed on the output device or the like) instructed to execute the search. 27. On the other hand, when the user selects to cancel the search, the search condition display unit 34 discards, for example, the search metadata for which the search is canceled. Then, it waits for input of new image data (reception of search metadata).

In the present embodiment, the case has been described where the pre-search condition information is displayed in three classifications: face, upper body, and lower body. However, the present invention can be implemented without being limited to the above case. For example, the display method or the comparison method may be changed as appropriate, such as adding shoe information or hand information, or displaying the upper and lower body at a time.

Further, the search condition display unit 34 is configured to determine whether or not the search metadata input from the keyword input unit 52 matches the search metadata input from the search metadata extraction unit 32. I can do it. Further, the search condition display unit 34 determines whether the search metadata input from the keyword input unit 52 matches the search metadata extracted by the search metadata extraction unit 32 as pre-search condition information. It can be configured to change the information displayed. For example, when the search metadata does not match, the search condition display unit 34 displays the search metadata and search availability options as pre-search condition information. On the other hand, when the search metadata matches, the search condition display unit 34 displays only the search metadata as the pre-search condition information. By configuring in this way, the search condition display unit 34 is configured to search before search when the search metadata input from the keyword input unit 52 matches the search metadata input from the search metadata extraction unit 32. After displaying the search metadata as the condition information, the search can be executed promptly.

The search processing unit 27 has a function of searching for metadata stored in the metadata storage unit 24 using the search metadata displayed as pre-search condition information. Referring to FIG. 13, the search processing unit 27 provides a function including a matching engine common base 61, an object matching ENG 62, a face matching ENG 63, an age-sex matching ENG 64, and a clothing color matching ENG 65.

The collation engine common base 61 of the search processing unit 27 receives the search metadata from the search condition display unit 34. The collation engine common base 61 receives metadata stored in the metadata storage unit 24 from the metadata storage unit 24. Subsequently, the face collation ENG 63, the age and gender collation ENG 64, and the clothing color collation ENG 65 respectively determine the sameness between the same type of metadata and the search metadata. After the determination of the identity between the same type of metadata and the search metadata, the object collation ENG 62 determines whether the metadata and the search metadata are the same. Judgment is made. Thereafter, the collation engine common base 61 transmits the search result to the search result accumulation unit 66.

Then, the search processing unit 27 stores the search result in the search result storage unit 66. Thereafter, the search result (metadata) stored in the search result storage unit 66 is transmitted to the search result display unit 28, and the search result display unit 28 displays video data related to the metadata. . Since the flow of displaying the video data by the search result display unit 28 is the same as that already described, a description thereof will be omitted.

Note that the search processing unit 27 can be configured to perform a search even when the search metadata extracted from the image data is different from the search metadata by the keyword. That is, for example, when the search execution is selected in a state where the search metadata extracted from the image data is different from the search metadata by the keyword, the search processing unit 27 uses each search metadata. Thus, the metadata storage unit 24 can be configured to perform a search. In this case, the search metadata extracted from the image data and the search metadata based on the keyword may be searched as an AND condition or may be searched as an OR condition. In the above case, the search processing unit 27 can be configured to perform a search on the metadata storage unit 24 using either one of the predetermined search metadata. The search processing unit 27 may be configured to be able to select the various search methods described above.

The above is the configuration of the video search device 2. By providing the video search device 2 with the above configuration, the video search device 2 uses the search metadata and keywords extracted from the image data before searching the metadata storage unit 13 using the search metadata. Search metadata can be displayed as pre-search condition information. As a result, it can be confirmed more reliably whether or not the pre-search condition information is intended by the user.

In the present embodiment, the video search device 2 includes the metadata storage unit 24 and the video data storage unit 25 separately. However, the video search device 2 may be configured to include one storage device that stores metadata and video data.

Further, in this embodiment, the analysis engine common base 41 includes four engines. However, the present invention is not limited to the case where the analysis engine common base 41 includes four engines. The number of engines on the analysis engine common base 41 may be increased or decreased according to a predetermined type of metadata extracted from the image data. Similarly, the number of engines on the collation engine common base 61 may be increased or decreased according to the number of metadata to be searched.

Next, the operation of the video search apparatus 2 according to this embodiment will be described. In the present embodiment, operations that are particularly characteristic in the present embodiment will be described.

First, an operation when the metadata extraction unit 23 receives video data from the video data processing unit 22 and extracts metadata will be described.

When the metadata extraction unit 23 receives video data from the video data processing unit 22, execution of the processing shown in FIG. 14 is started. The analysis engine common base 41 of the metadata extraction unit 23 receives the video data. Next, the action detection ENG 42 detects the moving person by extracting the image data from the video data and taking the difference between the continuous image data (S301). Then, the action detection ENG 42 passes the face image of the moving person to the face identification ENG 43 and the age and gender identification ENG 44. In addition, the behavior detection ENG 42 passes the clothing image of the moving person to the clothing color identification ENG 45.

The face identification ENG 43 that has received the face image uses the face image passed from the behavior detection ENG 42 to extract the facial features of the person reflected by the face image as metadata (S302). The extraction of facial features by the face identification ENG 43 may be configured to extract features that stand out from the facial image, or may be configured to extract facial features using the facial image itself as one pattern. Further, the face identification ENG 43 may identify an individual from the extracted facial features and extract the name of the identified person as metadata.

Also, the age and gender identification ENG 44 that has received the face image uses the face image passed from the behavior detection ENG 42 to extract the estimated age and gender data of the person shown by the face image (S303). Age and gender data are extracted from, for example, facial shape features such as eyebrows and facial shading features.

Also, the clothing color identification ENG 45 that has received the clothing image extracts the clothing color data of the upper body and the clothing color data of the lower body using the clothing image passed from the behavior detection ENG 42 (S304).

In this way, the movement detection ENG 42 detects a moving person, and extracts a predetermined type of metadata of the person detected by the face identification ENG 43, the age and gender identification ENG 44, and the clothing color identification ENG 45. The metadata extracted by each engine is transmitted from the analysis engine common base 41 to the metadata storage unit 24 and stored in the metadata storage unit 24. Note that the metadata extracted by the metadata extraction unit 23 includes information for identifying the moving image from which the metadata is extracted.

The above is the operation when the metadata extraction unit 23 receives video data from the video data processing unit 22 and extracts metadata. The operation when the search metadata extraction unit 32 extracts the search metadata from the image data is the same as the operation of the metadata extraction unit 23 described above. Next, an operation when searching for video data stored in the video search device 2 will be described.

Referring to FIG. 15, image data is input to the image data input unit 51 (S401). Then, the image data input unit 51 that has received the image data transmits the input image data to the search metadata extraction unit 32.

Next, the search metadata extraction unit 32 receives the image data transmitted by the image data input unit 51. Then, the search metadata extraction unit 32 extracts a predetermined type of search metadata from the received image data (S402). Thereafter, the search metadata extraction unit 32 transmits the extracted search metadata to the search condition comparison unit 33.

Also, a keyword is input to the keyword input unit 52 simultaneously with the input of image data to the image data input unit 51 or before or after the input of the image data (S403). Then, the keyword input unit 52 to which the keyword has been input transmits the input keyword (search metadata by keyword) to the search condition comparison unit 33.

By the above operation, the search condition comparison unit 33 is transmitted with two search metadata, that is, search metadata extracted from the image data and search metadata by keywords. Therefore, the search condition comparison unit 33 compares the two types of search metadata, that is, the search metadata extracted from the received image data and the search metadata based on the keywords, with the same type (S404). Then, the search condition comparison unit 33 transmits the comparison result to the search condition display unit 34.

Subsequently, the search condition display unit 34 receives the result of comparison by the search condition comparison unit 33. Then, the search condition display unit 34 uses the search metadata to search the metadata for search extracted from the image data and the search metadata by keywords before performing a search for the metadata storage unit 24. Information is displayed on an output device such as a monitor (S405). Then, the search condition comparison unit 33 confirms whether or not to perform a search using the search metadata expressed as the displayed pre-search condition information (S406).

Note that when the search condition comparison unit 33 displays the search metadata extracted from the image data and the search metadata based on keywords as pre-search condition information, the search condition comparison unit 33 has various information so that the user can easily confirm the comparison result. You can take an expression method.

When it is determined that the user does not perform the search using the pre-search condition information displayed on the output device (S406, NO), the user does not perform the search using the pre-search condition information displayed on the output device. The search condition comparison unit 33 is instructed. In this case, for example, the user replaces the image data and starts again from the input of the image data. The operation here is not limited to the case of replacing image data. For example, the search metadata may be corrected using a keyboard or the like connected to the video search device 2.

On the other hand, when it is determined that the user performs a search using the pre-search information displayed on the output device (S406, YES), the search condition comparison unit 33 performs the search indicated by the pre-search condition information displayed on the output device. The metadata is transmitted to the search processing unit 27. Then, the search processing unit 27 that has received the search metadata searches for metadata stored in the metadata storage unit 24 using the search metadata (S407).

Thereafter, the search processing unit 27 stores the metadata searched from the metadata storage unit 24 using the search metadata in the search result storage unit 66. In addition, the search processing unit 27 transmits the searched metadata to the search result display unit 28. Then, the search result display unit 28 that has received the metadata transmitted by the search processing unit 27 acquires the video data related to the received metadata from the video data storage unit 25 and displays the acquired video data ( S408).

By such an operation, the video search device 2 can search the video data stored by itself using the input image data and keywords.

As described above, the video search device 2 in this embodiment includes the search condition display unit 34. With such a configuration, the video search apparatus 2 can display the search metadata as pre-search condition information before performing a search on the metadata storage unit 24 using the search metadata. As a result, it is possible to check the search metadata as pre-search condition information before searching for metadata using search metadata that requires time, and the search intended by the user in a short time. It becomes possible to search by condition.

Also, the video search device 2 in this embodiment includes an image data input unit 51 and a keyword input unit 52. With this configuration, it is possible to input a keyword when inputting image data. As a result, the search metadata extracted from the image data by the video search device 2 and the search metadata by the keyword input by the user can be displayed side by side, and the search metadata extracted from the image data and the user's Comparison with the intended search condition (search metadata by keyword) can be performed reliably.

Further, the video search device 2 in the present embodiment includes a search condition comparison unit 33. With such a configuration, the video search device 2 compares the search metadata extracted from the image data with the search metadata by keywords, and displays the comparison result on the search condition display unit 34. It becomes possible. As a result, it becomes possible to clearly grasp the difference between the search metadata extracted from the image data and the search metadata based on the keyword, and the search can be performed more reliably with the search condition intended by the user. .

In the present embodiment, the video search apparatus capable of searching video data showing a desired person has been described. However, the present invention can be practiced without being limited to a video search apparatus that searches for a person. The present invention can be applied to a video search apparatus that searches video data using metadata extracted from image data.

Next, a third embodiment of the present invention will be described with reference to the drawings. In the third embodiment, a video search apparatus that searches video data stored by itself using image data will be described. In this embodiment, an outline of the video search device will be described.
[Third embodiment]
Referring to FIG. 16, a video search device 7 according to the third embodiment of the present invention includes a metadata extraction unit 71, a metadata storage unit 72, an image data input unit 73, and a search metadata extraction unit 74. And a search metadata confirmation unit 75.

The metadata extraction unit 71 has a function of extracting a predetermined type of metadata from video data input from the outside. Here, the predetermined type of metadata is metadata for identifying video data and a person in the video data. As metadata extracted by the metadata extraction unit 71, for example, the facial features, age, gender, clothing color of the upper body and lower body of a person, and the like can be considered. Further, the metadata extraction unit 71 can be configured to extract, for example, the date and time of shooting of an image, the shooting location, the title and comment set by the user, the name of a person, and the like as metadata. In addition to the above description, the metadata extraction unit 71 can extract various types of metadata for identifying video data and persons. Note that the metadata extraction unit 71 extracts metadata so that the metadata to be extracted includes information for identifying video data from which the metadata is extracted.

In this way, the metadata extraction unit 71 extracts a predetermined type of metadata from video data acquired from the outside. Then, the metadata extraction unit 71 transmits the extracted metadata to the metadata storage unit 72.

The metadata storage unit 72 includes a storage device such as a hard disk or a RAM (Random Access Memory). The metadata storage unit 72 receives the metadata transmitted by the metadata extraction unit 71. Then, the metadata storage unit 72 stores the received metadata. Note that the metadata stored in the metadata storage unit 72 includes information for identifying video data from which the metadata is extracted. That is, the metadata storage unit 72 stores the metadata in association with the source video data.

The image data input unit 73 has a function of inputting image data. The image data input unit 73 receives image data used when searching the video search device 7.

The search metadata extraction unit 74 has a function of extracting a predetermined type of search metadata from the image data input by the image data input unit 73. The search metadata extraction unit 74 receives the image data transmitted from the image data input unit 73. The search metadata extraction unit 74 then extracts a predetermined type of search metadata from the received image data. Here, the predetermined type of search metadata refers to the same type of metadata as the metadata extracted by the metadata extraction unit 71 described above. That is, the metadata extraction unit 71 and the search metadata extraction unit 74 are configured to be able to extract the same type of metadata. Then, the search metadata extraction unit 74 transmits the extracted search metadata to the search metadata confirmation unit 75.

The search metadata confirmation unit 75 searches the search metadata and searchable options before performing a search on the metadata storage unit 72 using the search metadata extracted by the search metadata extraction unit 74. It has a function to display as precondition information. The search metadata confirmation unit 75 receives the search metadata transmitted by the search metadata extraction unit 74. Then, the search metadata confirmation unit 75 displays the received search metadata and search availability options on, for example, an output device such as a monitor connected to the video search device 7. Thereafter, in response to the display of the search metadata by the search metadata confirmation unit 75, it is determined whether or not to perform a search using the search metadata, and an actual search is performed.

The above is the outline of the configuration of the video search device 7.

Also, the video search device 7 described above can be realized by incorporating a predetermined program into the video search device 7. Specifically, a program according to another aspect of the present invention includes a metadata extraction unit that extracts a predetermined type of metadata from video data input from the outside, and a metadata extraction unit. A metadata storage unit that stores the extracted metadata in association with the source video data, an image data input unit that inputs image data, and a predetermined type of search from image data input by the image data input unit Search metadata extraction unit that extracts the metadata for search, and the search metadata is searched before using the search metadata extracted by the search metadata extraction unit before searching the metadata storage unit. This is a program for causing a search metadata confirmation unit to be displayed as information.

The video search method executed by the operation of the video search device 7 described above extracts metadata of a predetermined type from externally input video data, and extracts the metadata from the source video data. And storing image data, inputting image data, extracting search metadata of a predetermined type from the input image data, and extracting metadata extracted from video data using the search metadata This is a method of displaying search metadata as pre-search condition information before performing a search.

Even in the invention of the program or the video search method having the above-described configuration, the above-described object of the present invention can be achieved because it has the same operation as the video search device 7 described above.

Next, a fourth embodiment of the present invention will be described with reference to the drawings. In the fourth embodiment, a metadata confirmation apparatus 8 that searches video data using image data will be described. In the present embodiment, an outline of the metadata confirmation apparatus 8 will be described.
[Fourth Embodiment]
Referring to FIG. 17, the metadata confirmation apparatus 8 in the present embodiment includes an image data input unit 81, a search metadata extraction unit 82, and a search metadata confirmation unit 83.

The image data input unit 81 has a function of inputting image data. The image data input unit 81 inputs image data used for searching.

The search metadata extraction unit 82 has a function of extracting a predetermined type of search metadata from the image data input by the image data input unit 81. The search metadata extraction unit 82 receives the image data transmitted from the image data input unit 81. Then, the search metadata extraction unit 82 extracts search metadata of a predetermined type from the received image data. Then, the search metadata extraction unit 82 transmits the extracted search metadata to the search metadata confirmation unit 83. Note that the metadata check device 8 is configured to be able to search, for example, video data stored in an external device using search metadata extracted by the search metadata extraction unit 82. Therefore, the metadata extraction unit 82 extracts search metadata so that video data can be searched from an external device.

The search metadata confirmation unit 83 has a function of displaying search metadata and search availability options as pre-search condition information. The search metadata confirmation unit 83 receives the search metadata transmitted by the search metadata extraction unit 82. Then, the search metadata check unit 83 displays the received search metadata and search availability options on, for example, an output device such as a monitor connected to the metadata check device 8. Thereafter, in response to the display of the search metadata by the search metadata confirmation unit 83, it is determined whether or not to perform a search using the search metadata, and an actual search is performed.

<Appendix>
Part or all of the above-described embodiment can be described as in the following supplementary notes. The outline of the video search apparatus and the like in the present invention will be described below. However, the present invention is not limited to the following configuration.

(Appendix 1)
An image data input unit for inputting image data;
A search metadata extraction unit for extracting a predetermined type of search metadata from the image data input by the image data input unit;
A search metadata confirmation unit for displaying the search metadata and search availability options as pre-search condition information;
A metadata confirmation apparatus comprising:

(Appendix 2)
It has a keyword input part to input search metadata by keyword,
The search metadata confirmation unit displays the search metadata extracted by the search metadata extraction unit and the search metadata input from the keyword input unit as the pre-search condition information. Metadata confirmation device.

(Appendix 3)
The search metadata confirmation unit compares the search metadata extracted by the search metadata extraction unit with the search metadata input from the keyword input unit, and displays the comparison result. The metadata confirmation apparatus according to attachment 2.

(Appendix 4)
When the search metadata input unit displays the search metadata input to the keyword input unit, the search metadata extraction unit extracts the search metadata input to the keyword input unit. The metadata confirmation apparatus according to appendix 2 or 3, wherein image data required in some cases is displayed as a reference image.

(Appendix 5)
The search metadata confirmation unit determines whether the search metadata input from the keyword input unit matches the search metadata extracted by the search metadata extraction unit, and from the keyword input unit When the input search metadata and the search metadata extracted by the search metadata extraction unit do not match, the search metadata and the search availability option are displayed as the pre-search condition information. When the search metadata input from the keyword input unit matches the search metadata extracted by the search metadata extraction unit, the search metadata is displayed as the pre-search condition information. The metadata confirmation apparatus according to any one of appendices 2 to 4.

(Appendix 6)
The metadata check device according to any one of appendices 1 to 5, wherein the search metadata check unit corrects the search metadata displayed as the pre-search condition information in accordance with an input from a user.

(Appendix 7)
The metadata confirmation apparatus according to any one of appendices 1 to 5, wherein the predetermined type of metadata is at least one of age, sex, and clothing color.

(Appendix 8)
A metadata extraction unit that extracts predetermined types of metadata from video data input from the outside;
A metadata storage unit that stores the metadata extracted by the metadata extraction unit in association with the video data of the extraction source;
An image data input unit for inputting image data;
A search metadata extraction unit for extracting a predetermined type of search metadata from the image data input by the image data input unit;
A search processing unit for searching for the metadata related to the search metadata based on the search metadata and the metadata;
A search metadata confirmation unit for displaying the search metadata and search availability options as pre-search condition information;
A video search device comprising:

According to this configuration, the video search device includes the search metadata confirmation unit. With this configuration, the video search apparatus can display the search metadata as pre-search condition information before performing a search on the metadata storage unit using the search metadata. As a result, the search metadata can be confirmed as pre-search condition information before searching for metadata using the search metadata that requires time. As a result, it is possible to perform a search with a search condition intended by the user in a short time. In addition, since the search is performed after the search metadata is confirmed, the suspicion (for the device) with respect to the search result can be solved.

(Appendix 9)
It has a keyword input part to input search metadata by keyword,
The search metadata confirmation unit displays the search metadata extracted by the search metadata extraction unit and the search metadata input from the keyword input unit as the pre-search condition information. The video search device described in 1.

According to this configuration, the video search device includes a keyword input unit. With this configuration, it is possible to input a keyword when inputting image data. As a result, the search metadata extracted from the image data by the video search device and the search metadata by the keyword input by the user can be displayed side by side, and the search metadata extracted from the image data and the user's intention can be displayed. It is possible to reliably compare with the search condition (search metadata by keyword) to be performed.

(Appendix 10)
The search metadata confirmation unit compares the search metadata extracted by the search metadata extraction unit with the search metadata input from the keyword input unit between the same types, and compares the comparison results. The video search device according to appendix 9, which displays.

According to this configuration, the search metadata confirmation unit compares the search metadata extracted from the image data and the search metadata by the keyword with the same type, and displays the comparison result. With this configuration, the video search apparatus can display the search metadata as a result of the comparison after comparing the search metadata extracted from the image data with the search metadata based on the keyword. Become. As a result, it becomes possible to clearly grasp the difference between the search metadata extracted from the image data and the search metadata based on the keyword, and the search can be performed more reliably with the search condition intended by the user. .

(Appendix 11)
The search metadata confirmation unit displays an image required when the search metadata extraction unit extracts the search metadata when the search metadata input to the keyword input unit is displayed. The video search device according to appendix 9 or 10, wherein data is displayed as a reference image.

According to this configuration, the search metadata confirmation unit is required when the search metadata extraction unit extracts the search metadata when displaying the search metadata input to the keyword input unit. Display image data as a reference image. With this configuration, the reference image is displayed when the search metadata based on the keyword is displayed. As a result, it becomes possible to clearly grasp the difference between the search metadata extracted from the image data and the search metadata based on the keyword, and the search can be performed more reliably with the search condition intended by the user. .

(Appendix 12)
The search metadata confirmation unit determines whether the search metadata input from the keyword input unit matches the search metadata extracted by the search metadata extraction unit, and from the keyword input unit When the input search metadata and the search metadata extracted by the search metadata extraction unit do not match, the search metadata and the search availability option are displayed as the pre-search condition information. When the search metadata input from the keyword input unit matches the search metadata extracted by the search metadata extraction unit, the search metadata is displayed as the pre-search condition information. The video search device according to any one of appendices 9 to 11.

(Appendix 13)
The video search device according to any one of appendices 8 to 11, wherein the search metadata confirmation unit corrects the search metadata displayed as the pre-search condition information in accordance with an input from a user.

According to this configuration, the search metadata confirmation unit corrects the search metadata displayed as the pre-search condition information according to the input from the user. With such a configuration, when the pre-search confirmation information displayed by the search metadata confirmation unit is not what the user intended, the search metadata can be directly corrected. As a result, the search metadata can be corrected without re-inserting the image data, and the search can be performed with the search condition intended by the user in a shorter time.

(Appendix 14)
14. The video search device according to any one of appendices 8 to 13, wherein the predetermined type of metadata is at least one of age, sex, and clothing color.

(Appendix 15)
Extract metadata of a predetermined type from video data input from the outside, store the metadata in association with the video data of the extraction source,
Before inputting image data, extracting a predetermined type of search metadata from the input image data, and searching for metadata extracted from the video data using the search metadata A video search method for displaying the search metadata as pre-search condition information.

(Appendix 16)
Enter search metadata by keyword,
The video search method according to claim 15, wherein the search metadata extracted from the image data and the search metadata based on the keyword are displayed as the pre-search condition information.

(Appendix 17)
The video search method according to supplementary note 16, wherein the search metadata extracted from the image data and the search metadata by the keyword are compared with each other and the comparison result is displayed.

(Appendix 18)
Video search device
A metadata extraction unit that extracts predetermined types of metadata from video data input from the outside;
A metadata storage unit that stores the metadata extracted by the metadata extraction unit in association with the video data of the extraction source;
An image data input unit for inputting image data;
A search metadata extraction unit for extracting a predetermined type of search metadata from the image data input by the image data input unit;
A search metadata confirmation unit that displays the search metadata as pre-search condition information before performing a search on the metadata storage unit using the search metadata extracted by the search metadata extraction unit Program to function as.

(Appendix 19)
It functions as a keyword input part for inputting search metadata by keywords,
The search metadata confirmation unit displays the search metadata extracted by the search metadata extraction unit and the search metadata input from the keyword input unit as the pre-search condition information. The program described in.

(Appendix 20)
The search metadata confirmation unit compares the search metadata extracted by the search metadata extraction unit with the search metadata input from the keyword input unit between the same types, and compares the comparison results. The program according to supplementary note 19 to be displayed.

Note that the programs described in the above embodiments and supplementary notes are stored in a storage device or recorded on a computer-readable recording medium. For example, the recording medium is a portable medium such as a flexible disk, an optical disk, a magneto-optical disk, and a semiconductor memory.

Although the present invention has been described with reference to the above embodiments, the present invention is not limited to the above-described embodiments. Various changes that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

The present invention enjoys the benefit of the priority claim based on the patent application of Japanese Patent Application No. 2014-026466, which was filed on February 14, 2014 in Japan, and is described in the patent application. The contents are all included in this specification.

1, 2, 7 Video search device 11, 21 Video data receiving unit 12, 23, 71 Metadata extraction unit 13, 24, 72 Metadata storage unit 14, 25 Video data storage unit 15, 51, 73, 81 Image data input Units 16, 32, 74, 82 Search metadata extraction units 17, 75, 83 Search metadata confirmation units 18, 27 Search processing units 19, 28 Search result display unit 22 Video data processing unit 26 Search condition input unit 31 Search Condition input item part 33 Search condition comparison part 34 Search condition display part 41 Analysis engine common base 42 Action detection ENG
43 Face Identification ENG
44 Age Gender Identification ENG
45 Clothing color identification ENG
52 Keyword Input Unit 61 Collation Engine Common Base 62 Object Collation ENG
63 Face verification ENG
64 Age gender verification ENG
65 Clothing color matching ENG
66 Search Result Accumulation Unit 8 Metadata Confirmation Device

Claims (20)

An image data input unit for inputting image data;
A search metadata extraction unit for extracting a predetermined type of search metadata from the image data input by the image data input unit;
A search metadata confirmation unit for displaying the search metadata and search availability options as pre-search condition information;
A metadata confirmation apparatus comprising: It has a keyword input part to input search metadata by keyword,
The search metadata confirmation unit displays the search metadata extracted by the search metadata extraction unit and the search metadata input from the keyword input unit as the pre-search condition information. The metadata confirmation device described. The search metadata confirmation unit compares the search metadata extracted by the search metadata extraction unit with the search metadata input from the keyword input unit, and displays the comparison result. The metadata confirmation apparatus according to claim 2. When the search metadata input unit displays the search metadata input to the keyword input unit, the search metadata extraction unit extracts the search metadata input to the keyword input unit. The metadata confirmation apparatus according to claim 2 or 3, wherein image data required in such a case is displayed as a reference image. The search metadata confirmation unit determines whether the search metadata input from the keyword input unit matches the search metadata extracted by the search metadata extraction unit, and from the keyword input unit When the input search metadata and the search metadata extracted by the search metadata extraction unit do not match, the search metadata and the search availability option are displayed as the pre-search condition information. When the search metadata input from the keyword input unit matches the search metadata extracted by the search metadata extraction unit, the search metadata is displayed as the pre-search condition information. The metadata confirmation apparatus according to any one of claims 2 to 4. The metadata check device according to claim 1, wherein the search metadata check unit corrects the search metadata displayed as the pre-search condition information in accordance with an input from a user. The metadata confirmation apparatus according to any one of claims 1 to 6, wherein the predetermined type of metadata is at least one of age, sex, and clothing color. A metadata extraction unit that extracts predetermined types of metadata from video data input from the outside;
A metadata storage unit that stores the metadata extracted by the metadata extraction unit in association with the video data of the extraction source;
An image data input unit for inputting image data;
A search metadata extraction unit for extracting a predetermined type of search metadata from the image data input by the image data input unit;
A search processing unit for searching for the metadata related to the search metadata based on the search metadata and the metadata;
A search metadata confirmation unit for displaying the search metadata and search availability options as pre-search condition information;
A video search device comprising: It has a keyword input part to input search metadata by keyword,
The search metadata confirmation unit displays the search metadata extracted by the search metadata extraction unit and the search metadata input from the keyword input unit as the pre-search condition information. The video search device described. The search metadata confirmation unit compares the search metadata extracted by the search metadata extraction unit with the search metadata input from the keyword input unit, and displays the comparison result. The video search device according to claim 9. When the search metadata confirmation unit displays the search metadata input to the keyword input unit, the search metadata extraction unit extracts the search metadata input to the search keyword input unit. The video search device according to claim 9 or 10, wherein image data necessary for the display is displayed as a reference image. The search metadata confirmation unit determines whether the search metadata input from the keyword input unit matches the search metadata extracted by the search metadata extraction unit, and from the keyword input unit When the input search metadata and the search metadata extracted by the search metadata extraction unit do not match, the search metadata and the search availability option are displayed as the pre-search condition information. When the search metadata input from the keyword input unit matches the search metadata extracted by the search metadata extraction unit, the search metadata is displayed as the pre-search condition information. The video search device according to claim 9. The video search device according to claim 8, wherein the search metadata confirmation unit corrects the search metadata displayed as the pre-search condition information in accordance with an input from a user. 14. The video search device according to claim 8, wherein the predetermined type of metadata is at least one of age, sex, and clothing color. Extract metadata of a predetermined type from video data input from the outside, store the metadata in association with the video data of the extraction source,
A video search method for inputting image data, extracting a predetermined type of search metadata from the input image data, and displaying the search metadata and search availability options as pre-search condition information. Enter search metadata by keyword,
The video search method according to claim 15, wherein the search metadata extracted from the image data and the search metadata based on the keyword are displayed as the pre-search condition information. The video search method according to claim 16, wherein the search metadata extracted from the image data and the search metadata based on the keyword are compared with each other and the comparison result is displayed. Video search device
A metadata extraction unit that extracts predetermined types of metadata from video data input from the outside;
A metadata storage unit that stores the metadata extracted by the metadata extraction unit in association with the video data of the extraction source;
An image data input unit for inputting image data;
A search metadata extraction unit for extracting a predetermined type of search metadata from the image data input by the image data input unit;
A search processing unit for searching for the metadata related to the search metadata based on the search metadata and the metadata;
A program for functioning as a search metadata confirmation unit that displays the search metadata and search availability options as pre-search condition information. It functions as a keyword input part for inputting search metadata by keywords,
The search metadata confirmation unit displays the search metadata extracted by the search metadata extraction unit and the search metadata input from the keyword input unit as the pre-search condition information. 18. The program according to 18. The search metadata confirmation unit compares the search metadata extracted by the search metadata extraction unit with the search metadata input from the keyword input unit between the same types, and compares the comparison results. The program according to claim 19 to be displayed.

Images