JP2014092955A - Similar content search processing device, similar content search processing method and program - Google Patents

Abstract

The present invention provides a similar content search processing apparatus that enables a similarity determination that effectively matches a user's intention according to the content of the content held by the user.
A similar content search processing apparatus acquires a content group as a search target, inputs at least one content data as an inquiry source at the time of search, and acquires acquired content or attributes of input content Information is extracted, and difference information between the attribute information of the input content and the attribute information of the acquired at least one content group is calculated, and similarity between contents is calculated based on the calculated difference information A similarity criterion as a reference for calculating the degree is determined, a similarity between the contents is calculated based on the determined similarity criterion, and content satisfying a predetermined condition is calculated based on the calculated similarity select.
[Selection] Figure 1

Description

  The present invention relates to a content processing technique capable of searching for similar content in consideration of the content of content based on attribute information of a content group held by a user.

  In recent years, for example, DSC (Digital Still Camera), digital single-lens reflex cameras, digital movie cameras, mobile phone cameras such as smartphones, tablet terminal cameras, and the like have been widely used to capture subject images. Shooting is possible. In addition, the recording medium for storing content data has been increased in scale, and the server capacity of cloud services and the like has been expanded. In addition, with the development of social media, diverse people share personal content.

  For this reason, it is natural for an individual user to own or share an enormous amount of content, but the user searches for the content that the user needs when using it from the enormous amount of content that the user has. In order to enjoy and share only necessary content with users, it is necessary to organize a large amount of content by manually tagging it, which requires a lot of time and effort. There is a problem of doing. Also, there are increasing opportunities to advertise business using content data such as photos and videos, and to use content as menus and real lists when purchasing food, clothes, furniture, and the like. At this time, a similar problem occurs.

  Therefore, by collecting only content groups that look similar to the content specified by the user using the feature amount acquired from the content so that the user can efficiently find and use the desired content, similar content can be collected. Similar search processing techniques for searching are attracting attention.

  In addition, there is a technique for performing a similarity determination that more effectively matches a user's intention in a similar image search for a large number of content images (see, for example, Patent Document 1 and Patent Document 2).

JP 2012-58940 A JP 2011-203776 A

  However, in the above-described conventional technology, it is required to perform similarity determination that more effectively matches the user's intention.

  A main object of the present invention is to provide a similar content search processing apparatus that can perform similarity determination that fits the user's intention effectively in accordance with the content of the content held by the user.

  In order to solve the above-described conventional problem, a similar content search processing device according to the present invention includes a content data acquisition unit that acquires a content group that is a search target, and at least one content data that is an inquiry source at the time of search. The query information input means for inputting the content information, the content information extraction means for extracting the attribute information of the content acquired by the content data acquisition means or the content input by the query information input means, and the content information extraction means The difference information of the attribute information is calculated from the attribute information of the content input by the query information input unit and the attribute information of at least one content group acquired by the content data acquisition unit, and the calculated difference Calculate similarity between the contents based on information A similarity criterion determining unit that determines a similarity criterion that is a criterion for the determination, a similarity calculating unit that calculates a similarity between the contents based on the similarity criterion determined by the similarity criterion determining unit, Matching content selection means for selecting content satisfying a predetermined condition based on the similarity calculated by the similarity calculation means.

  This configuration makes it possible to calculate the degree of similarity according to the difference in attribute information of content data, even for a variety of content data. It is possible to perform a similar content search with an appropriate degree of similarity.

  According to the similar content search processing apparatus and method of the present invention, the content attribute information that is the search query and the content groups that are the search targets of the various content groups that the user holds and the content groups that are specific to the application. Since the process for determining the similarity to be calculated based on the attribute information is performed, a similar content search process for searching for content with a similarity suitable for the user's search purpose or search intention can be performed.

Block diagram of similar content search processing apparatus and method according to first to third embodiments of the present invention The flowchart which showed the procedure which performs the similar content search control process of Embodiment 1 of this invention The figure which shows an example of the apparatus metadata information in the content information extracted from a content group The figure which shows an example of the analysis metadata information in the content information extracted from a content group The figure which shows an example of the utilization metadata information in the content information extracted from a content group The block diagram which shows the function structure of the similarity standard determination means 4 of Embodiment 1 of this invention. The flowchart which showed the procedure which performs the similarity reference | standard determination process between the contents in Embodiment 1 of this invention The figure which shows an example of the difference information calculated in Embodiment 1 of this invention The flowchart which showed the procedure which performs the similarity reference | standard determination process between the contents based on the statistical characteristic item in Embodiment 2 of this invention The block diagram which shows the function structure of the attribute information comparison means 43 of Embodiment 2 of this invention. The flowchart which showed the procedure which performs the similarity reference | standard determination process between the contents for collation determination in Embodiment 3 of this invention The block diagram which shows the function structure of the reference | standard information determination means 44 of Embodiment 3 of this invention. Block diagram of similar content search processing apparatus and method according to Embodiment 4 of the present invention The flowchart which showed the procedure which performs the similar content search control process of Embodiment 4 of this invention. The figure which shows an example of the method of displaying the result of having switched and searched the one-dimensional similarity reference according to the change of the attribute information in Embodiment 4 of this invention. The figure which shows an example of the method of displaying the result of having searched and switched two-dimensional similarity reference according to the change of the attribute information in Embodiment 4 of this invention. The figure which shows an example of the display method at the time of searching a content by switching similarity criteria by the folder structure in Embodiment 4 of this invention

(Background to obtaining one embodiment of the present invention)
The inventor has studied in detail a technique for performing a similarity determination more effectively matching a user's intention in a similar image search for a large number of content images.

  First, a similar image search technique for a large number of content images will be described in detail.

  Normally, in the above-described conventional technology, the content of the search source and the search target are basically based on the color and shape of the entire image as the content feature amount, the assigned keyword, the feature amount specified by the user, and the like. A user can search for a desired content by browsing a content group having a high feature amount similarity to the content group. Thereby, it becomes possible for the user to perform a similar image search such as searching for an image of the same color according to a certain standard. However, when searching for content that the user wants to find from a group of unknown content by a similar search, it is more effective to search according to a certain standard, but if it has already been seen or taken and held or When the target content group to be searched for has a meaningful structure, it is possible to provide a similarity search method for content that is easy for a user to easily search or remember to calculate similarity based on a specific standard for all content. I can't say. In addition, unlike the use of searching from all images on the Web, in the use of searching from a specific menu or list, there are cases where it is difficult to search by a certain standard similarity determination.

  Next, in the conventional similar image retrieval technique, the conventional technique for the purpose of performing the similarity determination more appropriately according to the user's intention will be described in detail.

  First, the technology according to Patent Document 1 uses a plurality of determination criteria obtained by combining the similarity between an input image and each image in the DB by weighting a difference between a plurality of feature values. By extracting an image group satisfying a predetermined condition from each reference, it is possible to calculate the similarity with a feature value effective for the content desired by the user. Specifically, in order to realize a similar image search suitable for each user, a plurality of similarity criteria based on a combination of feature amounts are prepared, and a similar image that can be selected by the user while maintaining the similarity criteria By providing a search IF and determining a similarity criterion suitable for the user based on the content selected by the user, it is possible to efficiently narrow down to similar content desired by the user and present a search result. is there. However, since the technique according to Patent Document 1 basically searches for similar images based on content specified by the user, only images close to the content specified by the user can be searched, and only similarity search processing based on a certain standard can be performed. Have the problem of not.

  In addition, the technique according to Patent Document 2 also has a method of using meta-information other than the content feature amount, holds the image feature amount and the text feature amount based on the text related to the image for each content image, Along with the designation of the image by the user, the similarity of the text feature amount is integrated in addition to the similarity of the designated image, so that a target similar content image group can be found from a large number of image groups. is there. However, in the technique according to Patent Document 2, not only image feature amounts but also linguistically similar images are searched by using not only image feature amounts but also more reliable text information as feature amounts and using them together for similarity determination. However, in addition to the premise that text information is given, there is a problem that even in this case, only content that is similar as text information can be searched.

  The inventor examined the problems of the conventional technique for performing the similarity determination more effectively matching the user's intention as described above.

  In the above-described conventional technology for more effectively performing similarity determination that matches the user's intention, a method of converging to a certain standard or a certain standard on the assumption that unknown content is searched when performing a retrieval process for similar content In order to calculate the feature amount and determine the similarity, the processing is to search for content having a high similarity evenly with respect to all content. This is due to the fact that the similarity calculation processing is performed without considering the information on the content group unit structured in advance by the user or the content group having a certain common point. Therefore, in such a process, it cannot be said that an appropriate similarity determination process can be performed in accordance with the content content that the user has. As a result, a similar search process that searches for and presents similar content suitable for the user without considering the difference of various metadata for the content group owned by the user or the content group held for a specific use. The problem is that the method is insufficient.

  Through the above examination, the inventor can search the content group by searching for similarities based on a certain standard based on at least the feature amount calculated from content analysis and text information attached to the content. By using each attribute information and user operation information of the content group owned by the user or the content group for specific use and user operation information without performing the processing method, various users can satisfy various types of content The present inventors have found that similar content search can be performed from the viewpoint, and have reached the present invention.

  Embodiments of the present invention will be described below with reference to the drawings.

(Embodiment 1)
Embodiments of the present invention will be described below with reference to the drawings. In the first embodiment, content data input as a search query is constant as a data group search target group of images, videos, documents, music, etc., which are various content data possessed by a user or for specific applications. In a similar content search processing apparatus that performs search control processing for extracting and presenting content data having similarities, a similarity criterion suitable for each content group to be searched is obtained by using attribute information of each content data. The present invention relates to a search control processing mechanism for determining and calculating similarity based on the similarity criterion and extracting content data similar to each content group. FIG. 1 is a block diagram showing the basic configuration of a similar content search processing apparatus of the present invention. In FIG. 1, the similar content search processing apparatus includes a content data storage unit 1, a query information input unit 2, a content information extraction unit 3, a similarity criterion determination unit 4, a similarity calculation unit 5, and a matching content selection. It is comprised from the means 6.

  The content data storage unit 1 is a storage medium in which various content file data held by a user or for a specific use are stored. For example, photographic images, moving image data, documents, music data, etc. taken at an event or acquired by a business are stored. The storage medium is a large-capacity media disk such as an HDD or a DVD or a storage device such as a semiconductor memory.

  The query information input means 2 inputs content data selected and input by the user or selected by the system as query information which is search source data. For example, data of the same type as the data stored in the content data storage unit 1 is selected.

  The content information extraction means 3 includes device metadata information automatically given by content acquisition devices represented by mobile phones, smartphones, single-lens reflex cameras, DSCs, DVCs, etc., and analysis metadata given by content analysis methods Content data attribute information such as usage metadata information given from information, user input information, usage history, and the like is acquired.

  The device metadata information includes EXIF (Exchangeable Image File Format) information, moving image metadata, music metadata, and the like. Typical information that can be acquired is GPS (Global), which is date information and location information at the time of shooting. Positioning System) information and shooting methods include shooting mode information at the time of shooting, information such as parameters of a content acquisition device at the time of shooting, various sensor information, music feature information, and the like.

  The analysis metadata information obtained from the content analysis method includes the features extracted by image analysis, such as edges, colors, textures, etc., from the basic low-order feature quantities of the object, which are called object-specific local feature quantities. A high-order feature amount representing the shape can be calculated. A recognition result of a recognizable category such as a subject object such as a face, a human body or an object in an image, a specific landscape scene such as a seaside, a forest, or an indoor, a shooting composition, or the like is also conceivable. In addition, the document includes an image arrangement location, configuration information, and the like. In the case of a moving image, time-series motion and scene analysis information, and analysis information such as voice information and music melody can be calculated.

  As usage metadata information, there are event names and operation information directly entered by humans, personal names, photographer information, etc., usage history information such as viewing frequency for content, service usage such as shared service and processing service etc. History information and the like can be extracted.

  The similarity criterion determination unit 4 extracts content information for content data that is a search source input by the query information input unit 2 and at least one content unit of a certain unit acquired from the content data storage unit 1. The difference information is calculated using the attribute information extracted from the means 3, and a similarity criterion is determined as a criterion for calculating the similarity between contents based on the calculated difference information. The similarity criterion determination method uses the difference information of the attribute information between contents to extract common items and non-common items and their degree information, and uses them to determine meaningful similar items by comparing them Determine the quantity and how to calculate it. For example, if the shooting time zone is an item common to the contents to be compared and a landscape scene is a non-common item, the landscape scene is different, so the local feature that is robust to the background area and its calculation method are used for similarity. It can be determined as a standard.

  The similarity calculation unit 5 calculates the feature amount of the content based on the similarity criterion determined by the similarity criterion determination unit 4, and sequentially calculates the similarity between the contents. For example, when calculating the similarity of the entire image between target contents, the color feature amount, edge feature amount, and texture feature amount of the entire image are calculated as the global feature amount, and the similarity of the entire image is calculated by a pyramid matching method or the like. Is calculated. When calculating the local similarity of an image, a partial feature quantity that is partially invariant of the image is calculated as the local feature quantity, and the local similarity of the image is calculated by a geometric matching method or the like.

  Based on the similarity between the contents calculated by the similarity calculation means 5, the matched content selection means 6 selects and extracts the content that is the search query information and the similarity that satisfies a predetermined condition. For example, content having a similarity equal to or greater than the threshold Th is selected as content that matches the condition.

  In the following, when a similar search is performed based on similarity information between data with respect to content data such as photos, videos, documents, music, etc. that are held by a user or for a specific use, the content as query information that is the search source By extracting and using the difference between the attribute information of the data and the content data group to be searched as difference information, a similarity criterion according to the attribute difference between contents is adaptively determined and the similarity is calculated. With this process, the similarity between various content data can be calculated by taking into account the difference in attributes between the contents. Explained.

  When a user or system performs content similarity search, when the content data of the query information that is the search source is specified and entered for the search target content data group, the difference from the attribute information in the search target content data is reflected. Content similarity criterion determination and similar content search control processing are performed. When the similar content search control process is started, the attribute information of the content data that is the input query information is extracted, and the attribute information of the content group in a certain unit that can be compared from the content data group that is the search target is extracted. Extract and extract difference information between contents. Based on the extracted difference information, a similarity criterion suitable for an attribute difference between contents is determined. Then, the similarity between the contents is calculated based on the determined similarity criterion, and search control processing for selecting and determining similar contents satisfying a predetermined condition is performed. FIG. 2 is a flowchart showing a procedure for performing similar content search control processing after performing content similarity criterion determination processing based on query information and attribute information of content data to be searched.

  Input of content data as search query information to be a search source by the user or system is performed by the query information input means, and search control processing of similar content data is started (step S1). The user can input a search query by indirectly selecting content data displayed on the display using a mouse or keyboard that is an operating device, or content data displayed on the display using a touch panel or touch pad. There is a method to select directly. Furthermore, it is possible to select a search query by an operation based on gaze detection using a camera or a pressure-sensitive selection using a floating touch panel. An operation can be performed by combining these operations. In addition, content data that the user may be interested in may be input as an initial search query by the system.

  When content data as query information is input, content data to be searched is acquired from the content data storage unit 1, and content information extraction means 3 performs processing for extracting the content of the query information and attribute information for the content to be searched. Performed (step S2). The attribute information of the extracted content includes device metadata information, analysis metadata information, usage metadata information, and the like.

  An example of device metadata information is shown in FIG. The device metadata information is attribute information included in metadata provided by the content acquisition device. The metadata provided by the content acquisition device is, for example, music metadata such as EXIF (Exchangeable Image File Format) information, extended metadata for moving images, CDDB information, and ID3 tags. In the content attribute information included in the metadata, for example, an ID number is assigned to each content data, and a file name, a data type, and an acquisition time exist for each content. In the case of image and video data, ISO (International) that adjusts the shooting time information indicating the shooting time and the longitude and latitude information obtained from the GPS information as the geographical position information at the time of shooting and the brightness at the time of shooting. (Organization for Standardization) Sensitivity information, exposure information for adjusting the brightness so that it can be properly viewed, device parameter information such as the type of photographing device, photographing mode information of the photographing device, and the like exist. In the case of music data, information such as sound quality and genre exists.

  Next, an example of analysis metadata information is shown in FIG. Metadata obtained from content analysis techniques. As metadata obtained by image analysis, for example, high-dimensional feature quantities that can represent the characteristics of a subject object from low-dimensional feature quantities such as color information, edge information, and texture information that are basic feature quantity information of images Can be calculated. As the color information of the image, RGB color values are calculated as in-image statistical values, calculated as hue information converted into HSV or YUV color space, calculated as statistical information such as color histograms and color moments, etc. can do. As the edge information, the edge feature detected in the line segment in the image can be calculated as an in-image statistical value for each fixed angle. As the high-dimensional feature, a feature of a local region centered around a characteristic point or a feature representing the shape of an object is conceivable, and specifically, there are feature quantities such as SIFT, SURF, and HOG. It is possible to determine the degree of similarity between contents using these feature amounts and to detect the same image. In addition, the presence / absence and number of faces is calculated as face information obtained from face detection technology, etc. In addition, image recognition information related to people based on face size, clothing color and shape, person detection information, etc. It is also possible to use. In addition, it is also possible to use the result by image recognition technology represented by the general object recognition which performs vehicle detection, pet detection, such as a dog and a cat, and general object recognition. Furthermore, the region information of the specific subject or the background scene can be used as the region information extraction of the image by using a region information calculation method using a Salicity Map or a Depth Map.

  An example of usage metadata information is shown in FIG. The usage metadata information is metadata provided by use of content such as information provided by the user, usage history, service usage history, and the like. For example, an operation content for assigning a content shooting event name, a character in each attached content, tag information, or the like can be considered. In addition, the number of times played or searched on the display device, the frequency of use, the shared destination, the service used, or the contents of services using contents such as photo development, DVD packaging, digital albums, slideshows, etc. . Further, the attribute information is statistical data information that can be calculated from a combination of information in each shooting event unit, where an ID is allocated in each shooting event unit, where a plurality of acquired content data exists and a set of captured data is a shooting event. It is conceivable to leave it as used metadata information that can be directly input.

  When the content attribute information is extracted, the similarity criterion determination unit 4 uses the extracted query information and the attribute information of the search target content group to indicate how the similarity between the contents is calculated. A similarity criterion is determined (step S3). Here, an example of a functional configuration of the similarity criterion determination unit 4 will be described with reference to FIG.

  FIG. 6 is a functional block diagram of the similarity criterion determination means 4. The similarity criterion determination unit 4 includes a query attribute extraction unit 41, a target group attribute extraction unit 42, an attribute information comparison unit 43, and a reference information determination unit 44. The query attribute extraction unit 41 extracts only attribute information to be used from the extracted attribute information regarding the content that is the input query information. The target group attribute extracting unit 42 extracts only attribute information that can be compared with the attribute information extracted by the query attribute extracting unit 41. The attribute information comparison unit 43 extracts difference information that can be calculated by comparing the extracted attribute information in a format that can be calculated as a difference value. Based on the difference information calculated by the attribute information comparison unit 43, the reference information determination unit 44 calculates and determines the feature amount and the calculation method to be calculated as the similarity between contents as the similarity criterion.

  The basic processing contents for determining the similarity criterion for calculating the similarity between the content that is the query information and the content group to be searched will be specifically described with reference to FIG. FIG. 7 is a flowchart illustrating a procedure for performing similarity criterion determination processing for calculating similarity between contents based on attribute information of content that is query information and attribute information of a content group that is a search target.

  When the content of the query information that is the search source and the attribute information thereof are input, the attribute information as the comparison content used when the query attribute extraction unit 41 calculates the difference information is extracted (step S11). The attribute information of the content that is the query information is at least one or more comparable metadata information among the attribute information extracted by the content information extraction unit 3. There are cases where the content is extracted from attribute information of the content itself and cases where the content is extracted from a content group to which the content belongs. The affiliated content group is a content group in which a plurality of contents can be regarded as one set based on attribute information. For example, content groups sorted into event folders created by the user, content groups for one day, content groups shot at the same place within a certain range, or the same shooting scene and shooting composition It may be a content group that can be determined.

  Then, when the attribute information is input from the content group to be searched, the target group attribute extracting means 42 extracts the attribute information as the comparison contents used when calculating as the difference information (step S12). This is information that can be calculated as the same content as the content of the attribute information calculated by the query attribute extraction means 41 or as difference information. The search target group is a plurality of content sets that can be determined to be the same content group in a certain unit, similar to the content group to which the search target group belongs.

  Next, using the attribute information extracted in step S11 and step S12, difference information is calculated from the attribute information items that can be compared by the attribute information comparison unit 43 (step S13). As the difference information, for example, the degree of coincidence of scene information is calculated from analysis metadata information. When normalizing from 0 to 1 from low to high, the matching level is 1 if the scene is indoors, 0 if indoor and outdoor, 0.5 if outdoor and waterside, etc. The difference can be calculated as the degree of coincidence based on the scene information. There are a plurality of pieces of scene information in a plurality of contents, but the scene information of each piece of content can be used equally and the number of each scene can be handled as the scene information of each group of contents. For example, when the number of contents is 10 and there are three types of scenes, indoor, waterside, and outdoor, when the number of various types is 7, 1 and 2, the scene information is normalized to 0.7 for each scene. And the degree of coincidence is calculated at the ratio. Then, difference information is calculated for each search target group.

  Subsequently, the degree of coincidence is calculated until all the difference information is calculated by calculating the degree of coincidence between the metadata information as each attribute information with respect to all comparable attribute information to be calculated as difference information. Is repeatedly calculated (step S14). The number and type of metadata information to be calculated can be determined in advance by a combination of metadata information that can be extracted by the content information extraction means 3, but a query can be made using metadata information held by the content group. Limit to only metadata information that is common to information and search targets, or to items with high comparison priority from content that can be extracted from metadata information extracted from a set of content that is a group of content in a fixed unit It is possible to decide.

  Finally, a similarity determination method is determined from the calculated difference information (step S15).

  An example of the calculated difference information is shown in FIG. When the content group to be searched is cID000001, the degree of coincidence of the shooting time zone and the scene is high. For example, when the average value of the total degree of coincidence calculated is high and the variance value is low, the content that matches generally is included. There is a high possibility of being. Therefore, the extracted feature value is a low-dimensional feature value such as color, edge, or texture as the overall feature value, and the similarity determination is based on the Euclidean distance after calculating the difference information in units of each pixel or pyramid hierarchy type. By selecting a method for determining the similarity based on the distance by a distance calculation method, such as N2 or N2 nom distance, a method for extracting similar content as the entire content is selected.

  In the case of cID000002, the degree of coincidence of the shooting location and composition is medium. For example, when the average value and the variance value of the calculated whole degree of coincidence are medium, content that partially matches is obtained. It is likely to be included. Therefore, the extracted feature value is a local feature value, and a high-dimensional feature value that can be determined as a partial match is calculated, and the similarity determination is a technique for finding a local match using BoVW (Bag of Visual Words). Or a method for determining a partial similarity such as a method for determining whether or not the same subject is included by a matching method.

  In the case of cID000003, the matching degree of the person information is high, but the matching degree of the event name is low. For example, when the average value and the variance value of the calculated total matching degree are both low, Is unlikely to be included, but there is a high possibility that only people match. Therefore, the feature quantity to be extracted is calculated as a feature quantity effective for face and human body detection, and the similarity determination is determined as a special subject determination method based on the matching degree determination based on the face feature quantity and the matching degree determination of clothes information and the like. . In this way, the higher the degree of coincidence of the difference information, the more similar the content is determined to be similar, and the lower the degree of coincidence of the difference information is, the more similar the local part of the content is to be determined to be similar It is possible to decide to switch.

  When the similarity criterion for calculating the similarity is determined by the similarity criterion determination means 4, the similarity calculation means 5 uses the determined similarity criterion to measure the similarity of the target content data. The feature amount is extracted, and the similarity between the content data is calculated by the designated similarity calculation method (step S4). The similarity is calculated from the content based on the method determined by the reference information determination unit 44, and the similarity between the contents is calculated.

  When the similarity calculated by the similarity calculation means 5 is output, the matched content selection means 6 selects and extracts content having a similarity satisfying a predetermined condition from the content group to be searched (step S5). . For example, options for a target content group are extracted by extracting all contents having a matching degree equal to or greater than a certain threshold Th or by extracting N contents in descending order of matching degree. In addition, it is possible to extract an appropriate content group by changing the condition for each content group of a certain unit having different similarity criteria.

  As described above, the content of various metadata that is attribute information between contents is not subjected to search control processing for similar contents with a certain standard similarity determined based on a specific feature amount that can be acquired from the contents. By determining feature quantities and similar items that should be calculated taking into account differences in content, it is possible to dynamically change and determine suitable similarity criteria to be calculated between contents. Search control processing of similar content suitable for various content data becomes possible, and the user can effectively perform similar searches on similar items with higher satisfaction and can easily search and view.

(Embodiment 2)
Embodiments of the present invention will be described below with reference to the drawings. In the second embodiment, the content data input as a search query is fixed as a data group search target group of images, videos, documents, music, etc., which are various content data possessed by a user or for specific applications. In a similar content search processing apparatus that performs a search control process for extracting and presenting content data having a similarity degree of the above, statistically superior information is extracted by attribute information for each content group, and between content based on the statistical information This is related to a search control processing mechanism for determining a similarity criterion to be calculated in (1) and extracting content data similar to each content group. Note that in this embodiment, components having the same functions as those in Embodiment 1 are assigned the same reference numerals, and descriptions thereof are omitted because they can be applied. In this embodiment, unless otherwise specified, the same reference numerals are given to configurations having the same functions as those in the first embodiment, and description thereof is omitted because the description can be applied.

  In the present embodiment, content that is a search query is used so that the user can easily perform similar searches or search and view content data such as photos, videos, documents, music, and the like that the user intends. By statistically analyzing and using the attribute information of the content group to be searched, the difference between the contents can be calculated with reasonable attribute contents with fewer errors as the difference information between the contents. Then, similar content search control processing is adaptively performed with the content suitable for the difference in shooting situation between the content. A detailed description will be given of a method capable of performing search control processing of similar content on a variety of content groups possessed by a user or for a specific purpose by using this process, based on a similarity criterion linked by a general difference of content groups. To do.

  Input of content data as search query information to be a search source by the user or the system is performed by the query information input means, and search control processing for similar content data is started. When the similar content search process is started, in addition to the process of the first embodiment, the statistical characteristic item of the attribute information extracted from the query information or the search target content is extracted, and the content between the contents is determined according to the item. The similarity information suitable for the difference between the contents is determined by extracting the difference information. Then, the similarity between the contents is calculated based on the determined similarity criterion, and search control processing for selecting and determining similar contents satisfying a predetermined condition is performed. Here, the various pieces of metadata information that is the attribute information of the content to be extracted is the same as that of the first embodiment. In particular, statistically calculated item units include shooting date and time, shooting location and shooting range, characters and family members, shooting scenes and compositions, object tags related to the subject, movement information on the screen, and the like.

  When the content attribute information is extracted, the similarity criterion determination unit 4 extracts the extracted query information and the attribute information of the search target content group, and calculates statistically characteristic items from the attribute information group. Then, a similarity criterion, which is information indicating how to calculate the similarity between contents from the difference information based on the characteristic item, is determined. The processing contents for determining the similarity criterion by calculating the similarity based on the statistical characteristic item of the attribute information from the content that is the query information and the content group to be searched will be specifically described with reference to FIG. . FIG. 9 shows a procedure for calculating a statistical characteristic item from the attribute information of the content that is the query information and the attribute information of the content group that is the search target, and performing the similarity criterion determination process from the difference information of the characteristic item. It is a flowchart.

  When the content of the query information that is the search source, its attribute information, and the content group to be searched and its attribute information are input, the content information extracting means 3 extracts the attribute information of the content (step S21).

  Next, using the attribute information extracted in step S21, an item having characteristics is extracted based on a statistical analysis of the attribute information group (step S22).

  Here, specific processing contents of an example of a functional configuration of the attribute information comparison unit 43 will be described with reference to FIG. FIG. 10 is a functional block diagram of the attribute information comparison unit 43. The attribute information comparison unit 43 includes a statistical information calculation unit 431, a characteristic item determination unit 432, and a characteristic item comparison unit 433. The statistical information calculation means 431 calculates statistical information in units that allow statistical analysis from a plurality of attribute information extracted from a plurality of content sets of a certain unit to which the content that is the input query information belongs or is a search target To do. The statistical information is a value that can be calculated statistically, such as a data amount and a distribution amount for each item of simple attribute information, and an average value and a variance value. The characteristic item determination unit 432 determines the characteristic item from the attribute information based on the statistical information calculated by the statistical information calculation unit 431.

  For example, if the content rate of how much various metadata information that is attribute information is included in all content is calculated, it is used when it is not used as a characteristic item when the content rate exceeds 10% You can make a decision to not. When the total content is 100 and the face detected content is 20, the face detection information is determined as the characteristic item. It is also possible to extract main items by using a statistical analysis method such as principal component analysis.

  The characteristic item comparison unit 433 extracts difference information that can be calculated by comparing each characteristic item determined by the characteristic item determination unit 432 in a format that can be calculated as a difference value (step S23). The type of difference information is the same as that of the first embodiment, but is determined by a combination of contents limited by characteristic items. The characteristic items can be configured to be limited to common items by using only attribute information extracted from the query information or only attribute information extracted from the search target group, or combining them. .

  Next, with respect to attribute information that is all comparable characteristic items to be calculated as difference information, calculation of all the difference information is completed by calculating the degree of coincidence between the metadata information that is each attribute information. The degree of coincidence is repeatedly calculated until it is (step S24).

  The number and types of metadata information to be calculated are the same as those in the first embodiment. Finally, a similarity determination method is determined from the calculated difference information (step S25). The distance calculation method based on the difference value between the feature amounts for determining the feature amount or the similarity degree to be calculated may be the same as that of the first embodiment, but the similarity is obtained using the degree of matching of the characteristic items in each attribute information. It is also possible to calculate.

The characteristic item: P _k (k is the number of the characteristic item), and the matching degree in each characteristic item is M _k . At this time, as the feature amounts determined from the respective characteristic items, feature amounts for determining the overall match degree, the local match degree, and the special match degree such as a face and a pet are selected, and the similarity value for calculating the overall match degree is set as V. If the overall matching degree calculated by the selected feature amount is G, the local matching degree is L, the special matching degree is S, and the weight of each item is α, β, and γ, V = α × G + β × L + γ × S Can be calculated as Here, alpha properties belonging to the entire degree of coincidence items: the sum of the degree of coincidence P _k: is calculated from the? M _k. β is calculated in the same manner from the characteristic item belonging to the local coincidence, and γ is similarly calculated from the characteristic item belonging to the local coincidence. It should be noted that the characteristic items can be configured to belong to a plurality of coincidence degree calculation targets or can be used only when the coincidence degree is high. In addition, the type of coincidence can be determined from a characteristic item, regardless of the type, and the central coincidence degree, the background coincidence degree, the partial coincidence degree for each region in a plurality of stages, and the like can be considered.

  When the similarity criterion determination unit 4 determines the similarity criterion for calculating the similarity based on the characteristic item, the similarity calculation unit 5 uses the determined similarity criterion to determine the similarity of the target content data. Feature amount extraction for measuring the degree is performed, and the degree of similarity between the content data is calculated by the designated degree of similarity calculation method. Then, when the similarity calculated by the similarity calculation unit 5 is output, the matched content selection unit 6 selects a content satisfying a predetermined condition, in particular, a content having a similarity with respect to the characteristic item, from the content group to be searched. And extracted. For example, if only “scene” information is extracted as a characteristic item for content shot at the same “sea bathing” event, the scene indicates the content in general, and content with similar content is preferentially extracted. In addition, if only “object tag” and “shooting time zone” information is extracted as characteristic items for content shot at the same “dining party” event, the shooting time zone and the object that is the subject are local parts of the content. In order to show the similarity, content with partial similarity or high semantic similarity is preferentially extracted. If only “Character” and “Shooting Composition” information is extracted as characteristic items for content shot at the same “Travel” event, the characters and shooting composition will be similar to the special person match of the content and general parts similar In order to show the nature, content with high similarity between the face and person of the content or the general partial area is preferentially extracted.

  As described above, instead of performing search control processing for similar content with a certain standard of similarity determined based on specific feature amounts that can be acquired from the content, statistics are obtained from various metadata that is attribute information between the content. By determining the feature quantity and similar items that should be calculated after taking into account the differences in the content extracted, the user can change the decision to a more appropriate similarity criterion that should be calculated between the contents. It is possible to search for similar content that is more generally suitable for a variety of content data for specific applications, enabling users to effectively search similar data with similar items with higher satisfaction, and easily You can search and watch.

(Embodiment 3)
Embodiments of the present invention will be described below with reference to the drawings. In the third embodiment, the content data input as a search query is constant as a data group search target group of images, videos, documents, music, etc., which are various content data possessed by a user or for specific applications. In a similar content search processing apparatus that performs search control processing for extracting and presenting content data having similarities, the similarity criteria to be calculated between the contents based on the difference in attribute information between the contents for each content group In addition, the present invention relates to a search control processing mechanism that determines an identity determination range and extracts content data identical to a search query in each content group. Note that in this embodiment, components having the same functions as those in Embodiment 1 are assigned the same reference numerals, and descriptions thereof are omitted because they can be applied. In this embodiment, unless otherwise specified, the same reference numerals are given to configurations having the same functions as those in the first embodiment, and description thereof is omitted because the description can be applied.

  In the present embodiment, content that is a search query is used so that the user can easily perform similar searches or search and view content data such as photos, videos, documents, music, and the like that the user intends. By using the attribute information of the content group to be searched, it is possible to determine whether the content is the same content by determining the identity determination range in addition to the difference information between the content. Then, a search control process for collating the same content generated due to a difference in shooting situation or usage situation between contents is performed. A method for performing search control processing for the same content existing on the same determination criterion suitable for the content group for various content groups owned by the user or for a specific application by this processing will be described in detail.

  Input of content data as search query information to be a search source by the user or the system is performed by the query information input means, and search control processing for matching with similar content data is started. When the content matching search process is started, the attribute information extracted from the query information or the content to be searched is extracted in addition to the process of the first embodiment, and the difference between the contents using the attribute information is extracted. Information is extracted, and a similarity criterion for content matching and an identity determination range are determined. Then, based on the determined similarity criterion, the similarity between the contents is calculated, and a matching search control process for selecting and determining the same content that satisfies the predetermined condition, here, satisfying the condition of the identity determination range is performed. Here, various types of metadata information that is attribute information of the content to be extracted is the same as that in the first or second embodiment. FIG. 11 is a flowchart showing a procedure for performing a matching search control process after performing a content matching similarity criterion determination process based on query information and attribute information of content data to be searched.

  When the user or system inputs content to be verified from the query information input means, search control processing for verification is started (step S31). The content input of the collation source can be selected and input by the user using an input device. However, the content can be automatically input by acquiring content from a plurality of devices and services by the system.

  When the collation source content is input, the content to be collated is acquired from the content data storage unit 1, and the content information extraction unit 3 performs attribute information extraction processing on the collation source content and the collation target content ( Step S32). The attribute information of the extracted content is the same as that of the first embodiment.

  When the content attribute information is extracted, the similarity criterion determination unit 4 uses the extracted attribute information of the collation source and collation target content group to calculate the similarity between the contents and determine the identity. A similarity criterion and identity determination range, which is information indicating whether or not, is determined (step S33). When the collation source content and its attribute information, and the content group to be collated and its attribute information are input, the content information extraction means 3 extracts the attribute information of the content, and uses the extracted attribute information for collation. The identity determination content of is calculated.

  Here, an example of the functional configuration of the reference information determining unit 44 when the functional configuration of the similarity criterion determining unit 4 is the same as that of the first embodiment will be described with reference to FIG. . FIG. 12 is a functional block diagram of the reference information determination unit 44. The reference information determination unit 44 includes processing information extraction unit 441, identity determination item calculation unit 442, and identity determination information determination unit 443.

  The processing information extraction unit 441 calculates processing content information that may have been changed from the original content data by using attribute information that is specific to the input collation source content. For example, the processing content information is information that can directly or indirectly determine from the editing information and service information that has been used that the data has been changed due to a simple encoding method change or resizing. It is also possible to calculate the degree of processing by calculating the entire content or partial editability by using analysis metadata. Note that it is also possible to adopt a configuration in which processing content information regarding a plurality of contents in a certain unit to be collated is calculated and used.

  The identity determination item calculation unit 442 determines the identity determination item from the difference information based on the processing content information extracted by the processing information extraction unit 441. For example, information indicating the possibility that the content data has been processed based on the processed content information is calculated from the large content of the difference information of the various metadata information. For example, because the usage service is different from the usage metadata information, the data storage method is resized to a fixed image size, or information such as the use of a processing filter or an area change by trimming is calculated. In addition, when the degree of processing varies depending on the content area from the analysis metadata information, content area information divided by a certain degree of processing is calculated.

  The identity determination information determination unit 443 includes a feature amount used as a similarity criterion for determining identity according to the identity determination item calculated by the identity determination item calculation unit 442, a similarity determination method thereof, and an identity determination range. To decide. For example, if the image size at the time of storage is resized and is different, it can be determined that the identity is the same when the identity determination is performed from the difference in content size using the feature amount for calculating the overall matching degree of the images Specify a range. The larger the size difference, the wider the same determination range, and the smaller the size difference, the narrower the setting can be made. In addition, in the case of area information change by filtering or trimming, a feature amount that calculates the local matching degree of images is used to specify a range that can be determined to be the same from the difference in area size, or between contents It is possible to set a range as a parameter that is linked to the degree of coincidence.

  When the similarity criterion determining unit 4 determines the similarity criterion and the identity determination range for collation, the similarity calculating unit 5 uses the determined similarity criterion to determine the similarity of the target content data. A feature amount for measurement is extracted, and a similarity between content data is calculated by a designated similarity calculation method (step S34). The similarity is calculated from the content based on the method determined by the reference information determination unit 44, and the similarity between the contents is calculated.

  When the similarity calculated from the similarity calculation unit 5 is output, the matched content selection unit 6 uses the similarity determination range determined by the similarity criterion determination unit 4 to determine the similarity satisfying the identity determination condition. Is selected from the content group to be collated and extracted (step S35). For example, when there is content whose coincidence is within the identity determination range, it can be managed that the content is the same content. If there is no such content, the same content may not exist. If there are multiple content, it may be limited to the content with the highest degree of matching, or it may be determined as one by presenting it to the user. Is possible. In addition, when the identity determination is performed more strictly, SIFT or SURF is used as a local feature amount in order to calculate a more detailed degree of coincidence after extracting a plurality of candidate contents whose similarity is higher than a certain threshold Th. It is also possible to adopt a multi-stage processing configuration in which identity is determined by strictly calculating the degree of coincidence using a geometric matching method between contents using feature quantities and the like. In order to perform identity determination more robustly, it may be possible to determine identity by calculating a plurality of similarity criteria.

  For example, if there are multiple feature quantities and similarity determination methods that can be determined step by step from global matching to local matching, identity determination is performed using multiple similarity criteria, and if all similarity criteria match It is determined that the content is the same as the original content that has not been processed, and if the content matches based on at least one similarity criterion, it is the same considering the level of processing by the number and degree of matching The sex determination result can be calculated. Furthermore, when statistically superior information exists by statistically extracting the processing content trends in specific service processing and editing processing from the usage metadata information, a feature amount calculation method and It is possible to adopt an update-type processing configuration in which identity determination is performed by updating the similarity calculation method.

  As described above, instead of performing search control processing for matching the same content with a certain standard of similarity determined based on a specific feature amount that can be acquired from the content, various types of attribute information between the content By deciding the feature quantity and the same judgment item and range that should be calculated in consideration of the difference in the content extracted from the metadata, it can be changed to a more appropriate identity judgment standard to be collated between contents, Search control processing for matching the same content that is more suitable for a variety of content data held by the user or for specific applications is possible, and the user effectively resembles the stored data with a more satisfactory identity determination result You can search and watch and watch easily.

(Embodiment 4)
Embodiments of the present invention will be described below with reference to the drawings. In the fourth embodiment, the content data input as a search query is fixed as a data group search target group of images, videos, documents, music, etc., which are various content data possessed by the user or for specific applications. Similarity criteria reflecting differences in attribute information based on differences in attribute information between contents for each content group in a similar content search processing apparatus that performs search control processing for extracting and presenting content data having similarities This is related to a search control processing mechanism for arranging and displaying extracted content data so that the change can be understood. Note that in this embodiment, components having the same functions as those in Embodiment 1 are assigned the same reference numerals, and descriptions thereof are omitted because they can be applied. In this embodiment, unless otherwise specified, the same reference numerals are given to configurations having the same functions as those in the first embodiment, and description thereof is omitted because the description can be applied.

  In the present embodiment, content that is a search query is used so that the user can easily perform similar searches or search and view content data such as photos, videos, documents, music, and the like that the user intends. By using the attribute information of the content group to be searched, the difference between the contents can be calculated step by step based on the difference information between the contents. Then, similar content search control processing and arrangement display processing are performed in which the similarity criterion is changed step by step in accordance with differences in the shooting status and usage status between the content. This process enables search control processing and arrangement display processing of similar content on a variety of content groups owned by a user or for a specific application, based on similarity criteria that are linked in stages to differences between content Will be described in detail.

  FIG. 13 is a block diagram showing the basic configuration of the similar content search processing apparatus of the present invention. In FIG. 13, the similar content search processing apparatus includes a content data storage unit 1, a query information input unit 2, a content information extraction unit 3, a similarity criterion determination unit 4, a similarity calculation unit 5, and a matching content selection. It comprises means 6, display means 7, and user operation input means 8.

  For the content data storage unit 1, the query information input unit 2, the content information extraction unit 3, the similarity criterion determination unit 4, the similarity calculation unit 5, and the matching content selection unit 6, basic operations are performed. It is the same as the content described in Form 1.

  The display unit 7 controls the display of the content group selected by the matching content selection unit 6 on a display device in a certain predetermined display format. For example, display is performed in accordance with a display configuration such as a touch panel display such as a smartphone or a tablet or a normal display.

  The user operation input means 8 determines input information for the similar content search processing device from the user operation. The input information is formed by a user operation such as pressing, dragging or releasing the touch panel on the display in the similar content search processing device, and includes physical parameters such as an operation start / end instruction and a pressed position coordinate on the display. It is not limited only to the case.

  When a user or system performs content similarity search, when the content data of the query information that is the search source is specified and entered for the search target content data group, the difference from the attribute information in the search target content data is reflected. After the content similarity criterion determination and the similar content search control process are performed, the search result is displayed to the user, and the query information and the like are re-input to repeatedly perform the similar search control process. When the user feedback type similar content repetitive search control process is started, difference information between the content is extracted from the content that is the input query information and the content group that is the search target based on the attribute information. Based on the extracted difference information, a similarity criterion suitable for the attribute difference between the contents is determined, the similarity between the contents is calculated based on the similarity criterion, and similar contents satisfying a predetermined condition are selected and determined. Search control processing is performed. Further, the search result is presented to the user by displaying it on the display. The user selects the content that the user wants to repeatedly perform similar searches from the presented content, so that repeated similar search control processing is performed. FIG. 14 is a flowchart showing a procedure for repeatedly performing similar content search control processing based on user feedback for performing similarity criterion determination processing based on query information and attribute information of content data to be searched.

  Input of content data as search query information to be a search source by the user or system is performed by the query information input means, and a search control process for similar content is started. When the similar content search process is started, the extracted content that is the search result is displayed in addition to the process of the first embodiment, and the user inputs feedback on the presentation result, so that the similar content is repeated based on the input content. The search control process is performed.

  When the similar content search control process is started, the query information and the attribute information of the content group to be searched are extracted as in the first embodiment (step S41).

  Next, the similarity criterion determination means 4 calculates difference information from the extracted attribute information group and determines a similarity criterion that defines how the similarity between contents is calculated (step S42). Then, the similarity calculation means 5 performs feature quantity extraction for measuring the similarity of the target content data using the determined similarity criterion, and calculates the similarity between the content data by the designated similarity calculation method. calculate.

  Then, when the similarity calculated by the similarity calculation unit 5 is output, the matched content selection unit 6 selects and extracts content having a similarity satisfying a predetermined condition from the search target content group (step S43). ).

  The extracted content group is presented to the user in a predetermined format by the display means 7 (step S44). For example, the extracted content group can be displayed in descending order of similarity, and information on the selected similarity criterion can also be displayed in a format that can be grasped by the user. FIG. 15 shows an example of an effective display method when searching by switching the similarity criterion one-dimensionally with changes in attribute information.

  By changing the similarity criterion based on specific attribute information, it is possible to arrange a similar content that is selected in accordance with the variation of the similarity criterion in order so that the user can easily understand the display method. . For example, since the same content is unlikely to become the same as the difference increases with time zone fluctuation or shooting location fluctuation, the similarity determination content is changed from overall matching to local matching and similarity determination by matching only the central part By changing the search results, the search results can be sequentially arranged and displayed as search results when the criteria for determining similarity are changed in order.

  In FIG. 15A, the similarity criterion is changed from 1 to 4 in order, and the extracted result is arranged in the horizontal direction when the similarity criterion is changed, and is arranged in the vertical direction when the similarity criterion is changed. It is a display example at the time. In FIG. 15B, the extraction result when the similarity criterion is sequentially changed from 1 to 5 is arranged centering on the content of the query information, and when the similarity criterion is changed, the extraction result is sequentially clockwise from the center. It is a display example when arranging and arranging the result of the same similarity criterion far from the center in the direction of the similarity criterion.

  In addition, fluctuations in attribute information can be extracted from various types of metadata information such as type information of shooting equipment used, time information such as dates, location information such as landscape scenes and store genres, and service information used for each processing type. Any type of information may be used as long as the similarity criterion varies according to the variation of the information. Furthermore, FIG. 16 shows an example of an effective display method when searching by switching the similarity criterion two-dimensionally with changes in attribute information.

  The arrangement method of the content accompanying the change of the basic similarity criterion is the same as the one-dimensional case, but there are a plurality of attribute information such as time information × location information or location information × service information or service information × time information. When changing, the similarity criterion also fluctuates in a plurality of dimensions, so that the content can be displayed in a multidimensional manner. In FIG. 16, when the similarity criterion fluctuates from 1 to 3 on the horizontal axis and the similarity criterion fluctuates from A to C on the vertical axis, the content is changed from the entire match according to each variation. It is a display example when content of a search result is arranged from partial match to center match. When a plurality of similarity criteria are crossed, content having a high similarity according to both similarity criteria is arranged.

  When an operation such as reselection is input to the displayed content group by the user input means 8, the process returns to step S41, the similarity criterion redetermination process is performed in conjunction with the operation content, and the search result The content is arranged in a predetermined format and presented to the user, but when there is no operation process by the user, the similarity search process ends (step S45). For example, when content that is query information is reselected as a user operation, a similarity criterion optimized for the content is selected, and a similarity search result based on the similarity criterion is presented to the user by the display means. By repeating this, the user can repeatedly search for similar content optimum for the selected content.

  In order to effectively search for content on the premise of user operation, it is possible to structure similar content search. FIG. 17 shows an example of a display method when searching for content by switching the similarity criterion when the folder structure is assumed. FIG. 17 shows a display example of a similar search result when contents are structured and managed in folders by date. Date folders containing a lot of content similar to the content that is the selected query information are arranged on the vertical axis. When the content of the selected query information is the content of the beach scene, the folder content of the beach scene is placed at the top, the next similar outdoor scene is placed, and the indoor scene with low similarity is placed at the end. Has been. Even in the same folder, content similar to the query content is displayed larger, and the lower the similarity is, the smaller the display size can be displayed. Since each date folder has different scene information, the similarity is determined on the basis of the similarity determined based on the difference between the scenes, so that the content is displayed based on the similarity determination result suitable for each folder. The size has been determined.

  In this way, it is possible to adopt a step-by-step processing configuration in which search processing is performed with a structure such as a folder in which content is managed, and the target content is narrowed down and then switched to similar content search processing for a single content. When content is managed in a hierarchical structure that can be created from various types of metadata information, processing is performed so that similar searches are performed step by step in accordance with the user's operation in a format corresponding to the hierarchical structure. It is also possible to adopt a processing configuration in which the unit is switched to perform a similar search for the target content.

  As described above, it is attribute information between contents rather than search control processing and arrangement display control processing of similar content with a certain standard similarity determined based on a specific feature amount that can be acquired from the content. By determining the feature quantities and similar items that should be calculated in stages, taking into account the differences in content extracted from various metadata, the decision was made to change to a more appropriate similarity criterion that should be calculated in stages between contents. Therefore, it is possible to perform search control processing and arrangement control processing of similar content that is more suitable for changes in the difference in attribute information for various content data held by the user or for specific applications, and the user is more satisfied It is possible to effectively perform similar searches of stored data with similar items and arrangement display, and to easily search and view.

  As described above, the similar content search processing device according to one aspect of the present invention has been described based on the embodiments. However, the present invention is not limited to these embodiments. Unless it deviates from the meaning of this invention, the form which carried out various deformation | transformation which those skilled in the art can think to this embodiment, or the structure constructed | assembled combining the component in different embodiment is also contained in the scope of the present invention. .

  For example, some or all of the constituent elements included in the similar content search processing apparatus according to the first embodiment may be configured by one system LSI (Large Scale Integration).

  The system LSI is an ultra-multifunctional LSI manufactured by integrating a plurality of components on one chip. Specifically, a microprocessor, a ROM (Read Only Memory), a RAM (Random Access Memory), etc. It is a computer system comprised including. A computer program is stored in the ROM. The system LSI achieves its functions by the microprocessor operating according to the computer program.

  Although the system LSI is used here, it may be called IC, LSI, super LSI, or ultra LSI depending on the degree of integration. Further, the method of circuit integration is not limited to LSI's, and implementation using dedicated circuitry or general purpose processors is also possible. An FPGA (Field Programmable Gate Array) that can be programmed after manufacturing the LSI, or a reconfigurable processor that can reconfigure the connection and setting of circuit cells inside the LSI may be used.

  Furthermore, if integrated circuit technology comes out to replace LSI's as a result of the advancement of semiconductor technology or a derivative other technology, it is naturally also possible to carry out function block integration using this technology. Biotechnology can be applied.

  In addition, the present invention can be realized not only as a similar content search processing device including such a characteristic processing unit, but also as a similar content search using the characteristic processing unit included in the similar content search processing device as a step. It can also be realized as a processing device. It can also be realized as a computer program that causes a computer to execute the characteristic steps included in such a method. Needless to say, such a computer program can be distributed via a computer-readable non-transitory recording medium such as a CD-ROM or a communication network such as the Internet.

(Supplement)
Hereinafter, the configuration of the moving image processing apparatus according to the embodiment of the present invention, its modified examples, and each effect will be described.

  (1) A similar content search processing apparatus according to the present invention includes a content data acquisition unit that acquires a content group that is a search target, and a query information input unit that inputs at least one content data serving as a query source at the time of search. Content information extraction means for extracting attribute information of content acquired by the content data acquisition means or content input by the query information input means, and the query information input means extracted by the content information extraction means The difference information of the attribute information is calculated from the attribute information of the content input by the content information and the attribute information of the at least one content group acquired by the content data acquisition unit, and the content information is calculated based on the calculated difference information. Similarity criteria used as the basis for calculating similarity Similarity criterion determining means to be determined, similarity calculating means for calculating similarity between the contents based on the similarity criterion determined by the similarity criterion determining means, and calculated by the similarity calculating means Matching content selection means for selecting content satisfying a predetermined condition based on the similarity.

  This configuration makes it possible to calculate the degree of similarity according to the difference in attribute information of content data, even for a variety of content data. It is possible to perform a similar content search with an appropriate degree of similarity.

  (2) In the above (1), the similarity criterion determination unit may include attribute information extracted from the input content itself or input content as the attribute information of the content input to the query information input unit. It further comprises query attribute extraction means for extracting attribute information extracted from the content group of a certain unit to which it belongs, or both attribute information, and the similarity criterion determination means includes attribute information extracted by the query attribute extraction means. Based on this, the calculation method of the difference information to be calculated may be changed.

  (3) In the above (2), the similarity criterion determining means further comprises target group attribute extracting means for extracting attribute information from a content group to be searched acquired by the content data acquiring means, and the similarity The reference determining means is configured such that a certain unit of content group from which the attribute information is extracted is composed of a plurality of content data, and the configuration conditions include at least the content group shooting time, shooting location, landscape scene, shooting composition, event content or It is good also as a structure which satisfy | fills any conditions of the unit which the user put together.

  (4) In the above (2), the similarity criterion determination unit compares the attribute information extracted by the query attribute extraction unit with the attribute information extracted from the target group attribute extraction unit, and is necessary for similarity determination Attribute information comparison means for calculating the difference information, and in the similarity criterion determination means, the comparison unit for calculating the difference information is at least the shooting time, shooting location, landscape scene, shooting composition of the content group, It is a constitutional unit that satisfies any of the conditions of the event content or the unit organized by the user, and may be configured to calculate the difference information from at least one content group composed of the same granularity conditions.

  With the configurations (2) to (4) above, since the granularity of the attribute information of the content to be extracted can be made uniform in calculating the difference information between the contents to be compared, the search target content group and the search query It is possible to perform a similar content search with a degree of similarity more suitable for a certain content.

  (5) In the configuration of (1), the attribute information extracted from the content information extraction unit can be extracted by analyzing the metadata information and the content data that can be extracted from the device in which the content data is generated. Metadata information or metadata information that can be extracted by using the content data, at least one piece of metadata information, and the similarity criterion determination means is extracted by the content information extraction means. The difference information may be calculated using the statistical information that can be extracted by a statistical analysis method for the attribute information for the content group.

  (6) In the configuration of (5), the similarity criterion determination unit further includes a criterion information determination unit that determines criterion information such as a feature amount and a similarity calculation method used for similarity determination based on the difference information. In the reference information determining means, information relating to the time of the content group, such as the shooting date and shooting time zone, information relating to the shooting location, shooting range, etc., information relating to the characters, people who appear frequently, specific poses, etc. The statistical information of the content group is extracted using at least one or more information out of the information about the photographing contents such as the scene, the composition and the movement information of the subject and the information about the photographing subject such as the photographing object, and the similarity criterion determining means May be configured to determine the reference information according to the extracted statistical information.

  (7) In the configuration of (5), the similarity criterion determination unit further includes a criterion information determination unit that determines criterion information such as a feature amount and a similarity calculation method used for similarity determination based on the difference information. The similarity criterion determination means may be configured to determine the reference information used for similarity determination using attribute information with a small difference of a certain value or more as difference information among the extracted statistical information.

  (8) In the configuration of (7), the similarity criterion determination means calculates the degree of coincidence obtained from the difference information of each attribute information in the statistical information, thereby weighting reference information used for similarity determination It is good also as a structure which determines the said reference | standard information by changing.

  (9) In the configuration of (7), the similarity criterion determination means relates to a feature amount relating to the entire content, a feature amount relating to a local portion of the content, a foreground or background region of the content as the use feature amount that is the reference information. Select and determine at least one of the feature quantities, the feature quantities related to the hierarchical structure of the content, and the feature quantities related to the subject of the content, and select a method that can calculate the similarity based on the selected and determined feature quantities. It is good also as a structure to determine.

  With the configurations (5) to (9) above, by calculating difference information based on the statistical information of the content attribute information, it is possible to calculate the similarity that correctly reflects the difference in the content attribute information. For this reason, it is possible to perform a similar content search with a degree of similarity that takes into account the difference between the contents that are more overhead.

  (10) In the configuration of (1), the similarity criterion determination unit determines whether the content is the same from the similarity between the content data of the inquiry source and the content data group to be searched. Based on the difference information, an identity determination range representing a similarity range in the case where the content is the same as the similarity criterion may be determined.

  (11) In the configuration of (10), the attribute information extracted from the content information extraction unit can be extracted by analyzing the metadata information and the content data that can be extracted from the device in which the content data is generated. Metadata information, or metadata information that can be extracted by using the content data, at least one piece of metadata information, and the similarity criterion determination means is metadata information that is the attribute information By determining whether the content is the same content based on the type of the content and the difference information, and extracting the trend information common to the content group for the target to be detected, the same content The sex determination range may be determined.

  (12) In the configuration of (10), the similarity criterion determination means should determine identity based on the similarity criterion, the identity determination range, and the content group selected by the matching content selection means. The content is narrowed down, the similarity criterion and the identity determination range that can determine the content to be determined are re-determined, and the matching content selection unit determines the similarity with the similarity criterion determined by the similarity criterion determination unit In the case where it is not possible to determine from the content group that matches the range that the content is the same, the same content may be recursively searched and selected by inputting the matched content group to the similarity determination means.

  (13) In the configuration of (10), the similarity criterion determining means determines a plurality of similarity criteria and identity determination ranges for each content group, and the matched content selecting means is the similarity degree The same content may be selected by using the similarity change degree or the identity change degree of each content from a plurality of similarity criteria determined by the reference determining means and the identity determination range.

  With the configurations (10) to (13), similarity calculation and identity determination according to differences in attribute information of various content data can be performed. It is possible to perform a similar content search that can make the same determination as the search query.

  (14) In the configuration of the above (1), the image processing apparatus further includes display means for displaying the content selected by the matching content selection means in an arrangement suitable for the similarity criterion, and the similarity criterion determination means includes a difference between attribute information The similarity criterion determined based on the information is changed at least once for the comparison content group, and the matching content selection unit uses the plurality of similarity criteria determined by the similarity criterion determination unit. A content group that matches the similarity calculated by the similarity calculation means under a certain condition may be selected.

  (15) In the configuration of (14), the similarity criterion determination unit changes the similarity criterion based on difference information of one specific attribute information or one complex attribute information, and the display unit The content selected by the matching content selection unit may be arranged and displayed in a one-dimensional manner in the order in which the difference information of the one attribute information increases or decreases according to the changed similarity criterion. .

  (16) In the configuration of (14), the similarity criterion determining means changes the similarity criterion based on difference information between two specific attribute information or two complex attribute information, and the display means The content selected by the matching content selection means may be arranged and displayed two-dimensionally in the order in which the difference information of the two attribute information increases or decreases according to the changed similarity criterion. Good.

  (17) In the configuration of (14) above, when a user-desired content or a predetermined unit of content group is selected from the content displayed on the display means or the predetermined unit of content group, the selection information, etc. The apparatus further comprises a user operation input means for inputting, and the similarity criterion determination means selects optimum attribute information between the contents for calculating difference information based on the selection information input by the user operation input means. The similarity criterion may be re-determined.

  (18) In the configuration of (17), the history information extracted by the content information extraction unit includes metadata structure or hierarchical structure information of content such as a folder configuration edited by the user, and the similarity The reference determining unit may determine the similarity criterion for each content group in a specific hierarchy or a specific structure in the structure information, and the display unit may arrange the content according to the structure information for which the similarity criterion is determined. .

  With the configurations of (14) to (18), it is possible to calculate various similarities according to differences in attribute information for various content data. It is possible to perform a similar content search in which content can be arranged and displayed with the similarity of the viewpoint.

  The similar content search processing device and method according to the present invention does not perform search control processing of a similar content group based on similarity determination based on a fixed criterion based on at least image characteristics of content based on a table-specific feature amount. For a variety of contents that are owned or for specific uses, feature quantities and similar contents that should be extracted between contents based on the difference in attribute information between the content that is the search query and the content group that is the search target Since the process of determining and extracting is performed, it is possible to calculate the similarity between the contents based on the similarity criterion to be calculated that is suitable for the shooting content of the content, and to perform an effective similar content search control process.

  For example, when similar searches of content data and collation of the same content are performed, similar content to be compared can be determined in consideration of the difference in attribute information between content. It is possible to perform similar content search control processing after performing appropriate and appropriate similarity calculation. Therefore, it is possible to effectively perform search control processing of similar content suitable for the content that is a search query, which is useful for various similar content search devices. It can also be applied to uses such as DVD / BD recorders, TVs, personal computer software, data servers and data services.

DESCRIPTION OF SYMBOLS 1 Content data storage part 2 Query information input means 3 Content information extraction means 4 Similarity criteria determination means 5 Similarity calculation means 6 Matched content selection means 7 Display means 8 User operation input means 41 Query attribute extraction means 42 Target group attribute means 43 Attribute information comparison means 44 Reference information determination means 431 Statistical information calculation means 432 Statistical item determination means 433 Characteristic item comparison means 441 Processing information extraction means 442 Identity determination item calculation means 443 Identity determination information determination means

Claims (20)

Content data acquisition means for acquiring a content group to be searched;
Query information input means for inputting at least one or more content data to be a query source at the time of search;
Content information extraction means for extracting attribute information of content acquired by the content data acquisition means or content input by the query information input means;
Difference information of attribute information from the attribute information of the content input by the query information input means extracted by the content information extraction means and the attribute information of at least one content group acquired by the content data acquisition means A similarity criterion determining means for determining a similarity criterion that is a criterion for calculating the similarity between the contents based on the calculated difference information;
Similarity calculating means for calculating the similarity between the contents based on the similarity criterion determined by the similarity criterion determining means;
Matching content selection means for selecting content satisfying a predetermined condition based on the similarity calculated by the similarity calculation means;
A similar content search processing apparatus comprising: The similarity criterion determination means is extracted as attribute information of the content input to the query information input means from attribute information extracted from the input content itself or from a certain unit of content group to which the input content belongs. Query attribute extraction means for extracting the attribute information or both of the attribute information,
2. The similar content search processing apparatus according to claim 1, wherein the similarity criterion determination unit changes a calculation method of difference information to be calculated based on attribute information extracted by the query attribute extraction unit. The similarity criterion determining means further comprises target group attribute extracting means for extracting attribute information from the content group to be searched acquired by the content data acquiring means,
The similarity criterion determining unit is configured such that a certain unit of content group from which the attribute information is extracted includes a plurality of content data, and the configuration conditions include at least a shooting time, a shooting location, a landscape scene, a shooting composition of the content group, The similar content search processing apparatus according to claim 2, wherein either a condition of an event content or a unit collected by a user is satisfied. The similarity criterion determination unit compares the attribute information extracted by the query attribute extraction unit with the attribute information extracted from the target group attribute extraction unit, and calculates difference information necessary for similarity determination Further comprising means,
In the similarity criterion determination means, the comparison unit for calculating the difference information is at least one of the conditions of the content group shooting time, shooting location, landscape scene, shooting composition, event content, or unit compiled by the user The similar content search processing apparatus according to claim 2, wherein difference information is calculated from at least one content group that is a structural unit satisfying the above and is configured with the same granularity condition. The attribute information extracted from the content information extraction unit uses metadata information that can be extracted from the device in which the content data is generated, metadata information that can be extracted by analyzing the content data, or the content data. Including at least one or more pieces of metadata information that can be extracted by
The similarity criterion determination unit calculates difference information for the attribute information for the content group extracted by the content information extraction unit, using statistical information that can be extracted by a statistical analysis method. Item 6. The similar content search processing device according to Item 1. The similarity criterion determination means further includes reference information determination means for determining reference information such as a feature amount and a similarity calculation method used for similarity determination based on the difference information,
In the reference information determining means, information related to time such as the shooting date and shooting time zone of the content group, information related to the location such as the shooting location and shooting range, information related to the characters, people who appear frequently, specific poses, shooting scenes, The statistical information of the content group is extracted by using at least one piece of information among shooting information such as composition and subject movement information, and shooting subject such as a shooting object, and the similarity criterion determination means includes: 6. The similar content search processing apparatus according to claim 5, wherein the reference information is determined according to the extracted statistical information. The similarity criterion determination means further includes reference information determination means for determining reference information such as a feature amount and a similarity calculation method used for similarity determination based on the difference information,
The similarity criterion determining means determines the criterion information used for similarity determination using attribute information with a small difference of a certain value or more as difference information among the extracted statistical information. 5. The similar content search processing device according to 5. The similarity criterion determination means determines the reference information by changing the weighting of the reference information used for similarity determination by calculating the degree of coincidence obtained from the difference information of each attribute information in the statistical information. 8. The similar content search processing apparatus according to claim 7, wherein: The similarity criterion determining means includes, as the used feature amount as the reference information, a feature amount relating to the entire content, a feature amount relating to the local portion of the content, a feature amount relating to the foreground or background area of the content, and a feature relating to the hierarchical structure of the content 8. The method according to claim 7, further comprising: selecting and determining at least one of the amount and the feature amount related to the subject of the content, and selecting and determining a method capable of calculating the similarity based on the selected and determined feature amount. Similar content search processing device. The similarity criterion determining means determines whether the content is the same content based on the similarity between the content data of the query source and the content data group to be searched, and the same content as the similarity criterion based on the difference information The similar content search processing apparatus according to claim 1, wherein an identity determination range representing a similarity range is determined. The attribute information extracted from the content information extraction unit uses metadata information that can be extracted from the device in which the content data is generated, metadata information that can be extracted by analyzing the content data, or the content data. Including at least one or more pieces of metadata information that can be extracted by
The similarity criterion determination means determines whether the content is the same content based on the type of metadata information that is the attribute information and the difference information, and for the target to be detected The similar content search processing apparatus according to claim 10, wherein the identity determination range is determined by extracting tendency information common to the content group. The similarity criterion determination means narrows down the content to be determined based on the similarity criterion, the identity determination range, and the content group selected by the matching content selection unit, and determines the content to be determined Re-determine possible similarity criteria and identity range,
The matching content selection unit determines the matching content group when the matching content group cannot be determined to be the same content from the content group that matches the similarity criterion determined by the similarity criterion determination unit. 11. The similar content search processing apparatus according to claim 10, wherein the same content is recursively searched and selected by inputting to the means. The similarity criterion determining means determines a plurality of similarity criteria and identity determination ranges for each content group,
The matching content selection unit selects the same content by using the similarity change degree or the identity change degree of each content from a plurality of similarity criteria determined by the similarity criterion determination unit and the identity determination range. The similar content search processing device according to claim 10. Display means for displaying the content selected by the matching content selection means in an arrangement suitable for the similarity criterion;
The similarity criterion determination means is configured such that the similarity criterion determined based on the difference information of the attribute information is changed at least once for the comparison content group,
The matching content selection unit selects a content group that matches the similarity calculated by the similarity calculation unit using a plurality of similarity criteria determined by the similarity criterion determination unit under a certain condition. The similar content search processing apparatus according to claim 1. The similarity criterion determination means changes the similarity criterion based on difference information of one specific attribute information or one complex attribute information,
The display means arranges and displays the contents selected one-dimensionally in the order in which the difference information of the one attribute information increases or decreases according to the changed similarity criterion. The similar content search processing apparatus according to claim 14, wherein: The similarity criterion determining means changes the similarity criterion based on difference information between two specific attribute information or two complex attribute information,
The display means displays the content selected in the matching content selection means in a two-dimensional manner in accordance with the changed similarity criterion in the order of increasing or decreasing difference information between the two attribute information. The similar content search processing apparatus according to claim 14, wherein: A user operation input means for inputting selection information or the like when a user-desired content or a predetermined unit of content group is selected in the content displayed on the display unit or the predetermined unit of content group;
The similarity criterion determination means re-determines the similarity criterion by selecting optimum attribute information between contents for which difference information is calculated based on the selection information input by the user operation input means. The similar content search processing device according to claim 14, wherein: The history information extracted by the content information extracting means includes metadata structure or hierarchical structure information of the content such as a folder structure edited by the user,
The similarity criterion determination means determines a similarity criterion for each content group in a specific hierarchy or a specific structure in the structure information,
18. The similar content search processing apparatus according to claim 17, wherein the display unit arranges the content according to the structure information for which the similarity criterion is determined. A content data acquisition step of acquiring a content group to be searched;
A query information input step for inputting at least one content data to be a query source at the time of search;
A content information extraction step of extracting attribute information of the content acquired by the content data acquisition step or the content input by the query information input means;
Difference information of attribute information from the attribute information of the content input by the query information input step extracted by the content information extraction step and the attribute information of at least one content group acquired by the content data acquisition step A similarity criterion determination step for determining a similarity criterion that is a criterion for calculating the similarity between the contents based on the calculated difference information;
A similarity calculation step of calculating a similarity between the contents based on the similarity criterion determined by the similarity criterion determination step;
A matching content selection step for selecting content satisfying a predetermined condition based on the similarity calculated in the similarity calculation step;
A similar content search processing method comprising: On the computer,
A content data acquisition step of acquiring a content group to be searched;
A query information input step for inputting at least one content data to be a query source at the time of search;
A content information extraction step of extracting attribute information of the content acquired by the content data acquisition step or the content input by the query information input means;
Difference information of attribute information from the attribute information of the content input by the query information input step extracted by the content information extraction step and the attribute information of at least one content group acquired by the content data acquisition step A similarity criterion determination step for determining a similarity criterion that is a criterion for calculating the similarity between the contents based on the calculated difference information;
A similarity calculation step of calculating a similarity between the contents based on the similarity criterion determined by the similarity criterion determination step;
A matching content selection step for selecting content satisfying a predetermined condition based on the similarity calculated in the similarity calculation step;
A program characterized by having executed.

Images