TWI444923B - Automatic pseudo-object-based image region of interest detection method and system - Google Patents

Description

Method and system for detecting representative area of automatic analog object image

The invention relates to a method and a system for detecting a representative region of an automatic analog object image, in particular to a method and system for automatically detecting a representative region of an image by using a feature point to generate a pseudo object.

Due to the popularity of digital cameras and the Internet in recent years, the number of digital images has increased exponentially. At the application level, the analysis of image content has become more and more important; the current research on image object detection Most of the image is segmented to generate a block by using low-level information of the image, such as using color change characteristics to detect edges or boundaries in the image, and then extracting visual features for the block and representing the block. However, since the image itself belongs to the 2D plane, it is a very difficult problem for each block to determine which blocks belong to the same object in the image, and in this way, often many broken and not In the meaning of the block, some studies try to extract the texture features of the block when extracting the features of the block, and then form the blocks with similar textures and similar positions, but there are still many disadvantages in this way, such as the same object. The textures on different parts may be very dissimilar, or the same kind of object will be very different due to the difference in shooting angle and brightness when shooting images. When applied to object detection, the effect is often not ideal.

Other studies allow the user to first select the possible object areas, then etch or diffuse the pixels in the area or use the edge information to generate image blocks, and then form objects in the area and extract features for further Use, however, this method requires user intervention and is not so convenient in use.

It can be seen that there are still many shortcomings in the above-mentioned methods of use. It is not a good design and needs to be improved.

In view of the shortcomings derived from the above-mentioned conventional methods, the inventor of the present invention has improved and innovated, and after years of painstaking research, he finally succeeded in research and development of an automatic pseudo-object-like image representative area detection method and system.

The object of the present invention is to provide a method and a system for quickly and automatically generating a pseudo object and detecting a representative or region of interest for an image, and can use image retrieval and searching and image compression of a large amount of image data on the Internet. , image thumbnails, image annotation and other applications.

An automatic pseudo-object-like image representative region detecting method and system for achieving the above object aims to automatically estimate the number of pseudo-objects by grouping the feature points obtained by analyzing the signal characteristics in the image according to the plane coordinate relationship. Position and scope, and by further exploring the information of the simulated object, develop a system that automatically detects the most representative area (or area of interest) in the full picture, which can be based on the location, size, and coverage of the simulated object. Information such as the number of feature points automatically detects the most representative area of the image; if combined with the presence of other image mimic objects in the existing database, the term text frequency (Term Frequency) and reverse file frequency (Inverse Document Frequency) And other statistical characteristics, it can detect different representative objects or regions that exist in the existing database.

The invention mainly uses regional visual feature points as an important basis for the generation of pseudo-objects, firstly using feature point detection methods such as Hessian-Affine or Maximally Stable Extremal Regions (MSER). The feature point coordinates are obtained, and the feature point coordinates are used to group, and the information of the pseudo object is generated to represent multiple objects in a picture; wherein the grouping method can use a G-means grouping algorithm, Based on K-means, a mechanism for automatically determining the appropriate number of groups by Gaussian distribution detection, or a Gaussian mixture model with Bayesian information criterion is also used. Automated grouping can be achieved and the number of groups can be determined.

The method for generating a pseudo-object, the flow chart of the steps, please refer to FIG. 1 , including: Step 1: Using the Hessian-Affine feature point detection method to select features from the image (for example: texture, border or color, etc.) Regional visual feature points; Step 2: Using the Speeded Up Robust Features (SURF) feature point description method, calculate and record all regional visual features based on the regional visual feature points found in Step 1. The corresponding feature parameters of the point and its plane coordinate information; Step 3: According to the plane coordinate information of all the regional visual feature points found in step 2, the Gaussian mixture model is matched with the Bayesian message criterion for automatic grouping, and is automatically divided into N groups. Where N>=1, the area covered by the feature points in each group constitutes a pseudo-object; wherein the position of the pseudo-object is the center of gravity of the feature points in each group, and the size is represented by the variation of the feature points in the group.

Wherein, when only two pseudo-objects are generated, the position of the pseudo-object is closer to the coordinate point of the entire image, which can be regarded as the foreground of the image, and the other is regarded as the background of the image, thereby automatically distinguishing the foreground and background of the image. The function.

Please refer to FIG. 2 , which is a schematic diagram of a first embodiment of a method for detecting a representative region of an automatic analog object image according to the present invention: estimating a suitable number of Gaussian distributions by using a Bayesian message criterion to establish a hybrid model, This model uses the feature points as a grouping legend. Firstly, the Gaussian mixture model established by multiple sets of Gaussian distributions is used to simulate the distribution of feature points, for example, by two, three, four and five Gaussian distributions. The feature point distribution is followed by the Bayesian message criterion to evaluate the complexity of each group of models, and the Gaussian mixture model with the lowest complexity is selected as the optimal grouping number; after selecting the Gaussian mixture model, all the feature points in the picture can be returned separately. From the class to the Gaussian distribution that it most likely belongs to, and finally treating each Gaussian distribution as a group, the clustering result is obtained.

In this embodiment, the feature points are divided into four groups, belonging to four pseudo objects 201, 202, 203, and 204, the range of which is represented by an ellipse, and the regional visual feature points within each pseudo object are taken out. This sample image is represented in the database by four pseudo-object features.

Please refer to FIG. 3 , which is a schematic diagram of a second embodiment of a method for detecting a representative region of an automatic analog object image according to the present invention. The number of pseudo-objects can be preset into groups and the image is segmented into The legend of the overlapping N fixed shape regions, the fixed segmentation method cuts the picture into a plurality of lattice regions, and each region is regarded as a pseudo object.

In this embodiment, the picture is divided into two parts from the horizontal and vertical directions, and four equalized areas of upper left, upper right, lower left, and lower right are obtained, and another equal-sized area is set in the center of the picture; therefore, In this embodiment, the image is divided into five grid-like pseudo-objects 301, 302, 303, 304, and 305. Therefore, the sample image is represented by five pseudo-object features in the database.

In an actual system, the position of the pseudo object can be the center of gravity of the feature points in each group, or the center point of the coverage area. The range of the pseudo object can be represented by a shape such as a circle, an ellipse or an arbitrary polygon, and the range size can also be determined to be the largest or the smallest. Coordinate values, or coordinate mean and standard deviation are determined in different ways; in addition, the full map of each graph can also be regarded as a single pseudo-object (ie, N=1); in addition, to avoid automatic clustering algorithm, the image is divided. For too many unimportant small-area pseudo-objects, we usually set an upper limit, such as N <= 5, to avoid this situation.

Please refer to FIG. 4, which is an architectural diagram of an automatic analog object image representative area detecting system according to the present invention, comprising: a pseudo object generating module 11 for automatically simulating an object image as shown in FIG. The step flow of the representative area detecting method generates the pseudo-object and related feature information of the input image and records it in the database 13.

A representative area detecting module 12 automatically detects the most representative image in the image by using information such as the position, size, number of feature points or reverse file association index stored in the database 13 Area.

The representative area detecting module 12 can calculate the position ratio parameter values of the pseudo-objects by using the distance between the center position of the pseudo-object in the database and the distance of the entire image center; wherein the representative area detection The module 12 can calculate the size ratio parameter values of the pseudo-objects by using the size of the pseudo-objects in the database; wherein the representative area detecting module 12 can utilize the regional visual feature points contained in the pseudo-objects in the database. The quantity, the quantity proportional parameter value of each pseudo object is calculated; wherein the representative area detecting module 12 can calculate each of the position proportional parameter values, the size proportional parameter values, and the quantity proportional parameter values according to the weighted linear combination manner. The importance parameter of the pseudo object; and sorting all the imitation object importance parameter values in the same image from large to small, and accumulating the sum of the first k sorted values to be greater than a predetermined threshold, then k The range covered by the pseudo-object corresponding to the value is the representative area of the image.

a database 13 for storing data generated or used by each module, for example, characteristic parameters of regional visual feature points generated by all images, plane coordinate information, corresponding visual characters, pseudo object numbers, and all images The pseudo-object number corresponding to its representative area.

A visual character generating module 14 is configured to generate a visual text codebook for the characteristic parameters of all the pseudo-object regional visual feature points recorded by the database 13, and corresponding to the visual text of the visual text code book.

A reverse file association index generation module 15 is configured to associate an inverted file (Inverted File) associated index with all the visual objects recorded in the database 13 and calculate the reverse file frequency of each visual text ( Inverse Document Frequency).

Among them, if the position of the pseudo object is closer to the center point of the whole image, it usually indicates that the imitation object is more important, because the average person will focus on the center of the image when taking photos, so it can be based on the mth mimetic object. The position of the m-th pseudo-object is proportional to the distance between the center position and the center of the entire image. L _m , m = 1, 2, ..., N, L ₁ + L ₂ +... + L _N =1; In addition, the larger the range of the pseudo-object, the more often it means that the pseudo-object is more important, and the easier it is to attract people's attention, so the m-th can be calculated according to the size of the m-th mimic object. The size ratio parameter values of the pseudo-objects S _m , m = 1, 2, ..., N, S ₁ + S ₂ + ... + S _N =1; of course, it is not the more important the object is, the more important it is, For example, an airplane flying in the sky, the background of the blue sky and white clouds is likely to be classified into a large area of the pseudo-object, so we also consider the number of feature points, if a pseudo-object contains the number of regional visual feature points The more it is, the more important it is to represent this mimic object, because the mimic With more important visual features, such as borders, lines, etc., we can calculate the amount based on the m-th quasi feature points regional visual objects contained in the m-th pseudo-object number ratio of values of the parameter P _{m, m} = 1,2,...,N,P ₁ +P ₂ +...+P _N =1; calculate the importance of the mth pseudo-object based on the weighted linear combination of the above three importance proportional parameter values The parameter O _m , where O _m =a*L _m +b*S _m +c*P _m ,m=1,2,...,N,a,b,c>=0 and a+b+c= 1; Finally, all of quasi importance parameter object O within an image with the value of _m by sorting large to small, the cumulative sum of the values _m k O before and after the sort to be greater than a predetermined threshold, then the k O _m a range of values corresponding to pseudo object contemplated is the representative of the image of the region, this detection scheme where both the presence or absence of an image can be useful in database 13.

Please refer to FIG. 2 and FIG. 5 , which are schematic diagrams of the first and third embodiments of the method for detecting the representative area of the automatic analog object image according to the present invention: in this embodiment, the representative area detecting module is about to be shown. The pseudo-objects 201, 202, and 203 of the second form constitute the most representative region 501 of the example graph, wherein a=0.5, b=0.25, c=0.25, the preset threshold value is 0.75, and the range of the pseudo-object is represented by an ellipse. The most representative areas are represented by rectangles.

In addition, by further introducing the concept of TF-IDF in text search, when the data inventory is in the existing image, the representative area detecting module of the present invention can have another detection mechanism (please refer to FIG. 4 at the same time). FIG. 6 and FIG. 7): wherein the visual character generating module 14 uses the K-means algorithm to group all the pseudo-object regional visual feature points 143 recorded in the database 13 by using the characteristic parameters thereof ( A total of Q groups), and a visual feature parameter grouping 142 is generated, and a visual feature parameter codebook, that is, a so-called visual character codebook 141, is generated according to the center of gravity of all the groups, and all the pseudo-objective regional visual feature points recorded in the database 13 are 143. The visual feature parameters are respectively corresponding to the most similar visual characters 141q according to the visual text codebook 141; and the inverted file association index generating module 15 is used, and then each of the pseudo-objects recorded in the database 13 is recorded. 110 (PO _i,j , representing the jth mimetic object of the i-th picture, a total of R) establishes an inverted file association index 151 with each visual character 141q, and calculates each visual text (VW) Reverse file frequency Inverse Document Frequency IDF _VW , IDF _VW = (total number of pseudo-objects / number of pseudo-objects where the VW appears), for each visual text, the IDF _VW value and the importance of the visual text become Just proportional.

The representative area detecting module 12 first generates the pseudo-object related information by using the pseudo object generating module 11 with respect to the image not existing in the database 13, and then visually features the regional visual feature points of all the pseudo-objects of the figure. The parameter corresponds to the most similar visual character 141q according to the visual text codebook 141 of the visual text generating module 14, and calculates the visual character frequency (Term Frequency, TF _VW ) contained in each pseudo-object, wherein: TF _VW = (number of Occurrences of the VW in certain pseudo-object/sum of number of occurrences of all VWs in certain pseudo-object), the TF _VW value is proportional to the importance of the visual text 141q relative to the pseudo-object; wherein the meaning of TF-IDF Therefore, if a visual character 141q appears more frequently in a certain object, the more important the visual character 141q is; however, if the visual character 141q appears more frequently in different pseudo-objects, it means that the visual character 141q is less Representative; therefore, the representative area detecting module 12 for all the visual characters 141q appearing in the mth mimic object in the new image (total T _m )), the reverse file frequency of the visual character 141q and the visual character frequency of the pseudo object are used, and the importance parameter O _m of the mth pseudo object 110 is calculated by multiplying and summing the two. , M = 1,2, ..., N ; Finally, the pseudo-all objects O importance parameter values from large to small _m ordering within the image, the cumulative value sum _m O k before and after the sort greater than When a threshold value is preset, the range covered by the pseudo object corresponding to the k Om values is the most representative region of the image.

Please refer to FIG. 2 and FIG. 8 , which are schematic diagrams of a first and fourth embodiment of a method and system for detecting a representative area of an automatic analog object image according to the present invention: a representative area detecting module in this embodiment That is, in the case where the existing image already exists in the database, the pseudo object 204 in FIG. 2 is determined as the most representative region 801 of the example image, where Q=10,000, the preset threshold is 0.5, and the pseudo object is The range is represented by an ellipse, and the most representative area is represented by a rectangle. This mechanism is suitable for the image representative area detection system to automatically detect the presence of existing images in the database. Representative objects or areas in the existing database, especially suitable for large database, will make the database naturally have learning and adaptability; for example, there is no image of Taipei 101 building in the initial database. As the image gradually began to increase, more and more images were detected in the Taipei 101 building. However, as the number of images in the Taipei 101 building became more frequent, the system would Gradually will be Taipei 10 The 1 building is considered a background object, and naturally it tries to find a representative area that is different from the Taipei 101 building, thus making the database more diverse and representative.

The method and system for detecting the representative area of the automatic analog object image provided by the invention have the following advantages when compared with other conventional technologies:

1. The present invention can jump off the traditional image processing method to try to cut out the precise object area, and the problem of which the feature points of the adjacent object edge belong to, and quickly find the figure by feature point coordinates and grouping algorithm. The range and number of pseudo-objects in the image.

2. The invention can effectively represent multiple objects in any image and increase the representativeness of the feature data.

3. The present invention automatically generates position and feature information for all possible objects in an image, facilitating use in a variety of object-based imaging applications.

4. The invention can further automatically detect the representative (interesting) region of the image through the information of the pseudo object, and does not need to manually designate or draw the region, and is particularly suitable for processing a large amount of image data on the Internet.

5. The invention transforms the concept of TF-IDF in text retrieval into the application level of the importance judgment of the object based on the visual character of the image.

6. The present invention automatically extracts images of new objects that do not exist in the existing database for a newly input image, so that the database is more diverse and representative.

The detailed description of the preferred embodiments of the present invention is intended to be limited to the scope of the invention, and is not intended to limit the scope of the invention. The patent scope of this case.

To sum up, this case is not only innovative in terms of technical thinking, but also able to enhance the above-mentioned multiple functions compared with conventional articles. It should fully comply with the statutory invention patent requirements of novelty and progressiveness, and apply in accordance with the law. I urge you to approve this article. Invention patent application, in order to invent invention, to the sense of virtue.

11. . . Analog object generation module

110. . . Quasi-like object

12. . . Representative area detection module

13. . . database

14. . . Visual text generation module

141. . . Visual textbook

141q. . . Visual text

15. . . Reverse file association index generation module

151. . . Reverse file association index

143. . . Regional visual feature point

142. . . Visual feature parameter grouping

201 ^~ 204. . . Quasi-like object

301 ^~ 305. . . Quasi-like object

501. . . Most representative area

801. . . Most representative area

1 is a flow chart of generating a pseudo-object of an automatic pseudo-object-like image representative region detecting method according to the present invention;

2 is a schematic diagram of a first embodiment of a method for detecting a representative area of an automatic analog object image according to the present invention;

FIG. 3 is a schematic diagram of a second embodiment of a method for detecting a representative area of an automatic analog object image according to the present invention; FIG.

4 is an architectural diagram of a representative area detection system for an automatic analog object image according to the present invention;

FIG. 5 is a schematic diagram of a third embodiment of a method for detecting a representative area of an automatic analog object image according to the present invention; FIG.

6 is a schematic diagram of visual text generated by a visual text generating module of an automatic analog object image representative region detecting method according to the present invention;

FIG. 7 is a schematic diagram of a reverse file association index of a reverse file association index generation module of an automatic analog object image representative area detection method according to the present invention; FIG.

FIG. 8 is a schematic diagram of a fourth embodiment of a method and system for detecting a representative area of an automatic analog object image according to the present invention.

11. . . Analog object generation module

12. . . Representative area detection module

13. . . database

14. . . Visual text generation module

15. . . Reverse file association index generation module

Claims (14)

An automatic pseudo-object-like image representative region detecting method is a method for generating a pseudo-like object by using a regional visual feature point, and the method for generating a pseudo-object, comprising: Step 1: using a regional feature point detecting method from the image Select regional characteristic feature points; Step 2: Calculate and record the corresponding feature parameters of all regional visual feature points according to the regional visual feature points found in Step 1 using the regional feature point description method and its Plane coordinate information; Step 3: According to step 2, find the plane coordinate information of all regional visual feature points, use clustering algorithm to automate grouping, and automatically divide into N groups, where N>=1, covered by feature points in each group The area constitutes a pseudo-object. An automatic analog object image representative region detecting method according to claim 1, wherein the number of the pseudo objects can be specified in advance, and the image is divided into N A superimposable area of a fixed shape. An automatic pseudo-object image representative region detecting method according to claim 1, wherein the number of the pseudo-objects can be automatically determined by a grouping algorithm, and the image is divided into N pieces. An area similar to an elliptical arbitrary shape and which may not be continuous. An automatic pseudo-object image representative area detecting method according to claim 1, wherein when only two pseudo-objects are generated, wherein the pseudo-objects are located closer to the center point coordinates of the entire image. The image is the foreground of the image, and the other is regarded as the background of the image, thereby achieving the function of automatically distinguishing the foreground and the background of the image. An automatic pseudo-object-like image representative area detecting system comprises: a pseudo-object generating module, which uses the generating method of the input object to generate the pseudo-object and related feature information, and records the data in the database. A representative area detection module that automatically detects the most representative image in the image by using information such as the position, size, number of feature points, or reverse file association index stored in the database. a database; a database that stores data generated or used by each module; a visual text generation module that records characteristic parameters of regional visual feature points of all pseudo-objects recorded in the database to generate a visual text book And corresponding to the visual text code most similar to the visual text; a reverse file association index generation module, which associates all the pseudo-objects recorded in the database with the visual text to create an inverted file (Inverted File) Index and calculate the Inverse Document Frequency of each visual text. An automatic analog object image representative area detecting system according to claim 5, wherein the database records characteristic parameters, plane coordinate information, and corresponding vision of regional image feature points of all images. The text, the imitation object number, and the pseudo object number corresponding to the representative area of all images. For example, an automatic analog object image representative area detecting system according to item 5 of the patent application scope, wherein the representative area detecting module can utilize the position of each mimic object in the database to be inversely proportional to the distance of the entire image center. Calculate the position proportional parameter values of each mimic object. For example, an automatic pseudo-object image representative area detecting system according to item 5 of the patent application scope, wherein the representative area detecting module can calculate the size proportional parameter of each quasi-object by using the size of each mimic object in the database. value. An example of an automatic analog object image as in item 5 of the patent application scope The area detecting system, wherein the representative area detecting module can calculate the quantity proportional parameter values of each of the pseudo-objects by using the number of regional visual feature points included in each of the quasi-objects in the database. An automatic analog object image representative area detecting system according to claim 5, wherein the representative area detecting module generates the image that is not present in the database by using the pseudo object. The module generates the information related to the object, and the visual character generating module generates visual characters corresponding to the visual characteristic parameters of all the regional visual feature points of the pseudo object, and calculates the visual frequency of each pseudo object (Term Frequency). . An automatic analog object image representative area detecting system according to claim 5, wherein the representative area detecting image generates a representative area of the image by one or more pseudo-objects Composition, this area can be a continuous or discontinuous block and recorded in the database. An automatic pseudo-image-like image representative region detecting method is applicable to the case where an existing image exists in a database, and the steps thereof include: a. generating an image of the pseudo-object and related features of the input image, and recording In the database; b. Calculate the position proportional parameter values of the pseudo-objects according to the positions of the pseudo-objects and the distance from the center of the whole image; c. Calculate each according to the size of each mimic object in the figure The size and proportion parameter values of the pseudo-objects; d. According to the number of regional visual feature points contained in the pseudo-objects of the figure, calculate the quantity proportional parameter values of the pseudo-objects; e. The three proportional parameter values of the step ac are The weighted linear combination method calculates the importance parameter values of each mimic object; f. sorts all the imitation object importance parameter values in the image from large to small, and accumulates the sum of the first k sorted importance parameter values to More than one pre When the threshold is set, the range covered by the pseudo object corresponding to the k importance parameter values is the representative region of the image. An automatic pseudo-object-like image representative region detecting method is applicable to the case where there is no existing image in the database, and the steps include: a. generating an image of the pseudo-object and its related features, and recording the input image In the database; b. Calculate the position proportional parameter values of the pseudo-objects according to the position of the pseudo-objects and the distance from the center of the whole image; c. Calculate according to the size of each mimic object in the figure The size and proportion parameter values of each of the pseudo-objects; d. Calculate the number-of-proportional parameter values of each mimic object according to the number of regional visual feature points contained in the pseudo-objects of the figure; e. The three proportional parameter values of step ac, Calculate the importance parameter values of each mimic object according to the weighted linear combination method; f. Sort all the imitation object importance parameter values in the image from large to small, and accumulate the sum of the first k sorted importance parameter values. When the value is greater than a predetermined threshold, the range covered by the pseudo object corresponding to the k importance parameter values is a representative region of the image. An automatic pseudo-image-like image representative region detecting method is applicable to the case where an existing image exists in a database, and the steps thereof include: a. generating an image of the pseudo-object and related features of the input image, and recording In the database; b. grouping all the visual features of the pseudo-objects recorded in the database with their visual feature parameters to generate a visual text code book; c. Recording all the visual features of the pseudo-objects in the database The visual characteristic parameters are corresponding to the most similar visual text according to the visual text code book; d. an inverted file association index is established for all the pseudo-objects recorded in the database and the visual characters, and each visual text is calculated. Reverse File frequency; e. For the new image that does not exist in the database, the pseudo-object related information is generated, and the visual feature parameters of the regional visual feature points of all the pseudo-objects of the figure are corresponding to the most similar visual text according to the visual text code book. And calculating the frequency of the visual characters contained in each of the pseudo-objects; f. multiplying and adding the reverse file frequency of the visual text of step d to the visual text frequency of the pseudo-object of step e, and calculating the importance parameters of each pseudo-object Value; g. sort all the imitation object importance parameter values in the image from large to small, and accumulate the sum of the first k sorted values to be greater than a predetermined threshold, then the k values correspond to The range covered by the pseudo-object is the most representative area of the image.