CN111914920A - Sparse coding-based similarity image retrieval method and system - Google Patents

Abstract

The invention proposes a method and system for retrieving similarity images based on sparse coding, and relates to the field of image recognition. A similarity image retrieval method based on sparse coding, comprising the steps of: performing basis vector characterization according to a reference image to obtain a first sparse characterization result; performing basis vector characterization according to an image in an image library to obtain a second sparse characterization result; calculating a first sparse representation result The similarity between the sparse representation result and the second sparse representation result; determine whether the similarity is greater than a preset threshold, if so, it is determined to be a similarity image, and if not, it is determined to be a non-similar image. It can more fully extract image feature information, and is more accurate and targeted in the process of similarity calculation. In addition, the present invention also provides a similarity image retrieval system based on sparse coding, which includes: a first acquisition module, a second acquisition module, a calculation module and a judgment module.

Description

A method and system for similarity image retrieval based on sparse coding

technical field

The present invention relates to the field of image recognition, in particular, to a method and system for retrieving similarity images based on sparse coding.

Background technique

With the advent of the era of digital media, massive digital images have become an indispensable part of our lives, and are also widely used in life sciences, education, culture and other fields. Many classic machine learning methods, especially deep learning methods, can retrieve target images from massive image libraries. How to use a single image to retrieve semantically similar images from a massive image library has very good practical application value.

However, traditional machine learning methods cannot be applied very efficiently to the retrieval of single similar images. The current mainstream approach suffers from the following problems:

1. Feature extraction methods are not very effective and are highly dependent on training samples;

2. There is no targeted measurement standard in the process of similarity calculation.

SUMMARY OF THE INVENTION

The purpose of the present invention is to provide a similarity image retrieval method based on sparse coding, which can more fully extract image feature information, and is more accurate and targeted in the process of similarity calculation.

Another object of the present invention is to provide a similarity image retrieval system based on sparse coding, which can run a similarity image retrieval method based on sparse coding.

Embodiments of the present invention are implemented as follows:

In a first aspect, an embodiment of the present application provides a method for retrieving similarity images based on sparse coding, which includes the following steps to obtain a first sparse characterization result by performing basis vector characterization according to a reference image, and obtaining a first sparse characterization result by performing basis vector characterization according to images in an image library. For the second sparse representation result, calculate the similarity between the first sparse representation result and the second sparse representation result, and determine whether the similarity is greater than a preset threshold. image.

In some embodiments of the present invention, before obtaining the first sparse representation result by performing the basis vector characterization according to the reference image, the method further includes using representative images in the image library to train the basis vectors.

In some embodiments of the present invention, the above-mentioned training of the basis vectors using the representative images in the image library includes fixing the basis vectors in the dictionary, and adjusting the coding coefficients so that the objective function is minimized.

In some embodiments of the present invention, the above-mentioned training of the basis vectors using the representative images in the image library includes fixing coding coefficients, and adjusting the basis vectors in the dictionary so that the objective function is minimized.

In some embodiments of the present invention, the above-mentioned training of the basis vectors using the representative images in the image library includes obtaining a set of basis vectors that well express the sample images through continuous iteration until convergence.

In some embodiments of the present invention, before obtaining the first sparse representation result by performing the basis vector characterization according to the reference image, the method further includes optimizing the basis vectors according to cross-examination.

In some embodiments of the present invention, before obtaining the first sparse characterization result by performing the basis vector characterization according to the reference image, the method further includes optimizing the characterization of the image with inaccurate characterization.

In some embodiments of the present invention, before obtaining the first sparse characterization result by performing the basis vector characterization according to the reference image, the method further includes performing saliency detection on the image.

In a second aspect, an embodiment of the present application provides a similarity image retrieval system based on sparse coding, which includes a first acquisition module for performing basis vector characterization according to a reference image to obtain a first sparse representation result, and a second acquisition module for using The second sparse representation result is obtained by performing the basis vector representation according to the images in the image library, the calculation module is used to calculate the similarity between the first sparse representation result and the second sparse representation result, and the judgment module is used to determine whether the similarity is greater than the pre-determined similarity. A threshold is set, if yes, it is determined to be a similar image, if not, it is determined to be a non-similar image.

In some embodiments of the invention, the above also includes at least one memory for storing computer instructions, and at least one processor in communication with the memory, wherein when the at least one processor executes the computer instructions, the at least one processor causes the system to perform: A first acquisition module, a second acquisition module, a calculation module and a judgment module.

Compared with the prior art, the embodiments of the present invention have at least the following advantages or beneficial effects:

It can more fully extract image feature information, and is more accurate and targeted in the process of similarity calculation. By exploring different but invariant features and structures between related fields and making full use of existing resources, a sparse feature transfer model is constructed. Transfer knowledge from other related tasks to target tasks with few labeled samples to help improve the learning efficiency of target tasks. Transfer sparse coding image retrieval technology combines sparse coding and transfer learning technology, which is a new technology to realize the reuse of sparse features in different fields.

Description of drawings

In order to illustrate the technical solutions of the embodiments of the present invention more clearly, the following briefly introduces the accompanying drawings used in the embodiments. It should be understood that the following drawings only show some embodiments of the present invention, and therefore do not It should be regarded as a limitation of the scope, and for those of ordinary skill in the art, other related drawings can also be obtained according to these drawings without any creative effort.

1 is a schematic diagram of steps of a method for retrieving similarity images based on sparse coding provided by an embodiment of the present invention;

2 is a schematic diagram of detailed steps of a method for retrieving similarity images based on sparse coding provided by an embodiment of the present invention;

FIG. 3 is a schematic diagram of modules of a similarity image retrieval system based on sparse coding according to an embodiment of the present invention.

Detailed ways

In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments It is a part of the embodiments of the present application, but not all of the embodiments. The components of the embodiments of the present application generally described and illustrated in the drawings herein may be arranged and designed in a variety of different configurations.

Thus, the following detailed description of the embodiments of the application provided in the accompanying drawings is not intended to limit the scope of the application as claimed, but is merely representative of selected embodiments of the application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

Some embodiments of the present application will be described in detail below with reference to the accompanying drawings. The various embodiments described below and various features of the embodiments may be combined with each other without conflict.

Example 1

Please refer to FIG. 1. FIG. 1 is a schematic diagram of steps of a sparse coding-based similarity image retrieval method provided by an embodiment of the present invention, which includes the following steps:

Step S100, performing basis vector characterization according to the reference image to obtain a first sparse characterization result;

Specifically, the reference image is the selected image to be retrieved, and the basis vector is the most basic component in the vector space, because the basis vector must be linearly independent, this part can represent each vector in the vector space, and reduce the image dimension, Obtain the first sparse representation result.

In some embodiments, the electronic device shoots in various scenarios, such as night scenes or backlit environments. Under the same shooting scene, the electronic device can shoot multiple frames of images, and perform image registration on the multiple frames of images. After image registration, the electronic device may perform fusion processing on the multiple frames of images. However, in the related art, the image quality of the image obtained by the fusion of the electronic device is poor. For example, the electronic device may first acquire multiple frames of images to be processed. Afterwards, the electronic device may determine a frame of reference image from the multiple frames of images to be processed. For example, the electronic device may determine the image to be processed with the highest definition as the reference image. Then, the images other than the reference image in the multi-frame to-be-processed images are non-reference images.

Step S110, performing basis vector characterization according to the images in the image library to obtain a second sparse characterization result;

Specifically, the basis vector representation is performed on each image in the image library, so as to obtain the second sparse representation result.

In some embodiments, there may be A, B, C, D, and 4 images in the image library, and the A, B, C, D, and 4 images are represented by basis vectors, and the image can be obtained by reducing the dimensionality of the images. The second sparse characterization result of A, the third sparse characterization result of image B, the fourth sparse characterization result of image C, and the fifth sparse characterization result of image D.

Step S120, calculating the similarity between the first sparse representation result and the second sparse representation result;

Specifically, the multi-weight Euclidean distance calculation is used to calculate the similarity between the first sparse representation result and the second sparse representation result.

In some embodiments, multi-weight Euclidean distance can be used for calculation. Euclidean distance is a similarity measurement method. The distance between two vectors can be calculated. The value ranges from 0 to positive infinity. If the distance between two vectors is The smaller the distance, the more similar the vectors must be. Because the importance of different regions in the image is different, the ordinary Euclidean distance cannot accurately calculate the similarity of different images. In addition to data processing, weighted Euclidean distance can also be used, and different weights can be used according to different regional saliency, so as to calculate the similarity between the first sparse representation result and the second sparse representation result through multi-weight Euclidean distance calculation , the similarity between the first sparse representation result and the other one or more sparse representation results can also be calculated separately, for example, the similarity between the first sparse representation result and the third sparse representation result can be calculated by multi-weight Euclidean distance calculation , the similarity between the first sparse representation result and the fourth sparse representation result can also be calculated by multi-weight Euclidean distance calculation, and the difference between the first sparse representation result and the fifth sparse representation result can also be calculated by multi-weight Euclidean distance calculation similarity between.

Step S130, judging whether the similarity is greater than a preset threshold;

Specifically, it is determined whether the similarity is greater than the preset threshold, and if the similarity is greater than the preset threshold, the process proceeds to step S140, and if the similarity is less than or equal to the preset threshold, the process proceeds to step S150.

In some embodiments, the preset threshold may be changed according to different pictures. For example, it is assumed that a picture has a total of n pixels, wherein the number of pixels whose gray value is less than the threshold value is n1, and the number of pixels whose gray value is greater than or equal to the threshold value is n2 (n1+n2=n). w1 and w2 represent the respective proportions of these two types of pixels, and the average and variance of all pixels with gray values less than the threshold are μ1 and σ1, respectively, and the average and variance of all pixels with gray values greater than or equal to the threshold are μ2 and σ2, find the histogram of the image, use a series of thresholds from small to large to take them, and substitute them into the BBS formula. The value that makes "the smallest difference between classes" or "the largest difference between classes" is the final threshold, for example, it can be 0.4, 0.5, 0.6 and so on.

Step S140, it is determined that it is a similarity image;

Specifically, the retrieved pictures are determined as similar images.

In some embodiments, the similarity between the first sparse representation result and the fourth sparse representation result is the highest, so it is determined that the reference image and the C image in the image library are similar images.

Step S150, it is determined to be a dissimilar image.

Example 2

Please refer to FIG. 2. FIG. 2 is a schematic diagram of detailed steps of a sparse coding-based similarity image retrieval method provided by an embodiment of the present invention, which includes the following steps:

Step S200, fixing the basis vector in the dictionary, and adjusting the coding coefficients so that the objective function is the smallest;

Specifically, by keeping the base vector in the dictionary unchanged and changing the coding coefficients, the objective function is minimized, thereby completing the dictionary training.

Step S210, fixing the coding coefficients, and adjusting the basis vectors in the dictionary so that the objective function is the smallest;

Specifically, by keeping the coding coefficient unchanged, the basis vector in the dictionary is changed to minimize the objective function, thereby completing the dictionary training.

Step S220, through continuous iteration until convergence, obtain a set of basis vectors that well express the sample image;

Specifically, through multiple iterations until the model converges, a set of basis vectors that well express the sample image can be obtained.

Step S230, optimize the basis vector according to the cross-check;

In some embodiments, multiple sets of basis vectors are obtained by using multiple sets of experiments based on different image libraries, and we only retain the set of core basis vectors with the highest similarity with other sets of basis vectors.

Step S240, performing optimized characterization on the images with inaccurate characterization;

In some embodiments, the fidelity-constrained optimized characterization is performed on the imprecisely characterized images. The mean value is subtracted from each image block, and the dictionary represents the texture of the image. In the reconstruction stage, the mean value of the low-resolution image is directly used as the mean value of the high-resolution image. After the encoding reconstruction, the gradient descent method is used to continue the optimization.

Step S250, performing saliency detection on the image;

In some embodiments, the brain can think quickly after acquiring an image, find parts of interest and ignore others, a mechanism that enables a person to quickly react to what they see. With the development of computer technology, the use of computer to identify and process images has become an important development trend. The essence of saliency detection is feature extraction.

For example, for a 300*400 input image, 30FPS can be achieved. Some local modules have been changed. In one aspect, the global guidance module (GGM) is proposed, which provides the position information of some salient objects to the features of different scales, and the feature aggregation module (FAM) is also proposed. It is used to fuse feature maps of different scales. Putting the two modules together allows the network to obtain different receptive fields, thereby improving the detection performance of salient objects. In addition, an edge detection branch is added for synchronous training. This edge detection branch can also improve the detection performance of saliency targets.

Step S260, performing basis vector characterization according to the reference image to obtain a first sparse characterization result;

Specifically, the reference image is the selected image to be retrieved, and the basis vector is the most basic component in the vector space, because the basis vector must be linearly independent, this part can represent each vector in the vector space, and reduce the image dimension, Obtain the first sparse representation result. Reference may be made to the relevant description of step S100, which will not be repeated here.

Step S270, performing basis vector characterization according to the images in the image library to obtain a second sparse characterization result;

Specifically, the basis vector representation is performed on each image in the image library, so as to obtain the second sparse representation result. Reference may be made to the relevant description of step S110, which will not be repeated here.

Step S280, calculating the similarity between the first sparse representation result and the second sparse representation result;

Specifically, the multi-weight Euclidean distance calculation is used to calculate the similarity between the first sparse representation result and the second sparse representation result. Reference may be made to the relevant description of step S120, which will not be repeated here.

Step S290, judging whether the similarity is greater than a preset threshold;

Specifically, it is determined whether the similarity is greater than the preset threshold, and if the similarity is greater than the preset threshold, the process goes to step S3000, and if the similarity is less than or equal to the preset threshold, the process goes to step S310.

Step S300, it is determined that it is a similarity image;

Specifically, the retrieved pictures are determined as similar images.

In some embodiments, the similarity between the first sparse representation result and the third sparse representation result is the highest, so it is determined that the reference image and the B image in the image library are similar images.

Step S310, if not, it is determined as a dissimilar image.

Example 3

Please refer to FIG. 3 . FIG. 3 is a schematic diagram of modules of a similarity image retrieval system based on sparse coding according to an embodiment of the present invention. A similarity image retrieval system based on sparse coding, which includes a first acquisition module for performing basis vector characterization according to a reference image to acquire a first sparse representation result, and a second acquisition module for performing basis vector characterization according to images in an image library. The vector representation obtains the second sparse representation result, the calculation module is used to calculate the similarity between the first sparse representation result and the second sparse representation result, the judgment module is used to determine whether the similarity is greater than the preset threshold, and if so, it is determined as yes Similar images, if not, it is determined as a non-similar image.

It also includes a memory, a processor and a communication interface, the memory, the processor and the communication interface are electrically connected to each other directly or indirectly to realize the transmission or interaction of data. For example, these elements may be electrically connected to each other through one or more communication buses or signal lines. The memory can be used to store software programs and modules, and the processor executes various functional applications and data processing by executing the software programs and modules stored in the memory. The communication interface can be used for signaling or data communication with other node devices.

Wherein, the memory may be, but not limited to, random access memory (Random Access Memory, RAM), read only memory (Read Only Memory, ROM), programmable read only memory (Programmable Read-Only Memory, PROM), erasable only memory Read memory (Erasable Programmable Read-Only Memory, EPROM), Electric Erasable Programmable Read-Only Memory (Electric Erasable Programmable Read-Only Memory, EEPROM), etc.

The processor may be an integrated circuit chip with signal processing capabilities. The processor 102 may be a general-purpose processor, including a central processing unit (Central Processing Unit, CPU), a network processor (Network Processor, NP), etc.; it may also be a digital signal processor (Digital Signal Processing, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), Field-Programmable Gate Array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

In the embodiments provided in this application, it should be understood that the disclosed apparatus and method may also be implemented in other manners. The apparatus embodiments described above are merely illustrative, for example, the flowcharts and block diagrams in the accompanying drawings illustrate the architectures, functions and possible implementations of apparatuses, methods and computer program products according to various embodiments of the present application. operate. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code that contains one or more functions for implementing the specified logical function(s) executable instructions. It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It is also noted that each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations, can be implemented in dedicated hardware-based systems that perform the specified functions or actions , or can be implemented in a combination of dedicated hardware and computer instructions.

In addition, each functional module in each embodiment of the present application may be integrated together to form an independent part, or each module may exist independently, or two or more modules may be integrated to form an independent part.

If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be embodied in the form of a software product in essence, or the part that contributes to the prior art or the part of the technical solution. The computer software product is stored in a storage medium, including Several instructions are used to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, Read-Only Memory (ROM, Read-Only Memory), Random Access Memory (RAM, Random Access Memory), magnetic disk or optical disk and other media that can store program codes .

To sum up, the method and system for similarity image retrieval based on sparse coding provided by the embodiments of the present application can more fully extract image feature information, and the process of similarity calculation is more accurate and targeted. Explore different but invariant features and structures between related fields, make full use of existing resources, build a sparse feature transfer model, transfer knowledge from other related tasks to target tasks with few labeled samples, and help improve the learning efficiency of target tasks. Transfer sparse coding image retrieval technology combines sparse coding and transfer learning technology, which is a new technology to realize the reuse of sparse features in different fields.

The above descriptions are only preferred embodiments of the present application, and are not intended to limit the present application. For those skilled in the art, the present application may have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this application shall be included within the protection scope of this application.

It will be apparent to those skilled in the art that the present application is not limited to the details of the above-described exemplary embodiments, but that the present application can be implemented in other specific forms without departing from the spirit or essential characteristics of the present application. Accordingly, the embodiments are to be regarded in all respects as illustrative and not restrictive, and the scope of the application is to be defined by the appended claims rather than the foregoing description, which is therefore intended to fall within the scope of the claims. All changes that come within the meaning and scope of equivalents to are included in this application. Any reference signs in the claims shall not be construed as limiting the involved claim.

Claims (10)

1. A similarity image retrieval method based on sparse coding is characterized by comprising the following steps:

performing base vector characterization according to the reference image to obtain a first sparse characterization result;

performing base vector characterization according to the images in the image library to obtain a second sparse characterization result;

calculating the similarity of the first sparse representation result and the second sparse representation result;

and judging whether the similarity is greater than a preset threshold value, if so, judging the image to be a similar image, and if not, judging the image to be a non-similar image.

2. The sparse coding-based similarity image retrieval method as claimed in claim 1, wherein before the performing base vector characterization on the basis of the reference image to obtain the first sparse characterization result, further comprises:

the basis vectors are trained using representative images in the image library.

3. The sparse coding-based similarity image retrieval method as claimed in claim 2, wherein the training of the basis vectors by using the representative images in the image library comprises:

and fixing the base vectors in the dictionary, and adjusting the coding coefficients to minimize the target function.

4. The sparse coding-based similarity image retrieval method as claimed in claim 2, wherein the training of the basis vectors by using the representative images in the image library comprises:

and fixing the encoding coefficient, and adjusting the base vector in the dictionary to minimize the target function.

5. The sparse coding-based similarity image retrieval method as claimed in claim 2, wherein the training of the basis vectors by using the representative images in the image library comprises:

and obtaining a group of base vectors of the well-expressed sample image by continuously iterating until convergence.

6. The sparse coding-based similarity image retrieval method as claimed in claim 1, wherein before the performing base vector characterization on the basis of the reference image to obtain the first sparse characterization result, further comprises:

and optimizing the basis vectors according to cross mutual experiments.

7. The sparse coding-based similarity image retrieval method as claimed in claim 1, wherein before the performing base vector characterization on the basis of the reference image to obtain the first sparse characterization result, further comprises:

and performing optimized characterization on the images with inaccurate characterization.

8. The sparse coding-based similarity image retrieval method as claimed in claim 1, wherein before the performing base vector characterization on the basis of the reference image to obtain the first sparse characterization result, further comprises:

and carrying out significance detection on the image.

9. A sparse coding-based similarity image retrieval system, comprising:

the first acquisition module is used for performing base vector representation according to the reference image to acquire a first sparse representation result;

the second acquisition module is used for performing base vector characterization according to the images in the image library to obtain a second sparse characterization result;

the calculation module is used for calculating the similarity between the first sparse representation result and the second sparse representation result;

and the judging module is used for judging whether the similarity is greater than a preset threshold value, if so, judging the image to be a similar image, and if not, judging the image to be a non-similar image.

10. The sparse coding-based similarity image retrieval system of claim 9, further comprising:

at least one memory for storing computer instructions;

at least one processor in communication with the memory, wherein the at least one processor, when executing the computer instructions, causes the system to perform: the device comprises a first acquisition module, a second acquisition module, a calculation module and a judgment module.

Images