CN106846339A - Image detection method and device - Google Patents

Abstract

The embodiment of the invention discloses an image detection method and device. The image detection method comprises the steps of obtaining an image to be detected, extracting a maximum stable extremum MSER region from the image to be detected, wherein the MSER region is a connected region, and filtering the MSER region to obtain a text region in the image to be detected. The MSER region is extracted from the image to be detected, the MSER region is extracted in a mode of dividing the connected region to serve as a candidate region, then the extracted MSER region is filtered and screened, and finally the text region in the image to be detected is obtained.

Description

An image detection method and device

technical field

The present invention relates to the technical field of image processing, in particular to an image detection method and device.

Background technique

With the maturity and popularization of digital camera equipment, people have been able to record all aspects of the real world under different perspectives very conveniently and quickly. As a human language, visual text has a special and irreplaceable position in human activities. Text detection in natural scenes is one of the important research topics of computer vision and pattern recognition technology in the field of target detection and recognition. The purpose of this technology is to accurately detect text information in the captured natural scene images. It has broad application prospects in natural scene understanding and analysis, robot-assisted navigation, video retrieval, assisted reading for the blind, and text translation.

Currently, natural scene text detection methods are divided into two types: methods based on sliding windows and methods based on connected regions.

The method based on the sliding window refers to sliding a multi-scale window from left to right and from top to bottom in the image, and classifying the image in the sliding window to determine whether it is a text area. In order to be able to detect all For the text area, this method usually requires a large number of sliding windows, which leads to increased computational complexity and cannot meet the real-time requirements.

The method based on connected regions refers to the similarity clustering of pixels according to the inherent properties of the text, such as color, texture, stroke width, etc., to generate a large number of connected regions, and perform characteristics on the connected regions (such as text height, width and spacing, etc.) extraction, filtering non-text areas, so as to complete the text detection, compared with the method based on the sliding window, the calculation amount of this method is relatively reduced, but it has high requirements for the extraction of connected regions, that is, the extracted connected regions To include all text areas, and it is difficult to deal effectively with complex backgrounds.

Contents of the invention

In order to solve related technical problems, the present invention provides an image detection method and device, which can quickly and accurately detect text regions in complex natural scenes.

In order to achieve the above purpose, the embodiment of the present invention adopts the following technical solutions:

In a first aspect, an embodiment of the present invention provides an image detection method, including:

Obtain the image to be detected;

Extracting a maximum stable extremum MSER region from the image to be detected, wherein the MSER region is a connected region;

Filter the MSER region to obtain the text region in the image to be detected.

In the second aspect, the embodiment of the present invention also provides an image detection device correspondingly, including:

The image acquisition module to be detected is used to obtain the image to be detected;

The MSER region extraction module is used to extract the maximum stable extremum MSER region from the image to be detected, wherein the MSER region is a connected region;

The MSER region filtering module is used to filter the MSER region to obtain the text region in the image to be detected.

Beneficial effects brought by the technical solutions provided by the embodiments of the present invention:

In the technical solution, the image to be detected is acquired, and the maximum stable extremum MSER area is extracted from the image to be detected, wherein the maximum stable extremum area is a connected area, and the MSER area is filtered to obtain the text area in the image to be detected. By extracting the MSER area from the image to be detected, the MSER area is extracted as a candidate area by dividing the connected area, and then the extracted MSER area is filtered and screened, and finally the text area in the image to be detected is obtained. The area division is conducive to reducing calculations. At the same time, extracting the MSER region can reduce the interference of the image background and improve the accuracy of detecting the image to be detected with a complex background.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the accompanying drawings that need to be used in the description of the embodiments of the present invention. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention , for those skilled in the art, other drawings can also be obtained according to the content of the embodiment of the present invention and these drawings without any creative effort.

FIG. 1 is a schematic flow chart of an image detection method provided in Embodiment 1 of the present invention;

FIG. 2A is a schematic flowchart of an image detection method provided in Embodiment 2 of the present invention;

FIG. 2B is a schematic flowchart of an optional implementation manner of S250 in FIG. 2A;

FIG. 2C is a schematic structural diagram of a convolutional neural network model used in Embodiment 2 of the present invention;

FIG. 3 is a schematic diagram of the structure of an image detection device provided in Embodiment 3 of the present invention;

FIG. 4A is a schematic structural diagram of an image detection device provided in Embodiment 4 of the present invention;

FIG. 4B is a schematic structural diagram of an alternative implementation of the MSER region filtering module 450 in FIG. 4A .

detailed description

In order to make the technical problems solved by the present invention, the technical solutions adopted and the technical effects achieved clearer, the technical solutions of the embodiments of the present invention will be further described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are only the technical solutions of the present invention. Some, but not all, embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative efforts fall within the protection scope of the present invention.

Embodiment one

Please refer to FIG. 1 , which is a schematic flowchart of an image detection method provided by Embodiment 1 of the present invention. The method of this embodiment can be executed by a mobile device such as a smart phone, a tablet computer, or a notebook computer equipped with a camera, and is applicable to detection and recognition of text regions in natural scene images.

An image detection method provided in this embodiment may include the following steps:

S110: Acquire an image to be detected.

Exemplarily, in the embodiment of the present invention, the image to be detected may be an original image, or may be an image obtained by preprocessing the original image. In one embodiment of the present invention, the original image is preferably preprocessed to obtain the image to be detected.

S120: Extract the maximum stable extremum MSER region from the image to be detected.

Exemplarily, the Maximally Stable Extrernal Regions (MSER) region refers to a connected region formed after the image to be detected undergoes a certain threshold change, and multiple MSER regions can be extracted from the image to be detected, and the connected regions can be Minimum enclosing rectangle to represent the MSER region. Among them, features such as color, texture, and character stroke width in the same connected region are basically the same.

Each rectangular frame displayed in the image to be detected represents an MSER region, and multiple MSER regions can be extracted from the image to be detected, or no MSER region can be extracted, that is, there is no text region in the image to be detected.

S130: Filter the MSER region to obtain a text region in the image to be detected.

Exemplarily, there are many methods for filtering MSER regions, for example, filtering according to regional characteristics of MSER regions. Embodiment 2 of the present invention provides an optional implementation manner of filtering MSER regions, which will not be repeated here.

In summary, in this technical solution, the image to be detected is obtained, and the maximum stable extremum MSER region is extracted from the image to be detected, wherein the maximum stable extremum region is a connected region, and the MSER region is filtered to obtain the text region in the image to be detected . By extracting the MSER area from the image to be detected, the MSER area is extracted as a candidate area by dividing the connected area, and then the extracted MSER area is filtered and screened, and finally the text area in the image to be detected is obtained. The area division is conducive to reducing calculations. At the same time, extracting the MSER region can reduce the interference of the image background and improve the accuracy of detecting images with complex backgrounds.

Embodiment two

Please refer to FIG. 2A, FIG. 2B and FIG. 2C, wherein FIG. 2A is a schematic flowchart of an image detection method provided in Embodiment 2 of the present invention, FIG. 2B is a schematic flowchart of an optional implementation of S250 in FIG. 2A, and FIG. 2C is a schematic structural diagram of the convolutional neural network model used in Embodiment 2 of the present invention. The main difference between this embodiment and the first embodiment is that the contents of S210, S220, S260 and S270 are added on the basis of the first embodiment, and an optional implementation manner of S250 is further provided.

An image detection method provided in this embodiment may include the following steps:

S210: Receive an initial image.

Exemplarily, the initial image may be an image obtained by shooting a natural scene with a camera, usually an RGB image.

S220: Perform color space conversion on the initial image to obtain an image to be detected.

Exemplarily, by performing color space conversion on the initial image, images of 7 channels of R, G, B, Grayscale, H, S, and V are obtained as images to be detected, and these 7 images are all processed in subsequent steps operate.

S230: Acquire an image to be detected.

S240: Extracting the maximum stable extremum MSER region from the image to be detected.

Exemplarily, the MSER region can be extracted from the image to be detected by the MSER algorithm. The main process is: binarize the image to be detected, and adjust the binarization threshold to change within the range of [0, 255]. When the area of the connected region changes When V(i) is less than the set change range value, it is determined that the connected region is an MSER region; for example, when binarizing the grayscale image of the detected image, all pixels whose pixel value is smaller than the binarization threshold are set to The pixel value is 0, and the pixel value is set to 255 for the pixel points whose pixel value is not less than the binarization threshold, then the corresponding binarized image will go through a process from completely black to completely white (like a bird's-eye view of the rising water level. Figure), in this process, the area of some connected regions changes very little with the change of the binarization threshold, that is, V(i) is less than the set change range value (such as 0.25), this connected region is the MSER region .

in, Q _i represents the area of the connected region when the binarization threshold is i; Δ represents the small change of the binarization threshold; the area change range V(i) represents the area change of the connected region when the binarization threshold value i changes slightly degree.

S250: Filter the MSER region to obtain a text region in the image to be detected.

Optionally, as shown in Figure 2B, filtering the MSER region may include four steps of S251, S252, S253 and S254, wherein:

S251: Count pixel values or area aspect ratios of the MSER area.

Exemplarily, in practical applications, there are almost no text images with less than 30 pixels in the captured natural scene images, and the aspect ratio of the general text area is also within a certain range, for example, the aspect ratio of the text area is usually In the range of 0.3-3, therefore, the non-text area in the MSER area can be preliminarily filtered according to the pixel value or the aspect ratio in the determined rectangular frame of the MSER area.

S252: Filter the MSER region whose pixel value is smaller than a preset pixel threshold or whose aspect ratio is not within a preset range.

Exemplarily, MSER regions with less than 30 pixels or region aspect ratios not in the range of 0.3-3 are filtered.

In addition, when a text area has multiple rectangles, in order to reduce the amount of calculation, one of the multiple rectangles can be selected to represent the text area. For example, for any rectangular frame A, when the ratio of the overlapping area of another rectangular frame B and rectangular frame A to the total area of the union of rectangular frame A and rectangular frame B is greater than 0.8, it is considered that rectangular frame A and rectangular frame Frame B is located at the same position and represents the same text area. Merge the rectangular frame A and the rectangular frame B, traverse all the remaining rectangular frames, merge the rectangular frame that meets the above merging conditions with the rectangular frame A, and also the image to be detected Similar operations can be performed on other rectangles in the frame, which can minimize the amount of subsequent calculations.

S253: Continuously perform convolution and down-sampling processing on the remaining MSER regions after filtering to obtain a feature map.

Exemplarily, in this embodiment, the binarized image extracted from the MSER region is used to train the convolutional neural network model. As shown in Figure 2C, first input a 32*32 image, and convolve the input image through six 5*5 kernel matrices to obtain six 28*28 feature maps of the C1 layer; The map is down-sampled, and every 4 pixels (2*2) get a value, and then six 14*14 feature maps of the S2 layer are obtained; then the feature map of the S2 layer is processed using a 5*5 kernel matrix Convolve to obtain 16 feature maps of 10*10 in C3 layer; similarly to S2, downsampling the feature map of C3 layer can obtain 16 feature maps of 5*5 in S4 layer; use The 5*5 kernel matrix convolutes the feature map of the S4 layer to obtain 120 1*1 feature maps of the C5 layer; similarly, downsampling the feature map of the C5 layer can obtain the F6 layer The 84 1*1 feature maps.

S254: Input the feature map into the classifier, and determine the MSER region as the text region according to the output result of the classifier.

Exemplarily, the feature map of the F6 layer obtained in S253 above is input into the softmax classifier, and the input image is determined to be a text image according to the output result of the softmax classifier, and the corresponding MSER region is a text region. In other embodiments, other classifiers such as SVM can also be used.

After the convolutional neural network model classifies the MSERA area, the area of a single character or text in the image to be detected can be basically determined, the rectangular frame of the non-text area is basically filtered, and the rectangular frame of the text area is retained.

S260: Merge adjacent text regions in the horizontal direction.

Exemplarily, for an image to be detected that contains English words, it is also necessary to combine each character combination into a word. Calculate the distance between all adjacent character regions, and calculate the average distance; find the leftmost character region that has not been processed, and then look for the character region closest to the character region in the horizontal direction, when two adjacent characters When the height ratio of the region is within the preset height ratio range, for example, when the height ratio is between 0.5-2, the two character regions are merged, and when the distance between two adjacent character regions is greater than the set distance ( When it is 3 times of the above average distance), the iteration is stopped, so that the text area in the same line can be divided.

S270: Perform intra-region word segmentation on the merged text region.

Exemplarily, for each group of text regions merged by the above S260, if the distance between two adjacent character regions is greater than the above-mentioned average distance, then the adjacent two character regions are divided, so that the same different words in the line.

Repeat S260 and S270 until all text areas are processed.

It should be noted that, the MSER region mentioned in the embodiment of the present invention also means a region image corresponding to the MSER region.

To sum up, in this technical solution, the initial image is received, the color space conversion is performed on the initial image, the image to be detected is obtained, and the maximum stable extremum MSER region is extracted from the image to be detected, wherein the maximum stable extremum region is a connected region, Filter the MSER region to obtain the text region in the image to be detected, and further merge the text region between regions and segment words within the region. By extracting the MSER area from the image to be detected, the MSER area is extracted as a candidate area by dividing the connected area, and then the extracted MSER area is filtered and screened, and finally the text area in the image to be detected is obtained. The area division is conducive to reducing calculations. At the same time, extracting the MSER region can reduce the interference of the image background and improve the accuracy of detecting images with complex backgrounds.

The following is an embodiment of an image detection device provided by the embodiment of the present invention. The image detection device and the above-mentioned image detection method belong to the same inventive concept. For the details not described in detail in the embodiment of the device, please refer to the implementation of the above method example.

Embodiment three

Please refer to FIG. 3 , which is a schematic structural diagram of an image detection device provided by Embodiment 3 of the present invention.

An image detection device 300 provided in this embodiment may include the following:

The image to be detected acquisition module 310 is configured to acquire an image to be detected.

The MSER region extraction module 320 is configured to extract the maximum stable extremum MSER region from the image to be detected, wherein the MSER region is a connected region.

The MSER region filtering module 330 is configured to filter the MSER region to obtain a text region in the image to be detected.

In summary, in this technical solution, the image to be detected is obtained, and the maximum stable extremum MSER region is extracted from the image to be detected, wherein the maximum stable extremum region is a connected region, and the MSER region is filtered to obtain the text region in the image to be detected . By extracting the MSER area from the image to be detected, the MSER area is extracted as a candidate area by dividing the connected area, and then the extracted MSER area is filtered and screened, and finally the text area in the image to be detected is obtained. The area division is conducive to reducing calculations. At the same time, extracting the MSER region can reduce the interference of the image background and improve the accuracy of detecting images with complex backgrounds.

Embodiment Four

Please refer to FIG. 4A and FIG. 4B , wherein FIG. 4A is a schematic structural diagram of an image detection device provided in Embodiment 4 of the present invention, and FIG. 4B is a schematic structural schematic diagram of an optional implementation of the MSER region filtering module 450 in FIG. 4A . The main difference between this embodiment and the third embodiment is that on the basis of the third embodiment, the content of the initial image receiving module 410, the color space conversion module 420, the text region merging module 460 and the word segmentation module 470 are added, and further provides An optional implementation of the MSER region filtering module 450 .

An image detection device 400 provided in this embodiment may include the following:

The initial image receiving module 410 is configured to receive an initial image.

The color space conversion module 420 is configured to perform color space conversion on the initial image to obtain the image to be detected.

The image to be detected acquisition module 430 is configured to acquire an image to be detected.

The MSER region extraction module 440 is configured to extract the maximum stable extremum MSER region from the image to be detected, wherein the MSER region is a connected region.

Preferably, the MSER region extraction module 440 is specifically used for:

Carry out binarization processing on the image to be detected, adjust the binarization threshold to change within the range of [0, 255], when the area change range V(i) of the connected region is less than the set change range value, determine the connected region as the MSER region;

in, Q _i represents the area of the connected region when the binarization threshold is i, and Δ represents the small change of the binarization threshold.

The MSER region filtering module 450 is configured to filter the MSER region to obtain the text region in the image to be detected.

Optionally, as shown in FIG. 4B, the MSER region filtering module 450 may include a statistical unit 451, a filtering unit 452, a feature map obtaining unit 453, and a text region determining unit 454, wherein:

A statistical unit 451, configured to count pixel values or area aspect ratios of the MSER area.

The filtering unit 452 is configured to filter MSER regions whose pixel values are smaller than a preset pixel threshold or whose aspect ratio is not within a preset range.

The feature map obtaining unit 453 is configured to continuously perform convolution and downsampling processing on the remaining MSER regions after filtering to obtain a feature map.

The text region determination unit 454 is configured to input the feature map into the classifier, and determine the MSER region as the text region according to the output result of the classifier.

The text area merging module 460 is used for merging adjacent text areas in the horizontal direction.

The word segmentation module 470 is configured to perform intra-region word segmentation on the merged text region.

To sum up, in this technical solution, the initial image is received, the color space conversion is performed on the initial image, the image to be detected is obtained, and the maximum stable extremum MSER region is extracted from the image to be detected, wherein the maximum stable extremum region is a connected region, Filter the MSER region to obtain the text region in the image to be detected, and further merge the text region between regions and segment words within the region. By extracting the MSER region from the image to be detected, the MSER region is extracted as a candidate region by dividing the connected region, and then the extracted MSER region is filtered and screened, and finally the text region in the image to be detected is obtained. Region division is beneficial to reduce calculation At the same time, extracting the MSER region can reduce the interference of the image background and improve the accuracy of detecting images with complex backgrounds.

Note that the above are only preferred embodiments of the present invention and applied technical principles. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the protection scope of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and can also include more other equivalent embodiments without departing from the concept of the present invention, and the present invention The scope is determined by the scope of the appended claims.

Claims (10)

1. An image detection method, characterized in that, comprising: Obtain the image to be detected; Extracting a maximum stable extremum MSER region from the image to be detected, wherein the MSER region is a connected region; Filter the MSER region to obtain the text region in the image to be detected. 2. The method according to claim 1, wherein, before receiving the image to be detected, further comprising: receive the initial image; Perform color space conversion on the initial image to obtain the image to be detected. 3. The method according to claim 2, wherein the extracting the maximum stable extremum MSER region from the image to be detected comprises: Perform binarization processing on the image to be detected, adjust the binarization threshold to change within the range of [0, 255], and determine the The connected area is the MSER area; in, Q _i represents the area of the connected region when the binarization threshold is i, and Δ represents a slight change of the binarization threshold. 4. The method according to claim 3, wherein said filtering said MSER region to obtain the text region in said image to be detected comprises: Counting pixel values or area aspect ratios of the MSER area; Filter out MSER regions whose pixel values are smaller than a preset pixel threshold or whose aspect ratio is not within a preset range. 5. The method according to claim 4, wherein after filtering the MSER region whose pixel value is smaller than a preset pixel threshold or whose region aspect ratio is not within a preset range, further comprising: Continuously perform convolution and downsampling processing on the remaining MSER regions after filtering to obtain feature maps; The feature map is input into a classifier, and the MSER region is determined as a text region according to the output result of the classifier. 6. The method according to any one of claims 1-5, wherein, after filtering the MSER region and obtaining the text region in the image to be detected, further comprising: Merge adjacent text areas horizontally; Intra-region word segmentation is performed on the merged text regions. 7. An image detection device, characterized in that it comprises: The image acquisition module to be detected is used to obtain the image to be detected; The MSER region extraction module is used to extract the maximum stable extremum MSER region from the image to be detected, wherein the MSER region is a connected region; The MSER region filtering module is used to filter the MSER region to obtain the text region in the image to be detected. 8. The device of claim 7, further comprising: An initial image receiving module, configured to receive an initial image; a color space conversion module, configured to perform color space conversion on the initial image to obtain the image to be detected; The text area merging module is used for merging adjacent text areas in the horizontal direction; A word segmentation module for intra-region word segmentation on merged text regions. 9. The device according to claim 8, wherein the MSER region extraction module is specifically used for: Perform binarization processing on the image to be detected, adjust the binarization threshold to change within the range of [0, 255], and determine the The connected area is the MSER area; in, Q _i represents the area of the connected region when the binarization threshold is i, and Δ represents a slight change of the binarization threshold. 10. The device according to claim 9, wherein the MSER region filtering module comprises: A statistical unit, used to count pixel values or area aspect ratios of the MSER area; A filtering unit, configured to filter MSER regions whose pixel values are smaller than a preset pixel threshold or whose aspect ratio is not within a preset range; The feature map obtaining unit is used to continuously perform convolution and down-sampling processing on the remaining MSER regions after filtering to obtain the feature map; A text region determining unit, configured to input the feature map into a classifier, and determine the MSER region as a text region according to an output result of the classifier.