KR101390561B1 - Method and apparatus for subtitles detection - Google Patents

Abstract

A subtitle detection method and apparatus therefor are disclosed. The subtitle detection method includes: converting an input image into a gray scale image; Detecting a first caption candidate region by applying a Harris corner detection algorithm to the gray-scale transformed image; Performing a discrete cosine transform on the gray-scale-converted image to detect a second caption candidate region; And determining an overlapping area between the first caption candidate region and the second caption candidate region as a caption region.

Description

TECHNICAL FIELD The present invention relates to a method and apparatus for detecting subtitles,

The present invention relates to a method and apparatus for more efficiently detecting subtitles in an input image including subtitles.

Recently, the interest of the media industry is growing as various types of multimedia service technologies are developed. One of the most urgent technologies is to use embedded subtitles in video content. The caption in this image provides a viewing environment convenient for viewers and various information about the contents. It is a useful part to convey information directly to the parts that are not heard directly, as it serves as an effective supplementary function that delivers the contents of news reports or documentary images in concrete texts. Such embedded subtitles are useful for techniques such as video information search, image search, and scene search.

Conventionally, caption regions are extracted considering texture base, color base, and edge characteristics. The texture-based caption extraction method extracts the caption area using the characteristics of the whole image. It extracts feature points of the caption using the Harris corner detector. And the similarity is measured to detect the caption area. However, the detection of the caption area using the texture-based method has a problem in that not only the environment variable is varied and complex, but also the area of the texture type included in the background of the image is detected, resulting in erroneous results.

The color-based caption region detection detects the caption region using the characteristic that the caption is distributed in a specific color band. Since the caption in the image is in the same color and brightness region and there is a contrast difference between the caption and the background, And changing the value. Such color-based caption region detection has a problem in that color information values are lost or color values in an original image are difficult to maintain when moving picture compression techniques are developed.

Finally, a method of extracting a caption area by considering an edge characteristic generally detects a caption by performing various detectors for extracting an edge mask and an edge map. However, such a method can efficiently detect images that do not have a complicated structure, but has a disadvantage in that an additional processing process is required in the case of a complicated structure image and the accuracy is also deteriorated.

The present invention provides a subtitle detection method and apparatus capable of accurately detecting a subtitle area even in an image having a complex structure background.

According to an aspect of the present invention, there is provided a method of accurately and efficiently detecting subtitles in an image and a recording medium on which a program for performing the method is recorded.

According to an embodiment of the present invention, there is provided a method of converting an input image into a gray scale image, Detecting a first caption candidate region by applying a Harris corner detection algorithm to the gray-scale transformed image; Performing a discrete cosine transform on the gray-scale-converted image to detect a second caption candidate region; And determining a region overlapping the first caption candidate region and the second caption candidate region as a caption region.

Wherein the step of detecting the first caption candidate region comprises: deriving brightness average and standard deviation of the gray-scale-converted image; Normalizing the image according to the normal distribution value using the brightness average and the standard deviation; Detecting the corner points by applying the Harris corner detection algorithm to the normalized image; Blurring an image in which the corner points are detected to remove noise; And labeling adjacent regions of the noise-removed corner points to detect the first caption candidate region.

The step of detecting the second caption candidate region may include: normalizing the discrete cosine transformed image by applying the discrete cosine transformed image to a sine function; Performing inverse discrete cosine transform on the normalized image; Applying the Gaussian filter to the inverse discrete cosine transformed image; And binarizing the Gaussian filtered image to a threshold value to detect the second caption candidate region.

The threshold value is a value obtained by summing the average and variance of the Gaussian filtered image.

In the case where the Gaussian filtered image is transformed into a first value when each pixel value of the Gaussian filtered image is greater than or equal to the threshold value and each pixel value is converted into a second value when the pixel value is less than the threshold value, Can be converted and binarized.

The detecting of the second caption candidate region may detect the second caption candidate region by labeling an adjacent region of the pixel values corresponding to the second value.

According to another aspect of the present invention, there is provided an apparatus for accurately and efficiently detecting subtitles in an image.

According to an embodiment of the present invention, there is provided an image processing apparatus including a preprocessing unit for converting an input image into a gray scale image; A first signal processing unit for detecting a first caption candidate region by applying a Harris corner detection algorithm to the gray-scale converted image; A second signal processing unit for performing a discrete cosine transform on the gray-scale-converted image to detect a second caption candidate region; And a subtitle detection unit for detecting a region overlapping the first subtitle candidate region and the second subtitle candidate region as a subtitle region.

Wherein the first signal processor derives brightness average and standard deviation of the gray-scale-converted image, normalizes the brightness average and standard deviation of the image into an image according to a normal distribution value using the brightness average and standard deviation, Algorithm is applied to detect corner points, blurring is performed, and adjacent regions of the corner points are labeled to detect the first caption candidate region.

Wherein the second signal processor performs an inverse discrete cosine transform by applying the discrete cosine transformed image to a sine function to normalize the transformed image and applies the Gaussian filter to the inverse discrete cosine transformed image to binarize the inverse discrete cosine transformed image into a threshold value, 2 subtitle candidate area can be detected.

There is an advantage that the caption can be efficiently and accurately detected even in an image having a complicated structure background by providing the caption detection method and apparatus according to an embodiment of the present invention.

1 is a block diagram schematically showing an internal configuration of a caption detecting apparatus according to an embodiment of the present invention;
FIG. 2 is a diagram for comparing a caption candidate region detection result through a conventional method and a first signal processing unit according to an embodiment of the present invention; FIG.
FIG. 3 is a view for explaining caption candidate region detection through a first signal processing unit according to an embodiment of the present invention; FIG.
FIG. 4 is a diagram for explaining caption candidate region detection through a second signal processing unit according to an embodiment of the present invention; FIG.
FIG. 5 is a diagram for explaining subtitle detection result according to an embodiment of the present invention; FIG.
6 is a flowchart illustrating a method of detecting a caption in a caption detection apparatus according to an exemplary embodiment of the present invention.
7 is a table comparing subtitle detection performance according to an embodiment of the present invention with a conventional method.

BRIEF DESCRIPTION OF THE DRAWINGS The present invention is capable of various modifications and various embodiments, and specific embodiments are illustrated in the drawings and described in detail in the detailed description. It is to be understood, however, that the invention is not to be limited to the specific embodiments, but includes all modifications, equivalents, and alternatives falling within the spirit and scope of the invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, the present invention will be described in detail with reference to the accompanying drawings.

The terms first, second, etc. may be used to describe various components, but the components should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from another.

The terminology used in this application is used only to describe a specific embodiment and is not intended to limit the invention. The singular expressions include plural expressions unless the context clearly dictates otherwise. In the present application, the terms "comprises" or "having" and the like are used to specify that there is a feature, a number, a step, an operation, an element, a component or a combination thereof described in the specification, But do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, or combinations thereof.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

FIG. 1 is a block diagram schematically showing the internal structure of a caption detection apparatus according to an embodiment of the present invention. FIG. 2 is a block diagram illustrating a conventional method according to an embodiment of the present invention and a caption candidate region detection FIG. 3 is a view for explaining caption candidate region detection through a first signal processing unit according to an embodiment of the present invention, and FIG. 4 is a view for explaining a method for detecting a caption candidate region according to an embodiment of the present invention FIG. 5 is a view for explaining caption candidate detection using a second signal processing unit according to an embodiment of the present invention, and FIG. 5 is a view for explaining caption detection result according to an embodiment of the present invention.

Referring to FIG. 1, a caption detection apparatus 100 according to an embodiment of the present invention includes a preprocessing unit 110, a first signal processing unit 120, a second signal processing unit 130, and a caption detection unit 140 .

The preprocessing unit 110 receives the input image, converts the input image into a gray-scale image, and outputs the gray-scale image to the first signal processing unit 120 and the second signal processing unit 130, respectively. At this time, the preprocessing unit 110 may convert the input image input in a moving image format to gray scale conversion for each frame, and output it to the first signal processing unit 120 and the second signal processing unit 130, respectively.

The first signal processing unit 120 is a unit for normalizing the gray scale image input through the preprocessing unit 110 and then detecting a caption candidate region using a Harris corner subtitle detection algorithm. In this specification, the caption candidate region detected by the first signal processing unit 120 is referred to as a first caption candidate region in order to facilitate understanding and explanation.

Generally, subtitles embedded in an image are significantly different in brightness and color from background or other objects. The average and standard deviation of the brightness of the grayscale image are calculated using the characteristics of the subtitles, and the normalization process is performed on the image according to the normal distribution value.

For example, the first signal processor 120 may normalize a gray-scale image using Equation (1).

Here, I represents the pixel value of the gray-scale-converted image, and AVG () represents the average brightness value of the gray-scale-converted image. Also, STD () represents the standard deviation obtained by using the brightness value of the gray-scale-converted image, and O (x, y) represents the resulting pixel values obtained by using the standardization of the standard normal distribution calculation equation.

Using the normalized image, the first signal processing unit 120 detects the corner points using the Harris corner detection algorithm. In the case of caption, Harris corner detection algorithm uses the feature that contains many corner points compared with other objects because the distribution area of the corner point is concentrated theoretically. Since the Harris corner detection algorithm is obvious to those skilled in the art, the description of the Harris corner detection algorithm itself will be omitted.

The first signal processing unit 120 detects corner points by applying the Harris corner detection algorithm, blurring an image to which a Harris corner detection algorithm is applied to minimize a candidate region to be detected as a caption candidate region, The caption candidate region can be detected.

In FIG. 2, reference numeral 210 denotes an image obtained by applying the Harris corner detection algorithm without normalization to the original input image, and reference numeral 220 denotes a result image obtained by applying the Harris corner detection algorithm after normalizing the input image.

As can be seen from the result image of FIG. 2, when the background area has a complicated structure or a texture similar to a caption is included, it can be seen that corner points are detected by variously distributing when the Harris corner detection algorithm is applied.

On the other hand, as can be seen from the result image 220, when the Harris corner detection algorithm is applied after normalizing the gray-scale-converted image like the first signal processing unit 120, except for a small number of corner points due to small noise It can be seen that the corner points are distributed only in the caption area. The first signal processing unit 120 can derive a caption candidate region by labeling corner points between adjacent regions.

3, reference numeral 310 denotes an original input image, 320 denotes a gray scale image normalized by the first signal processing unit 120, 330 denotes a gray scale image normalized by the first signal processing unit 120, The results of applying the detection algorithm are shown.

The second signal processor 130 is a means for detecting a candidate region of a subtitle by performing a discrete cosine (DCT) transformation on the gray-scale-transformed image.

First, the second signal processing unit 130 performs DCT conversion on the gray-scale-converted image (420 in FIG. 4 indicates a result of DCT conversion of a part of the input image). Next, the second signal processing unit 130 applies a sine function (sign ()) to reduce the low frequency component using the characteristic that the caption area has a contrast and a high frequency component compared to the background or other object areas, (See 420 in Fig. 4). Then, the second signal processor 130 performs inverse discrete cosine transform on the normalized image and applies blurring by applying a Gaussian filter (see 440 in FIG. 4) to remove the noise, The subtitle candidate region can be detected using the threshold value (see 450 in FIG. 4).

For example, the second signal processing unit 130 can detect a subtitle candidate region according to the DCT transform using Equation (2).

,

,

,

,

Here, I represents a grayscale-converted image, and x and y represent X-axis and Y-axis pixels of the image. sign () is a sine function to reduce the low-frequency component. It is a function to return -1 as a negative value, 0 as a zero value, and 1 as a positive value.

abs () denotes a function for returning an absolute value, G denotes a Gaussian filter, and DCT ^-1 denotes an inverse discrete cosine transform. It will be obvious to those skilled in the art that a separate description thereof will be omitted.

The second signal processing unit 130 may binarize a caption candidate region in an image to which a Gaussian filter is applied, derive a threshold value using an average and variance, and detect a caption candidate region using a threshold value. For example, the second signal processing unit 130 may convert pixel values of a threshold value or more to a first value (e.g., 255) in the Gaussian filtered image, and pixel values of less than a threshold value may be converted to a second value , 0), and then pixel values corresponding to the second value can be detected as a caption candidate region. At this time, the second signal processing unit 130 can detect the caption candidate regions by connecting adjacent pixels between the detected caption candidate regions and labeling them.

In this specification, the caption candidate region detected by the second signal processing unit 130 is referred to as a second caption candidate region in order to facilitate understanding and explanation.

The caption detection unit 140 receives the first caption candidate region and the second caption candidate region from the first signal processing unit 120 and the second signal processing unit 130, respectively, .

In FIG. 5, 510 indicates a result of detecting a candidate region of a caption using the DCT in the second signal processing unit 130, and 530 indicates a caption candidate region labeled by connecting adjacent pixels. Reference numeral 520 denotes corner points detected by the first signal processing unit 120 using the Harris corner detection algorithm, and reference numeral 540 denotes a caption candidate region labeled with corner points. As shown in FIG. 5, the first caption candidate region and the second caption candidate region detected by the first signal processing unit 120 and the second signal processing unit 130 are different from each other.

Accordingly, the subtitle detection unit 140 compares the first subtitle candidate region and the second subtitle candidate region detected by the first signal processing unit 120 and the second signal processing unit 130, Area can be detected as the caption area.

FIG. 6 is a flowchart illustrating a method of detecting a caption in a caption detection apparatus according to an exemplary embodiment of the present invention. FIG. 7 is a table comparing caption detection performance according to an exemplary embodiment of the present invention. Each of the steps described below is performed by the internal components of each of the caption detecting devices, but will be collectively referred to as a caption detecting device in order to facilitate understanding and explanation.

In step 610, the subtitle detection apparatus 100 receives an input image. Here, the input image may be in the form of a moving image.

In step 615, the caption detecting apparatus 100 converts the inputted input image into a gray-scale converted frame unit.

Subsequently, the subtitle detection apparatus 100 can detect each of the caption candidate regions by performing 601 and 602 in parallel. Hereinafter, a method of detecting a candidate region of a caption using the 601 Harris corner detection algorithm will be described in order to facilitate understanding and explanation.

In operation 620, the subtitle detection apparatus 100 derives a brightness average and a standard deviation of the gray-scale-converted image.

In step 625, the caption detection apparatus 100 normalizes the image according to the normal distribution value using the derived mean and standard deviation.

In step 630, the subtitle detection apparatus 100 detects the corner points using the normalized gray-scale image using the Harris corner detection algorithm.

In step 635, the subtitles detecting apparatus 100 labels the detected corner points among adjacent areas to detect a caption candidate region (referred to as a first caption candidate region).

For example, the caption detection apparatus 100 may detect the first caption candidate region by blurring the detected corner points to remove noise, and then label the adjacent regions with each other.

Next, a description will be made of a method 602 for detecting a candidate region of a caption using DCT.

In operation 640, the subtitle detection apparatus 100 performs DCT conversion on the gray-scale-converted image.

Subsequently, in step 645, the subtitle detection apparatus 100 normalizes the DCT-transformed image by applying a sine function.

In operation 650, the subtitle detection apparatus 100 performs the IDCT transformation again on the normalized image.

Subsequently, in step 655, the subtitle detection apparatus 100 performs blurring by applying a Gaussian filter to the IDCT-transformed image, determines a threshold value using an average and variance, and detects a candidate region using the threshold value . At this time, the caption detecting apparatus 100 can detect the caption candidate region (second caption candidate region) by labeling through the adjacent pixel connection between the candidate regions.

In step 660, the caption detecting apparatus 100 compares the first caption candidate region and the second caption candidate region, and detects an overlapping region as a final caption region.

FIG. 7 is a table comparing a conventional method and subtitle detection performance according to an embodiment of the present invention. FIG. 7 shows the subtitle detection accuracy and efficiency using Equation (3). Referring to FIG. 7, it can be seen that subtitle detection in a drama image in which news, documentary, and subtitles are inserted in which the subtitle structure and background can be easily distinguished are detected more efficiently than the conventional method.

Meanwhile, the caption detection method according to an embodiment of the present invention may be implemented in a form of a program command that can be executed through a variety of means for processing information electronically and recorded in a storage medium. The storage medium may include program instructions, data files, data structures, and the like, alone or in combination.

Program instructions to be recorded on the storage medium may be those specially designed and constructed for the present invention or may be available to those skilled in the art of software. Examples of storage media include magnetic media such as hard disks, floppy disks and magnetic tape, optical media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, magneto-optical media and hardware devices specifically configured to store and execute program instructions such as ROM, RAM, flash memory, and the like. Examples of program instructions include machine language code such as those produced by a compiler, as well as devices for processing information electronically using an interpreter or the like, for example, a high-level language code that can be executed by a computer.

The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa.

It will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the spirit or scope of the invention as defined in the appended claims. It will be understood that the invention may be varied and varied without departing from the scope of the invention.

100: Subtitle detection device
110:
120: first signal processor
130: second signal processor
140:

Claims (10)

Converting an input image into a gray scale image;
Detecting a first caption candidate region by applying a Harris corner detection algorithm to the gray-scale transformed image;
Performing a discrete cosine transform on the gray-scale-converted image to detect a second caption candidate region; And
And determining a region overlapping the first caption candidate region and the second caption candidate region as a caption region.
The method according to claim 1,
Wherein the step of detecting the first caption candidate region comprises:
Deriving brightness average and standard deviation of the gray-scale converted image;
Normalizing the image according to the normal distribution value using the brightness average and the standard deviation;
Detecting the corner points by applying the Harris corner detection algorithm to the normalized image;
Blurring an image in which the corner points are detected to remove noise; And
And labeling neighboring regions of the noise-removed corner points to detect the first caption candidate region.
The method according to claim 1,
Wherein the step of detecting the second caption candidate region comprises:
Performing normalization by applying the discrete cosine transformed image to a sine function;
Performing inverse discrete cosine transform on the normalized image;
Applying the Gaussian filter to the inverse discrete cosine transformed image;
And binarizing the Gaussian filtered image to a threshold value to detect the second caption candidate region.
The method of claim 3,
Wherein the threshold value is a value obtained by summing an average and variance of the Gaussian filtered image.
The method of claim 3,
In order to encode the Gaussian filtered image to a threshold value,
Wherein if the pixel value of the Gaussian filtered image is greater than or equal to the threshold value, the pixel value is converted to a first value, and if the pixel value is less than the threshold value, the pixel value is converted to a second value to binarize the pixel value.
5. The method of claim 4,
Wherein the step of detecting the second caption candidate region comprises:
And labeling the adjacent region of the pixel values corresponding to the second value to detect the second caption candidate region.
A recording medium on which a program for performing a subtitle detection method according to any one of claims 1 to 6 is recorded.
A preprocessing unit for converting an input image into a gray scale image;
A first signal processing unit for detecting a first caption candidate region by applying a Harris corner detection algorithm to the gray-scale converted image;
A second signal processing unit for performing a discrete cosine transform on the gray-scale-converted image to detect a second caption candidate region; And
And a subtitle detection unit for detecting a region overlapping the first subtitle candidate region and the second subtitle candidate region as a subtitle region.
9. The method of claim 8,
Wherein the first signal processor comprises:
The brightness average and standard deviation of the gray-scale-converted image are derived, normalized to an image according to the normal distribution value using the brightness average and standard deviation, and the normalized image is subjected to the Harris corner detection algorithm to calculate corner points And detects the first caption candidate region by performing labeling of neighboring regions of the corner point after blurring.
9. The method of claim 8,
Wherein the second signal processing unit comprises:
The image subjected to the discrete cosine transform is applied to a sine function to perform an inverse discrete cosine transform by normalizing the transformed image, and the inverse discrete cosine transform is applied to a Gaussian filter to binarize the inverse discrete cosine transformed image to a threshold value to detect the second subtitle candidate region And the subtitles are detected.

Images