JP2006331057A - Character information extraction device, character information extraction method, and computer program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To suitably extract an italic character drawn in a natural image such as a telop included in a television image as character information capable of character recognition processing.
A telop performs YUV conversion of an input image using the feature that the luminance is high with respect to the background, and performs edge calculation for the Y component. Further, binarization is performed to leave only the edge of the character portion. Subsequently, the edge image is degenerated and thinned, and a straight line is detected from the thinned edge image. By thinning the edge image, false detection can be reduced by right detection. Then, the average of the detected straight line inclinations is used as the inclination angle of the italic character.
[Selection] Figure 1

Description

  The present invention relates to a character information extraction device, a character information extraction method, and a computer program for extracting character information drawn in an image, and is particularly included in an image frame obtained by recording a television broadcast, for example. The present invention relates to a character information extraction apparatus, a character information extraction method, and a computer program for extracting character information from a telop.

  More specifically, the present invention relates to a character information extraction apparatus, a character information extraction method, and a computer program that extract character information drawn in an image in a form that allows character recognition processing by a general OCR engine. In particular, the present invention relates to a character information extraction device, a character information extraction method, and a computer program that extract a telop drawn as italic characters on a television image as character information that can be subjected to character recognition processing.

  In the modern information civilized society, the role of broadcasting is immeasurable. In particular, the influence of television broadcasting that delivers video information directly to viewers along with sound is significant. Broadcast technology includes a wide range of technologies such as signal processing, transmission / reception thereof, and information processing of audio and video. The penetration rate of television is extremely high, and it is installed in almost all households. Broadcast content distributed from each broadcasting station is viewed by an unspecified number of people.

  Recently, as another form of viewing broadcast content, a viewing form in which received content is once recorded with a hard disk recorder or the like and played back at a desired time is becoming common. Here, as the capacity of the hard disk increases, program recording for several tens of hours is possible, so it is almost impossible for the user to view all the recorded video content. Therefore, the style of searching for only scenes that the user is interested in and viewing the digest is more efficient and also makes effective use of the recorded content.

  In order to perform scene search and digest viewing in such recorded content, it is necessary to index the video.

  For example, in television broadcasting such as news programs and variety programs, it is common to display telops that express program topics explicitly or implicitly at the four corners of the frame as a method of program production and editing. ing. The telop displayed in the frame is an important clue for specifying or estimating the topic of the broadcast program in the display section. Therefore, it is considered possible to perform video indexing using character information obtained by extracting a telop from video content and character-recognizing the display content of the telop as an index.

  For example, a broadcast program content menu creation device that detects a telop in a frame as a feature image portion, extracts video data of only the telop and automatically creates a menu indicating the content of the broadcast program has been proposed (for example, , See Patent Document 1).

  Usually, in order to extract character information embedded in an image, it is necessary to perform character recognition using an OCR (Optical Character Reader) engine that recognizes character data from optical image data. For example, there has been proposed a telop information processing apparatus that detects a region where a telop is displayed from a video, extracts only pixels constituting a telop character, and performs recognition by OCR processing (see, for example, Patent Document 2). See

  However, while italics are often used for telops in TV images, existing OCR engines do not support italic characters. For this reason, most telops cannot be automatically recognized, and there is a problem that effective video indexing is performed.

  There are two main methods for resolving italic characters: the OCR engine supports italics, or italics are converted to upright characters.

  The former solution is not practical because the scale of the dictionary to be prepared by the OCR engine becomes very large, and the recognition process takes time in proportion to the dictionary size.

  Further, as the latter solution, the following methods are specifically mentioned.

(1) Method using edge direction (refer to Non-Patent Document 1)
(2) Method for detecting a vertical stroke using a character outline (refer to Non-Patent Document 2 and Non-Patent Document 3)
(3) Method of actually carrying out shearing treatment (see Non-Patent Document 4)

  Here, in the method of performing the shearing process, the shearing process is performed by limiting the angle, each processing result is weighted, the angle at which the weight is the largest is adopted as the inclination angle of the italic character, and the character inclination is corrected. Perform processing.

  However, many of the conventional character recognition technologies are based on recognizing black characters formed on a white background such as printed matter, and are not suitable for recognizing characters embedded in natural images such as telops in TV images. There is a problem. It is rare that the background of a telop is as uniform as a white background.

JP 2004-364234 A JP 2001-285716 A C. Sun and D.C. Si, “Skew and Sland Correction for Document Images Using Gradient Direction” (Proc. ICDAR, vol1, pp142-146, Aug. 1997). G. Kim and V.M. Govindaraju, “A lexicon driven approach to handwritten word recognition for real-time applications” (IEEE Trans. PAMI, Vol. 19, pp 366-379, April 367, 1997). F. Kimura, M .; Shridhar, and Z.M. Chen, “Improvements of a lexicon directed algorithm for recognition of unconstrained handwritten words” (Proc. 2nd ICDAR., Pp. 18-22, Oct. 1993). E. Kavallierato et al, “New algorithms for skewing correction and slant removal on word-level” (Proc. 6th ICECS, vol. 2, Sep. 1999).

  An object of the present invention is to provide an excellent character information extraction device, character information extraction method, and computer that can extract character information contained in an image frame in a form that allows character recognition processing by a general OCR engine.・ To provide a program.

  A further object of the present invention is to provide an excellent character information extraction apparatus and character information capable of suitably extracting characters drawn in a natural image such as a telop included in a television image as character information capable of character recognition processing. To provide an extraction method and a computer program.

  A further object of the present invention is to provide an excellent character information extraction apparatus and character that can suitably extract italic characters drawn in a natural image such as a telop included in a television image as character information that can be subjected to character recognition processing. To provide an information extraction method and a computer program.

The present invention has been made in consideration of the above problems, and a first aspect thereof is a character information extraction device that extracts character information drawn in an input image,
When the character information is an italic character, when calculating the inclination angle of the character information in order to perform character recognition after correcting the character inclination and converting it to an upright character,
Edge information detection means for detecting edge information of the input image;
Binarization processing means for binarizing the edge image detected by the edge information detection means using a predetermined threshold;
Thinning processing means for reducing and thinning the edge image binarized by the binarizing processing means to a line segment having a predetermined thickness while maintaining connectivity;
A straight line detecting means for detecting a straight line from the thinned edge image;
An inclination angle calculating means for calculating an inclination angle of character information using a straight line detected by the straight line detecting means;
Is a character information extracting device.

  The present invention relates to a character information extraction device that extracts character information drawn in an image. The present invention is particularly applied when extracting text information of a telop from a television image.

  Here, it is a problem in character recognition that italic characters are often used for telops drawn in a television image and that character information is drawn in a natural image. As a method for making italic characters processable by a general OCR engine, italic characters may be converted into upright characters. However, the conventional italic character conversion method is used for italic characters drawn in a natural image like a telop. Not suitable for processing.

  Therefore, the character information extracting apparatus according to the present invention first detects edge information from an input image and then binarizes and extracts character edge information when calculating the inclination angle of a type drawn in a natural image. Subsequently, the edge image is degenerated and thinned, a straight line is detected from the thinned edge image, and the inclination of the detected straight line is used as the inclination angle of the italic character.

  The telop uses the feature that the luminance is high with respect to the background to YUV convert the input image, and among these, the edge calculation in the horizontal direction is performed for the Y component. Furthermore, since the telop has a high brightness with respect to the background, it is possible to leave only the edge of the character portion by binarizing the edge information. Then, by performing thinning processing on the edge image, false detection can be reduced by straight line detection.

  In addition, even if it is an italic character, there is a limit to the angle of inclination due to visibility (usually, it is not inclined more than 30 degrees). Therefore, the straight line detection means can limit the inclination of the obtained straight line to reduce false detection.

According to a second aspect of the present invention, there is provided a computer program written in a computer-readable format so that character information extraction processing for extracting character information drawn in an input image is executed on a computer system. ,
When the character information is an italic character, when calculating the inclination angle of the character information in order to perform character recognition after correcting the character inclination and converting it to an upright character, the computer system,
Edge information detection procedure for detecting edge information of the input image;
A binarization processing procedure for binarizing the edge image detected in the edge information detection procedure using a predetermined threshold;
A thinning processing procedure for reducing and thinning the edge image binarized in the binarization processing procedure to a line segment having a predetermined thickness while maintaining connectivity;
A straight line detection procedure for detecting a straight line from the thinned edge image;
An inclination angle calculation procedure for calculating an inclination angle of character information using a straight line detected in the straight line detection procedure;
Is a computer program characterized in that

  The computer program according to the second aspect of the present invention defines a computer program described in a computer-readable format so as to realize predetermined processing on a computer system. In other words, by installing the computer program according to the second aspect of the present invention in the computer system, a cooperative action is exhibited on the computer system, and the character information according to the first aspect of the present invention. The same effect as the extraction device can be obtained.

  According to the present invention, an excellent character information extraction apparatus, character information extraction method, and computer that can extract character information contained in an image frame in a form that allows character recognition processing by a general OCR engine.・ Provide programs.

  In addition, according to the present invention, an excellent character information extraction device capable of suitably extracting italic characters drawn in a natural image such as a telop included in a television image as character information that can be subjected to character recognition processing, and A character information extraction method and a computer program can be provided.

  According to the present invention, it is possible to automatically calculate the inclination angle of a type drawn in a natural picture with high accuracy. In particular, since the inclination angle can be obtained correctly even in Japanese having a complicated character shape, the accuracy of character recognition by OCR can be improved by accurately correcting the character inclination.

  Other objects, features, and advantages of the present invention will become apparent from more detailed description based on embodiments of the present invention described later and the accompanying drawings.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  The present invention relates to a character information extraction system that extracts character information drawn in an image. The present invention is particularly applied when extracting text information of telop from a television image. As an application of this type of system, video indexing for performing scene search and digest viewing on recorded TV program content can be considered.

  Here, as problems when recognizing characters in a telop in a television image, there are many italic characters used in the telop and character information is drawn in a natural image. As a method for making italic characters processable by a general OCR engine, there is a method of converting italic characters into upright characters. That is, the inclination angle of the character information extracted from the input image is calculated, and the inclination of the character is corrected and converted into an upright character before being input to the OCR engine. However, the conventional italic character conversion method basically targets printed characters such as black characters drawn on a white background, and is not suitable for processing italic characters drawn in a natural image like a telop.

  Therefore, in the present invention, when calculating the inclination angle of a type drawn in a natural image, edge information is first detected from the input image, further binarized to extract character edge information, and then the edge image. Is thinned, a straight line is detected from the thinned edge image, and the inclination of the detected straight line is used as the inclination angle of the italic character.

  The telop uses the feature that the luminance is high with respect to the background, YUV-converts the input image, and performs edge calculation for the Y component. Furthermore, since the telop has a high brightness with respect to the background, it is possible to leave only the edge of the character portion by binarizing the edge information. Then, by performing thinning processing on the edge image, false detection can be reduced by straight line detection.

  Therefore, according to the character information extraction system according to the present invention, it is possible to automatically calculate the inclination angle of a type drawn in a natural image with high accuracy. In particular, since the inclination angle can be obtained correctly even in Japanese having a complicated character shape, the accuracy of character recognition by OCR can be improved by accurately correcting the character inclination.

  FIG. 1 schematically shows a functional configuration of a character information extraction system 10 according to an embodiment of the present invention. The illustrated system 10 includes functional modules such as an inclination angle calculation unit 11, a character inclination correction unit 12, and a character recognition unit 13. The system 10 can be designed as a dedicated hardware device, or can be realized by executing a predetermined application program on a general computer system such as a personal computer (PC). Hereinafter, each functional module will be described.

Input image:
An image input to the system 10 is an image frame in which character information such as a telop is drawn in a natural image such as a television image. For example, an image is input from a video storage device (not shown) that records a television program such as a hard disk recorder.

  For example, in a television broadcast of a genre such as a news program or a variety program, a method of displaying a telop is used for the purpose of obtaining the understanding and approval of the viewer or attracting interest in the program. In many cases, as shown in FIG. 2, since a stationary telop exists using any one of the four corners of the screen, character information may be extracted from these areas.

Inclination angle calculator:
The inclination angle calculation unit 11 calculates the inclination angle θ of characters in the input image. FIG. 3 shows an inclination angle calculation processing procedure in the form of a flowchart. As shown in the figure, the calculation process of the tilt angle includes the calculation of the horizontal edge of the input image (S1), the binarization process (S2), the thinning process (S3), and the straight line detection (S4). And straight line slope calculation (S5). FIG. 4 shows a state in which the input original image is processed by each step of these calculation processes.

  First, in step S1, an edge image as shown in FIG. 4B can be obtained from the original image shown in FIG. 4A by calculating a lateral edge with respect to the input original image. it can.

  When detecting edge information, the input image is YUV converted and edge calculation is performed on the Y component. This is because the telop uses the feature that the luminance is higher than the background. For example, an edge can be detected using a Sobel filter in the X direction. For the Soble filter, see, for example, “Computer Image Processing” written by Hideyuki Tamura (pp 188-191, Ohmsha). However, the gist of the present invention is not limited to using a Sobel filter for edge detection.

  For reference, FIG. 5 shows an example in which the edge in the horizontal direction, that is, the X direction is calculated from the original image, and the case where the edge in the vertical direction, that is, the Y direction is calculated. As can be seen from the figure, a vertical stroke can be detected by extracting an edge in the X direction.

  In the subsequent step S2, binarization processing is applied to the edge image obtained in step S1, and processing is performed to set all edges whose pixel values are less than the threshold to 0 and all edges that are greater than or equal to the threshold to 1, as shown in FIG. The edge image shown in c) is generated.

  Since the telop has a feature that the luminance is higher than that of the background, it is possible to leave only the edge of the character portion by appropriately performing such threshold processing.

  As a method for obtaining the threshold, Otsu's algorithm can be used. Regarding Otsu's algorithm, for example, Otsu's “Automatic threshold selection method based on discrimination and least squares criterion” (Electronic Communication Society Journal, Vol. J63-D, No. 4, pp 349-356, 1980). Please refer. However, the gist of the present invention is not limited to this.

  Further, in step S3, the thinning process is performed on the edge image after the binarization process. The thinning process referred to here is a process for reducing the edge portion to a line segment having a thickness of 1 (pixel) while maintaining connectivity, and generates an edge image as shown in FIG.

  The thinning process is performed for the purpose of reducing erroneous detection in the subsequent straight line detection process. FIG. 6 illustrates the difference in straight line detection when the edge image is thinned and when it is not thinned. When the edge of height h (pixel) is thinned, if a straight line passing through the point h or higher is detected, only a vertical straight line is detected. On the other hand, when thinning is not performed, a plurality of straight lines are erroneously detected as a result of detecting straight lines passing through the points h and higher.

  As the thinning process, an algorithm using a mask can be used. See, for example, “Skeletalization and Thinning” (C Magazine September 2000, pp122-131). However, the gist of the present invention is not limited to this.

  In step S4, straight line detection is performed from the thinned edge image. FIG. 4E shows a state in which a straight line is detected from the character “O”.

  For example, straight line detection can be performed on the thinned edge image using Hough transform. Regarding Hough conversion, see, for example, “Computer Image Processing” written by Hideyuki Tamura (pp 204-206, Ohmsha). However, the gist of the present invention is not limited to this.

  FIG. 7 shows a procedure of straight line detection processing in the form of a flowchart.

  First, as an initial value of the number N of pixels through which a desired straight line passes, a half of the image height is given (step S11).

  Then, straight line detection by Hough conversion is performed (step S12), and the number of detected straight lines is checked (step S13).

  Here, if a straight line is not detected, the number N of pixels through which the desired straight line passes is decremented (step S15), and then the process returns to step S12 to repeat the straight line detection.

  On the other hand, when one or more straight lines are detected, it is checked whether or not there is one or more straight lines whose inclination is in the range of 0 to 30 degrees (step S14). There is a limit to the inclination angle of characters due to visibility and the like, and it is usually impossible to incline more than 30 degrees. Therefore, it is possible to reduce false detection by limiting the slope of the straight line thus obtained.

  Here, when there is no straight line whose inclination is within the range of 0 to 30 degrees, the number N of pixels through which the obtained straight line passes is decremented (step S15), and then the process returns to step S12 to repeat the straight line detection. . On the other hand, if one or more straight lines are detected, this is output as a result of straight line detection.

  In step S5, the average of the detected inclinations of the straight lines is calculated, and this is output to the subsequent character inclination correction unit 12 as the inclination angle of the character information in the input image.

Character inclination correction part:
The character inclination correction unit 12 performs a shearing process using the character inclination angle θ calculated by the inclination angle calculation unit 11 to erect an italic character. The shearing process referred to here is calculated as shown in the following equation (see FIG. 8), where (x ′, y ′) is a position after movement of a certain pixel coordinate (x, y).

x ′ = x− (height−y) × tan θ
y '= y

  The image is scanned by TV raster scanning as shown in FIG. 9, and the above calculation is performed for all pixels. If there are pixels whose positions after the shearing process overlap, processing such as overwriting or averaging is performed.

Character recognition part:
The character recognition unit 13 performs character recognition on the input image and outputs character information. As the character recognition process, an existing OCR engine can be applied. As an example, a character recognition device described in Japanese Patent No. 2893781 already assigned to the present applicant can be cited.

  The present invention has been described in detail above with reference to specific embodiments. However, it is obvious that those skilled in the art can make modifications and substitutions of the embodiment without departing from the gist of the present invention.

  In this specification, the case where character information is extracted from a telop drawn on a television image has been described as an example. However, the gist of the present invention is not limited to this. It is possible to apply the present invention in the same way even when the character information is extracted from italic characters drawn in a natural image or italic characters formed as a normal printed matter, even if it is not a television image. Needless to say.

  In short, the present invention has been disclosed in the form of exemplification, and the description of the present specification should not be interpreted in a limited manner. In order to determine the gist of the present invention, the description of the scope of claims should be considered.

FIG. 1 is a diagram schematically showing a functional configuration of a character information extraction system 10 according to an embodiment of the present invention. FIG. 2 is a diagram showing a screen configuration example of a television program including a telop area. FIG. 3 is a flowchart showing the procedure for calculating the tilt angle. FIG. 4 is a diagram illustrating a state in which the input original image is processed by each step of the tilt angle calculation process. FIG. 5 is a diagram illustrating an example in which an edge in the horizontal direction, that is, the X direction is calculated from an original image, and an edge in the vertical direction, that is, an edge in the Y direction is calculated. FIG. 6 is a diagram for explaining a difference in straight line detection when the edge image is thinned and when it is not thinned. FIG. 7 is a flowchart showing the procedure of straight line detection processing. FIG. 8 is a diagram showing a state in which the italic character is made upright by performing a shearing process using the inclination angle θ of the character. FIG. 9 is a diagram showing a state in which TV-type raster scanning is performed.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Character information extraction system 11 ... Inclination angle calculation part 12 ... Character inclination correction part 13 ... Character recognition part

Claims (11)

A character information extraction device that extracts character information drawn in an input image,
When the character information is an italic character, when calculating the inclination angle of the character information in order to perform character recognition after correcting the character inclination and converting it to an upright character,
Edge information detection means for detecting edge information of the input image;
Binarization processing means for binarizing the edge image detected by the edge information detection means using a predetermined threshold;
Thinning processing means for reducing and thinning the edge image binarized by the binarizing processing means to a line segment having a predetermined thickness while maintaining connectivity;
A straight line detecting means for detecting a straight line from the thinned edge image;
An inclination angle calculating means for calculating an inclination angle of character information using a straight line detected by the straight line detecting means;
A character information extracting apparatus comprising: The input image is a television image, and a telop drawn in the television image is extracted as character information.
The character information extracting apparatus according to claim 1, wherein: The edge information detecting means detects a lateral edge of the input image;
The character information extracting apparatus according to claim 1, wherein: The edge information detection means performs YUV conversion on the input image and performs edge detection on the Y component.
The character information extracting apparatus according to claim 1, wherein: The straight line detecting means limits the inclination of the straight line to be detected.
The character information extracting apparatus according to claim 1, wherein: A character information extraction method for extracting character information drawn in an input image,
When the character information is an italic character, when calculating the inclination angle of the character information in order to perform character recognition after correcting the character inclination and converting it to an upright character,
An edge information detection step for detecting edge information of the input image;
A binarization processing step of binarizing the edge image detected in the edge information detection step using a predetermined threshold;
A thinning step for thinning the edge image binarized in the binarization processing step to a line segment having a predetermined thickness while maintaining connectivity;
A straight line detection step for detecting a straight line from the thinned edge image;
An inclination angle calculating step of calculating an inclination angle of the character information using the straight line detected in the straight line detecting step;
A character information extraction method comprising: The input image is a television image, and a telop drawn in the television image is extracted as character information.
The character information extracting method according to claim 6. In the edge information detection step, a lateral edge of the input image is detected.
The character information extracting method according to claim 6. In the edge information detection step, the input image is subjected to YUV conversion, and edge detection is performed on the Y component.
The character information extracting method according to claim 6. In the straight line detection step, the inclination of the straight line to be detected is limited.
The character information extracting method according to claim 6. A computer program written in a computer-readable format so as to execute character information extraction processing for extracting character information drawn in an input image on a computer system,
When the character information is an italic character, when calculating the inclination angle of the character information in order to perform character recognition after correcting the character inclination and converting it to an upright character, the computer system,
Edge information detection procedure for detecting edge information of the input image;
A binarization processing procedure for binarizing the edge image detected in the edge information detection procedure using a predetermined threshold;
A thinning processing procedure for reducing and thinning the edge image binarized in the binarization processing procedure to a line segment having a predetermined thickness while maintaining connectivity;
A straight line detection procedure for detecting a straight line from the thinned edge image;
An inclination angle calculation procedure for calculating an inclination angle of character information using a straight line detected in the straight line detection procedure;
A computer program for executing

Images