KR100615058B1 - Mobile handset and the method of selecting an objective area of the chatacter recognition on a mobile handset - Google Patents

Abstract

The present invention provides a method of selecting a processing target in character recognition of a portable information terminal device and a character string of a recognition target in an image in order to accurately perform character recognition processing in a portable information terminal with a camera as a portable information terminal device. It is necessary to prevent it. Alternatively, if there are two methods of writing vertically and horizontally in the description direction of the characters, it is necessary to specify the character line direction to be aligned in order to obtain a correct character recognition result. All of them had a heavy burden of designation or correction by the user. In addition, when attempting to search the recognition result as a key by recognizing Japanese characters, there is a problem in that a burden on a user to designate a search target language is large because the characters cannot be separated, unlike English words.

In order to properly correct the slope of the text line, an indicator indicating the slope of the text line is displayed on the image of the portable information terminal device. Alternatively, in order to search Japanese by character recognition and search for recognition results as keys, a technique of specifying a search target language using morphological analysis results and cursor position information is provided.

Description

Mobile handset and the method of selecting an objective area of the chatacter recognition on a mobile handset

1 is a plotter diagram of a portable information terminal in an embodiment of the present invention.

2 is a flowchart in the embodiment of the present invention.

3 is a flowchart in the embodiment of the present invention.

4 is an explanatory diagram of angle correction in the embodiment of the present invention.

5 is a flowchart in the embodiment of the present invention.

6 is a diagram illustrating a screen in an embodiment of the present invention.

7 is a flow chart in the embodiment of the present invention.

8 is an explanatory diagram of a character string circumscribed spherical generation method in the embodiment of the present invention.

9 is a block diagram of a portable information terminal according to the embodiment of the present invention.

10 is a flow chart in the embodiment of the present invention.

It is explanatory drawing of the spherical coordinate table in the Example of this invention.

12 is an explanatory diagram of a selection method of a search candidate word in the embodiment of the present invention.

It is explanatory drawing of the spherical coordinate table in the Example of this invention.

14 is an explanatory diagram of a selection area table in the embodiment of the present invention.

It is explanatory drawing of the spherical coordinate in the Example of this invention.

16 illustrates an embodiment of the present invention.

It is explanatory drawing of the spherical coordinate table in the Example of this invention.

18 is a diagram for explaining an embodiment of the present invention.

19 is a diagram illustrating an embodiment of the present invention.

It is explanatory drawing of the spherical coordinate table in the Example of this invention.

21 illustrates an embodiment of the present invention.

The present invention relates to a process object selection method when performing optical character recognition in a portable information terminal device.

Background Art A technique for performing optical character recognition on an image photographed in a portable information terminal apparatus such as a portable telephone terminal apparatus equipped with an image input function such as a camera has been developed. However, the image photographed by the camera of the portable information terminal is often photographed with the character string tilted with respect to the screen due to factors such as camera shake. If the character recognition result is wrong (misreading), the recognition result is corrected by an input device such as a keyboard to photograph the user while adjusting the angle at the time of photographing again.

As a technique for correcting the inclination of the character string to be recognized prior to the recognition process, an attempt to recognize two or more targets already written on a sheet of paper in the character string to be recognized in advance and automatically rotate according to the inclination obtained from the present is disclosed. It is disclosed in 1.

In addition, Non-Patent Document 1 discloses a method of combining character recognition and dictionary search, such as recognizing an English word in a portable information terminal to find an English-Japanese dictionary.

[Patent Document 1] Japanese Patent Application Laid-Open No. 11-250179 (paragraphs 4-7, 3)

[Non-Patent Document 1] H. Fujisawa, H. Sako, Y. Okada, and S-W. Lee, “Inbformation Capturng Camera and Developmental Issues,” In Proc. Int. Conf. Document Analysis and recognition, ICDAR '99, Bangalore, India, Sep. 20-22, 1999, pp. 205-208.

In order to perform the character recognition processing with high accuracy, it is important that the character string to be recognized is not inclined in the image. However, in the case of a digital image photographed by a portable information terminal device such as a digital camera or a mobile phone terminal device with a camera, the information terminal device is often held by hand, so that the character string of the object to be recognized is not tilted in the image. Special attention is required at the time of photography (first problem).

In the optical character recognition function of an image photographed by a portable information terminal device having a camera function, when a character string is photographed at an angle, the character recognition is not possible in the prior art, or sufficient recognition accuracy is not obtained. Therefore, in order to obtain a correct character recognition result, there is a problem of photographing a character to be recognized again.

Patent Document 1 discloses the purpose of detecting the tilt by two or more targets described on the paper, but this technique requires that the target be printed or written on the paper to be recognized in advance. Not practical When the character recognition processing is performed on an image that has been photographed in the past, the character string to be recognized may be inclined in the image unless the image is taken with the above special circumference. In this case, it is necessary to correct the angle of the image using image processing software or the like before performing the character recognition process (second task).

Furthermore, if the description direction of the text has two methods of vertical writing and horizontal writing, even if the angle of the image is extracted to extract the character string to be recognized, whether the character string to be recognized corresponds to the character recognition is vertical writing or horizontal writing. You need to set In the case of acquiring an image such as a business card and performing character recognition processing, there is a problem in that the recognition mode must be set again each time since the formats of vertical writing and horizontal writing exist. In addition, even in documents in which vertical writing strings and horizontal writing strings, such as newspapers and magazines, are mixed, the burden of changing the technical direction of letters is great (third problem).

In the case of using a portable information terminal device with a camera or the like, use in an environment in which an image is difficult to shoot at a stable position, such as during a vehicle movement, can be considered. However, in the prior art, the character recognition accuracy could not be sufficiently obtained unless the character string to be recognized is selected in such a state that the inclination is suppressed small with respect to the image. Therefore, the user needed special attention to adjust the angle when photographing the recognition target image (fourth problem).

In addition, in case of attempting to perform dictionary search or internet search using the result of character recognition in portable information terminal device with camera, it is easy to select the word to be searched because English words are divided by word by space. Do. However, if you try to apply the same processing to Japanese, unlike the English word, there is no paragraph by layout information such as space between words, so the user needs to select the character string to be searched one by one, and the burden of designation is burdensome. It was large (fifth problem).

SUMMARY OF THE INVENTION In view of these problems, the present invention reduces the burden of a photographer when photographing a character recognition object in a portable information terminal apparatus with a camera, or a system for reducing the burden of correcting an image after shooting at an angle that is very suitable for character recognition, or It is an object to provide a method.

Another object of the present invention is to provide a system or method for reducing the burden on a user who designates a character string to be searched even when searching a dictionary or the Internet by recognizing Japanese characters.

In order to solve the first problem, an angle indicator for presenting the degree of inclination in the image of the character string to be recognized to the photographer in real time is displayed on the image of the information terminal apparatus. The user can capture an image which is very suitable for the character recognition process by moving the information terminal device or the recognition object and shooting at an appropriate position while viewing the information displayed on the angle indicator.

In order to solve the second problem, a function of real time extracting a row of character strings that can be recognized at the same time as rotating a photographed image is provided. The user checks the recognition target character string candidate to be row-extracted in real time by rotating the captured image on the image of the information terminal apparatus by a simple key operation or the like. When the desired character string to be recognized is extracted, the user stops the rotation process, and when a plurality of rows are extracted, the user selects the character string to be recognized and executes the recognition process to execute the character recognition process for the image that has been taken in the past. We plan to simplify.

In order to solve the third problem, a function of automatically determining whether the recognition target character string is vertical writing or horizontal writing is provided. In the automatic determination of the character string direction to be recognized, the aspect ratio of the circumscribed rectangle of the extracted character string to be recognized is used. Specifically, the recognition process is performed by determining whether the ratio of the height and width of the circumscribed rectangle of the character string to be recognized is the vertical character string or the horizontal character string compared to the prescribed value. When the orientation of the screen of the portable information terminal device to be used is variable, the vertical writing and horizontal writing may be automatically converted in accordance with the orientation of the screen.

In order to solve the fourth problem, a means for generating an external spherical shape for automatically selecting a character string to be recognized following the inclination angle of the character string to be recognized is provided. Specifically, the angle of inclination of the character string to be recognized is measured using, for example, the Japanese Patent Application Laid-Open No. H7-141465 "Skew Detection Method for Document Images". In the case of vertical writing, the rotated image is created so as to be perpendicular to the image. Thereafter, a circumscribed rectangle of a character string to be recognized is generated, and a circumscribed rectangle image added to the rotated image is created. The rotating circumscribed spherical image is then rotated and returned to the inclination angle of the original character string to be displayed on the display device of the information terminal apparatus.

In order to solve the fifth problem, morphological analysis is performed on the result of character recognition in Japanese and the like, and the character string candidates to be searched are automatically generated, and the user selects individual candidates or a combination of the character string candidates. It provides a means to choose.

An embodiment very suitable for the present invention will be described with reference to FIGS. The text recognition method of the present invention can be applied to a portable information system that reads and recognizes a business card or the like, and can be applied to, for example, a text recognition function in a mobile phone.

1 is an example of a block diagram of a portable information terminal to which the present invention is applied. In this example, the portable information terminal main body 100, an image input device 110 such as a camera or scanner that optically inputs a business card to be recognized, an image or character recognition result of the recognition target, a cursor 121, or the like is displayed. The control unit mounted in the terminal main body 100 for controlling the display device 120 such as a CRT or liquid crystal, the input device 130 including the buttons 131 for operating the user, and the entire portable information terminal. An image processing unit having a character recognition unit 150 which performs 140 and character line extraction 151, character recognition processing 152, etc., a quantification function of character line inclination 161, and an image rotation processing function 162 ( 160).

The character recognition unit 150 and the image processing unit 160 operate on the same circuit as the control unit 140 as a function of the software, and do not interfere. The input device is not a problem as a general device such as a button, but the display device 120 and the input device 130 may be implemented as a display device and an input device such as a touch panel for improving operability.

Fig. 2 is an example of a flowchart in the case of performing a character recognition process to which the means for solving the first problem is applied (first embodiment). Input from the image input device 110 when the user starts the OCR function. The displayed moving image is displayed on the display device 120 (S201) .The character recognition unit 150 immediately executes the character line extraction processing for the area where the cursor 121, such as a cross, is displayed on the image display device 120. (S202) and an circumscribed rectangle surrounding the extracted character line is displayed (S203).

At the same time, the gradient of the character-up is quantified by the image processing unit 160 and the quantified value is visualized on the angle indicator 123 in the form of a bar graph or the like (S204). The quantified value may be any value as long as it reflects the degree of inclination of the text line and the image. For example, the angle indicator 123 may be used when the slope is small by employing a value proportional to the inverse of the angle θ produced by the text line and the image short side. A large value appears in the display and the user can intuitively manipulate it.

The process returns to step S201 until the user presses the photographing button (S205), and the subsequent processing is repeated (S201 to S204) to continuously update the screen display.

Referring to the angle indicator 123, the user moves the terminal device or the object to be recognized and presses the photographing button at an appropriate photographing position (S205) to perform image photographing (S206). If the continuous recognition button is pressed (S207), the character recognition process 210 is executed for the character string in the circumscribed rectangle displayed in step S203 to display the recognition result (S211).

If the cancel button is pressed after the photographing button is pressed (S205) (S208), the photographed image is discarded and the process returns to step S201. When other buttons are pressed, the process proceeds to the required operation (S209).

3 is an example of a flowchart in the case of performing a character recognition process to which the means for solving the second problem is applied (second embodiment). When the user of the portable information terminal performs an operation of loading a still picture previously taken (S301), the still picture is displayed on the display device 120 (S302). The character recognition unit 150 immediately performs character line extraction (S303), circumscribed spherical display (S304), and angle indicator display (S305). At this point, the process shifts to the user's key input wait (S306).

When the user presses the recognition execution button (S307), a character recognition process is immediately performed on the internal character string of the circumscribed rectangle displayed in step S304 (S310), and the character recognition result is displayed (S311). When the user presses the arrow button (S308), the button is rotated by a predetermined angle to the right or left direction (S309). At this time, the center of rotation is the character line rectangle, but the rotation process may be performed centering on one point on the image designated by the user.

If the arrow button is kept pressed, the image is rotated continuously, and the circumscribed spherical display is updated in real time accordingly. For the convenience of the user, the operation of pressing the recognition execution button may be replaced by the operation of stopping pressing the arrow button (releases the finger from the button).

4 is an example of an image displayed on the display unit 120 in the first embodiment and the second embodiment. In the first embodiment, the moving image input from the image input device 110 is displayed, and in the second embodiment, the still image photographed previously is displayed on the display unit 120 as the image 400.

The character string 401 to be recognized in the image 400 is inclined at this point. In the center of the screen, a cross cursor 402 serving as a rotation center is displayed. The character recognition unit 150 generates a circumscribed rectangle 403 surrounding the recognized character string 401 and displays it on the image. The angle indicator displays a bar graph indicating the degree of inclination of the inclined recognition character string 401 (404).

In the first embodiment, by rotating 406 the portable information terminal (camera) held by hand, in the second embodiment, the image displayed on the image is rotated by operating the cursor key (405). do. As the inclination of the character string 401 to be recognized decreases in accordance with the rotation of the image, the shape of the circumscribed sphere is also synchronized.

In addition, a large value indicating that the slope of the recognition target character string U401 is smaller is displayed on the angle indicator as a bar graph (408). The user can obtain a highly accurate character recognition result by repeating the rotation operation of the image and performing the character recognition process at the position where the value indicated by the angle indicator becomes large.

5 is an example of a flowchart in the case of performing a character recognition process to which the means for solving the third problem is applied (third embodiment). Steps S501 to S506 are the same as steps S201 to S209, and description is omitted.

At the point of time when the recognition button is pressed (S504), the aspect ratio (ratio of height and width) of the circumscribed rectangle of the character string to be recognized is calculated and compared with a predetermined value α (S507). If the aspect ratio is larger than the prescribed value α, it is determined that it is a vertical write character string, parameters are set for recognition of the vertical write character string (S510), a character recognition process is performed (S511), and the result is displayed (S512).

Similarly, if the aspect ratio is not greater than the prescribed value α, then the aspect ratio is compared with the prescribed value β (S508). If the aspect ratio is smaller than the prescribed value (β), it is determined that the horizontal character string is a character string, and a parameter for recognizing the character string is performed (S509), a character recognition process is performed (S511), and the result is displayed (S512). . If the aspect ratio is in the range of α or less and β or more, the character string is not sufficiently angle corrected and does not proceed to the recognition process.

6 is an example of an image displayed on the display unit 120 in the third embodiment. In the case of the horizontal writing character string 601, the height 602 of the circumscribed rectangle becomes smaller with respect to the width 603. If the aspect ratio is defined as the height of the circumscribed rectangle / the width of the circumscribed rectangle, the lateral rectangle becomes the lateral length if the aspect ratio is less than one.

For example, when the prescribed value β to be compared with the aspect ratio is set to 0.5, when the character recognition is started with the aspect ratio lower than 0.5, the necessary setting is automatically performed as the horizontal writing string. As described above, in the case of the vertical writing character string 604, the height 605 of the circumscribed rectangle increases with respect to the width 606.

In the above aspect ratio definition, when the aspect ratio is larger than 1, the circumscribed sphere is the longitudinal length. For example, when the prescribed value (α) to be compared with the aspect ratio is set to 1.5, when the character ratio is activated with the aspect ratio exceeding 1.5, the necessary setting as the vertical writing string is automatically performed.

7 is an example of a flowchart in the case of performing a character recognition process to which the means for solving the fourth problem is applied (fourth embodiment).

When the character recognition target image is input from the image input apparatus 110 (S701), the image processing unit 160 immediately calculates an angle with respect to the image of the character string to be recognized (S702), and corrects the input character recognition target image by the corresponding angle rotation. do. The correction direction is rotated so as to be horizontal with respect to the screen in the case of the horizontal writing string, and perpendicular to the screen in the case of the vertical writing string.

Next, character string extraction is performed on the post-rotation image, and an external spherical shape is added to the post-rotation image (S704) to store this image in the image buffer. The image to be stored in the image buffer may be the whole image, or may be only the added external spherical shape.

Next, the rotated image is reversely rotated by the angle detected in step S702, and an image inclined in the same manner as the original input image is created and displayed on the display unit 120 of the terminal device. If the user does not operate in particular, the process returns to step S701 and the processes of steps S701 to S707 are repeated for the new input image.

If the recognition button is pressed, the image stored in the image buffer is read out in this step S705 (S710), a character recognition process is executed for the image (S711), and the character recognition result is displayed (S712).

8 is an example of an image state processed in the fourth embodiment. The recognition target character string U802 on the recognition target image 801 input from the image input device 110 is inclined with respect to the screen. The image processing unit 160 detects the angle 803 where the recognition target character string 802 forms the edge of the screen, rotates the image only by the angle 803, and corrects the recognition character string so that it is horizontal with respect to the screen. The corrected image 804 is created.

With respect to the recognition target character string U806 on the rotation correction image 804, the character recognition unit 150 performs character line extraction to add a circumscribed rectangle 806. FIG. The image processing unit 160 generates the image 807 returned by the rotation correction image 804 to the same angle as the recognition target image 801 even if only the angle 803 is reversely rotated, and displays it on the display device 120. .

The first to fourth embodiments may be realized separately, or may be realized in a form in which all or part of them are selectively employed.

Next, 5th Embodiment for solving 5th subject is demonstrated using FIG. 9 thru | or FIG. 9 is an example of a block diagram of a portable information terminal to which the fifth embodiment is applied. The difference from FIG. 1 is that the search word extractor 170 and the electronic dictionary 171 are added.

FIG. 10 is an example of a flowchart in the case of performing the character recognition and the dictionary search process in the apparatus of FIG. 9 to which the means for solving the fifth problem is applied. When the character recognition target image is input from the image input device 110 (S1001), the image processing unit 160 performs appropriate image processing on the original image, and then the character line extracting unit 151 of the character recognition unit 150 executes the character. The row is extracted (S1002). Thereafter, the character line recognition unit 152 outputs a result of cutting and recognizing the character for each character with respect to the extracted character line (S1003). The recognition result includes the character code for each character and the corresponding spherical coordinates.

The result of the character recognition is input to the search word candidate extraction unit 170 to decompose the continuous string of characters into morphemes such as words by morpheme analysis (S1004). For example, in the case of the character string "to use the knowledge of grammar" as displayed on the display of the portable information terminal of FIG. 9, a table 1100 as shown in FIG. 11 is generated. The string decomposed by the morphological analysis and the corresponding rectangular coordinates are stored.

The candidate word to be searched for is displayed on the display of the portable information terminal using the data of the table 1100 (S1005). For example, the spherical coordinates of the candidate word including the center coordinates of the cursor are displayed on the display in a form as shown in 1200 of FIG. 12 by comparing the center coordinates of the cursor with the candidate word spherical coordinates of the table 1100.

Next, by pressing the cursor shift key 174 or 176, the spherical display of the search target word is moved as shown in 1201. When the rectangle to be searched is displayed, the selection target button 175 is pressed to determine the search target word (S1006). The determined search target word is output to the electronic dictionary unit 171. The electronic dictionary unit 171 searches the electronic dictionary with the input search target word as a key (S1007) and displays the search result on the display (S1008).

In this embodiment, although candidate words for search are generated by using morpheme analysis, delimiting candidate words are generated at the point where the category of the letter changes, such as changing from "Chinese character" to "Hiragana" in the character string of the recognition result. Also good. Or you may judge based on geometric information, such as a character category, a space, and the location of a change of a character size.

Next, 6th Embodiment for solving 5th subject is demonstrated using FIG. 9, FIG. 10, and FIG. 13 from FIG. In the present embodiment, it is assumed that the case where there is a kanji string composed of a plurality of morphemes, such as "Use grammar knowledge of ...", is adopted and the cursor is placed on either of the character strings of "grammar knowledge".

Since a user may want to search only morphemes such as "grammar" or "knowledge", and may want to search the whole compound word "grammar knowledge", the following process will reduce the burden of these selections. do. In the sixth embodiment, the other parts in the processing flowchart of FIG. 10 are candidate word extraction (S1004), candidate word display (S1005), and candidate word selection (S1006), and therefore only the processing will be described.

In candidate word extraction S1004, a candidate word is generated by morphological analysis by the processing as shown in the fifth embodiment, and a spherical coordinate table 1300 as shown in FIG. 13 corresponding thereto is generated. Next, the area coordinate table 1400 for selecting each of "grammar", "grammar knowledge" and "knowledge" from the table 1300 is generated. This table is used to display the rectangle of candidate words corresponding to the cursor center coordinates in the corresponding area.

15 schematically shows the X coordinate of the spherical coordinates of the main table. The corresponding selection areas of the "grammar", "grammar knowledge" and "knowledge" are 1500, 1501 and 1502, respectively, and the spherical coordinates which are displayed as search candidates according to which area the center coordinates of the cursor are included are included in the table 1400. Choose. Then, the selected spherical coordinates are displayed on the display (S1005).

An example of the displayed sphere is shown in FIG. 16. (a) is a display example when the cursor is in the selection area of "knowledge", (b) is "grammar knowledge", and (c) is "grammar", and is a display example when the cursor is in the selection area, respectively. If the rectangle of candidate words to be searched for is displayed, the user presses a selection key 175 to select a search target word (S1006). Although the present embodiment has described the selection of character strings in a single character bank, by selecting an area coordinate table for the upper limit character row, selection to cross over a plurality of character lines becomes possible.

Next, a seventh embodiment for solving the fifth problem will be described with reference to FIGS. 9, 10, 13, 17, and 18. The sixth embodiment adopts the case where there is a kanji string composed of a plurality of morphemes, such as "I use grammar knowledge of ... ...", and the cursor is located in the "grammar" part. Imagine. As in the sixth embodiment, only candidate word extraction (S1004), candidate word display (S1005), and candidate word selection (S1006) will be described.

In candidate word extraction S1004, a candidate word is generated by morphological analysis by the processing as shown in the fifth embodiment, and a spherical coordinate table 1300 as shown in FIG. 13 corresponding thereto is generated. From the table 1300, a spherical coordinate table 1700 (FIG. 17) corresponding to each of the morphemes and their compound words is generated. The table is assumed to be salted with the upper left coordinate.

In candidate word display S1005, as shown in FIG. 18, the morpheme rectangle 1800 whose center coordinates are contained in the rectangle area is displayed for the first time. Each time the cursor key 176 is pressed, the rectangle of the salted table 1700 is displayed. The user can search the electronic dictionary by pressing the selection key 175 at the time when the sphere to be searched is displayed (S1006).

Next, 8th Embodiment for solving 5th subject is demonstrated using FIG. 9, FIG. 10, FIG. 13, and FIG. As in the sixth embodiment, as in the case of using grammatical knowledge of ..., the case where there is a kanji string composed of plural morphemes is assumed, and the cursor is located in the "grammar" part. do. As in the sixth embodiment, only candidate word extraction (S1004), candidate word display (S1005), and candidate word selection (S1006) will be described.

In candidate word extraction 1004, a candidate word is generated by morphological analysis by the processing as shown in the fifth embodiment, and a spherical coordinate table 1300 as shown in Fig. 13 corresponding thereto is generated. In operation 1005, the candidate word rectangle including the center coordinates of the cursor is displayed in the form as shown in 1900 of FIG. 19.

In this case, when a compound word is to be selected as in "grammar knowledge", a spherical shape such as 1901 is displayed by pressing the "1" key giving the meaning of specifying the starting point of the selection range and then pressing the cursor movement key 176. If the cursor movement key 176 is pressed, the selection area as shown in 1902 is expanded. When the user presses the selection key 175 at the time when the appropriate sphere of the search target word is displayed by the cursor key (S1006).

Next, 9th Embodiment for solving 5th subject is demonstrated using FIG. 9, FIG. 10, FIG. 20, FIG. It is assumed that there is a kanji string composed of three or more morphemes, so that "grammar knowledge processing of ..." is called, and the cursor is located in the "knowledge" part. do.

In the case of a kanji string composed of three or more morphemes, the user generally thinks that the frequency is high whether one wants to search the entire kanji string or only the morphemes containing the cursor center coordinates. Here, the process of reducing the burden of these selection is demonstrated below. As in the sixth embodiment, only candidate word extraction (S1004), candidate word display (S1005), and candidate word selection (S1006) will be described.

In candidate word extraction (S1004), as shown in FIG. 20 in which candidate words are generated by morphological analysis by the processing as shown in the fifth embodiment, and the character type categories corresponding to the candidate words are stored in the spherical coordinates. Create the same table 2000. The character category category referred to here means a classification result for each category of character species when the candidate word belongs to "kanji" and "hiragana" if it belongs to "kanji". Then, the rectangles of candidate words having the same character type category as the candidate words (morphemes) including the center coordinates of the cursor are combined and displayed in the form as shown in 2100 of FIG. 21 (S1005).

In this case, when only a candidate word of "knowledge" is selected instead of the entire kanji string, a square such as 2101 is displayed by pressing the key of "#" which means changing the selection mode. In addition, to select a character string called "knowledge processing", after pressing the key of "1" which designates the starting point of the selection range, the cursor movement key 176 is pressed to display a rectangle such as 2102. . The user presses the select key 175 to select a search target word at a time when a suitable sphere of search target word is displayed by this key operation (S1006).

In the above embodiment, only the "Chinese character" and "Hiragana" are mentioned as the category of the character species, but any category that classifies the character species such as "katakana", "English character", "number", "symbol" and "foreign language" is applied. In addition, a part-of-speech belonging to the morpheme may be used instead of the category of the character species.

In addition, in the above embodiment, any key may be assigned as long as "1" and "#" are pressed to designate the starting point of the selection range or to change the mode of selection.

In the present embodiment, the center coordinates of the cursor of the cross mark are used as the selection information. However, any other information may be used as long as the method gives the same effect. For example, the parentheses may be displayed on the display and the center coordinates of the parentheses or the spherical coordinates of the parentheses may be used. In addition, although the horizontal writing was described in the Japanese selection, the same applies to the vertical writing.

The above embodiments may be realized separately, or may be realized in a form in which all or part of them are selectively employed.

By displaying the inclination state of the image to be recognized visually, it is easy to recognize that the image is inclined by the user and to easily take an image at an angle that is very suitable for the character recognition process.

In addition, for images captured in the past, the tilted image can be directly edited and the character recognition process can be performed on the result, so that it is not necessary to shoot again.

In addition, even when angle correction is difficult, the character string can be selected in an inclined state, so that the character recognition process can be performed even for an image inclined to some extent.

Claims (22)

An image photographing unit which photographs an image, A character line extracting unit for extracting a character line to be recognized from the image; Character recognition part which recognizes character in image, An image processor which rotates the image to correct tilt; A mobile telephone terminal device having an image display unit for displaying an image to be recognized, A mobile telephone terminal device characterized by displaying an angle indicator quantitatively displaying a shooting angle well suited for a character recognition process, and performing character recognition on a picked-up image determined at a very suitable angle. A character line extracting unit which extracts a character recognition target character line from the photographed still image, and a character recognition unit which recognizes characters in the image; An image processor which rotates the image to correct tilt; A mobile telephone terminal device having an image display unit for displaying an image to be recognized, Rotation processing of the photographed still image, Extraction processing of character recognition target character string candidates, Selection processing for selecting a desired recognition target character string from the extracted character recognition target character string candidate; And a character recognition process for the selected recognition target character string. The method according to claim 1 or 2, And the recognition mode is switched automatically by either the horizontal writing or the vertical writing, and the recognition mode is switched according to the extracted aspect ratio of the circumscribed rectangle. The method according to claim 1, A portable telephone terminal device which automatically determines either horizontal writing or vertical writing according to the direction of the display screen of the portable information terminal at the time of image shooting, and switches the recognition mode. The method according to claim 2, And a circumferential rectangle of the character string candidate to be recognized is automatically generated and displayed following the slope of the character string in the photographed image. An image photographing unit which photographs an image, A character line extracting unit for extracting a character recognition target character line from the image, and a character recognition unit recognizing a character in the image; A search candidate word generation unit for generating a search candidate word based on the output of the character recognition unit, and an image display unit for displaying the output of the search candidate word generation unit on a screen; And a user interface unit for selecting the screen displayed search candidate word. The method according to claim 6, And the search candidate word generation unit generates a search candidate word using morphological analysis on the output of the character recognition unit. The method according to claim 6, And the search candidate word generation unit assigns a character category corresponding to each character code in the character string of the character recognition result and outputs the same character code of the character category as one search candidate word. The method according to claim 6, The search candidate word generation unit outputs the search candidate word using both information of the result of the morpheme analysis on the output of the character recognition and the result of assigning a character category corresponding to each character code of the character recognition result. Mobile phone terminal device. The method according to claim 6, And the search candidate word generation unit generates a search candidate word using any one of the information of the individual character codes and the geometry information in the character string of the character recognition result. The method according to claim 6, The search candidate word generation unit generates a search candidate word by using morpheme analysis on the output of character recognition, and generates selection region coordinates for selecting any one of compound words combining morphemes and morphemes according to the position of the cursor. A mobile phone terminal device. As a character recognition method in a mobile phone terminal device which recognizes a character in a character recognition target character line extracted from a photographed image, A step of taking an image, Extracting a character recognition target character line from the image; Rotating the photographed image to correct tilt; A step of displaying an image to be recognized, Quantitatively displaying the shooting angle, which is very suitable for the character recognition process, to the portable information terminal user; Determining a very suitable angle according to the displayed shooting angle; A character recognition method in a mobile phone terminal device, characterized by comprising: a step of performing character recognition on an image photographed at the determined very suitable shooting angle. As a character recognition method in the mobile phone terminal device which recognizes the character of the character recognition target character line extracted from the photographing completed image, Extracting a character recognition target character line from the image; Rotating the image to correct tilt; A step of displaying an image to be recognized, Performing a rotation process of the photographed completed image; Extracting a character recognition target character string candidate simultaneously with the rotation process; Selecting a character string to be recognized from the extracted character character string candidate; And a character recognition method for the selected character string to be recognized. The method according to claim 12 or 13, The step of performing the character recognition, A step of automatically determining either horizontal writing or vertical writing based on the extracted rectangle aspect ratio aspect; And a step of changing the recognition mode on the basis of the determination result. The method according to claim 12, The step of performing the character recognition, Automatically determining either horizontal writing or vertical writing along the direction of the display screen of the portable information terminal at the time of image shooting; And a step of changing the recognition mode on the basis of the determination result. The method according to claim 13, The step of selecting a recognition target character string from the character recognition target character string candidate, And a step of automatically following the gradient of the character string in the photographed completed image and automatically generating and displaying a circumferential rectangle of the character string candidate to be recognized. A step of taking an image, Extracting a character recognition target character line from the image; Recognizing characters in the image; A search target word selection method in a mobile phone terminal device comprising the step of retrieving data stored inside and outside the mobile phone terminal device using the character recognition result. Generating a search candidate word based on the character recognition result, and displaying the search candidate word on a screen; And selecting a search candidate word displayed on the screen. The method according to claim 17, The step of generating the search candidate word further comprises the step of generating a search candidate word using morphological analysis on the result of the character recognition. The method according to claim 17, The step of generating the search candidate word, Assigning a character category corresponding to each character code in the character string of the character recognition result; And a step of outputting the same character code of said character category as one search candidate word. The method according to claim 17, The step of generating the search candidate word, In a mobile phone terminal apparatus, a search candidate word is output using both information of a result of a morpheme analysis of the character recognition result and a result of assigning a character category corresponding to each character code of the character recognition result. How to select search targets for. The method according to claim 17, The step of generating the search candidate word, A method for selecting a search target word in a mobile phone terminal device, characterized in that a search candidate word is generated using either the information of each character code or the geometric information in the character string of the character recognition result. The method according to claim 17, The step of generating the search candidate word, Generating a search candidate word using morpheme analysis on the character recognition result; And generating a selection area coordinate for selecting one of the compound word and the compound word in combination with the position of the cursor.

Images