WO2018107566A1 - Processing method and mobile device - Google Patents

Abstract

The present application relates to the technical field of communications and provides a processing method and a mobile device, for use in accurately and quickly locating target content in a photographed image. The method comprises: displaying a preview frame in a preview box of a mobile device, the preview frame comprising at least two quadrilaterals; determining that the time when a gaze focus of a user on a first quadrilateral exceeds a preset duration; prominently displaying an outline of the first quadrilateral in the preview frame; and after detecting a first photographing instruction, performing preset processing on a first image enclosed by the outline of the first quadrilateral, and outputting the first image subjected to the preset processing, the preset processing comprising performing quadrilateral correction on the first image, or performing OCR on text in the first image.

Description

Processing method and mobile device

The present application claims priority to Chinese Patent Application No. 2016-1169528.7, entitled "A Method and Apparatus for Processing", filed on Dec. 16, 2016, the entire contents of .

Technical field

The embodiments of the present invention relate to the field of communications technologies, and in particular, to a processing method and a mobile device.

Background technique

At present, many smart terminals have functions such as document correction and document translation. Among them, the document correction function can quickly locate the target content (for example, business cards, books, billboards, etc.) in the captured image, and the document translation function can be used for the user. Translate the text on the target content in the shooting screen.

When implementing the above functions such as document correction and document translation, it is necessary to locate the target content in the shooting screen. Generally, the smart terminal first determines one or more candidate quadrilaterals from the captured image, and further, in the candidate quadrilateral, a candidate quadrangle that is close to the center of the photographing screen and has a larger area and a shape closer to a rectangle is used as the target quadrilateral, thereby The content of the picture in the quadrilateral is used as the target content.

However, the positioning method of the above target content requires the user to manually adjust the shooting screen, especially when the user includes multiple candidate quadrilaterals in the shooting screen, and if the user does not perform left and right or backward movement, zooming the shooting screen, etc., the user calibrates the shooting screen. The target content of the smart terminal positioning is likely not the content that the user wants, that is, the target content content in the captured image cannot be accurately and conveniently located.

Summary of the invention

Embodiments of the present invention provide a processing method and a mobile device, which can accurately and conveniently locate a target content content in a captured picture.

In order to achieve the above object, embodiments of the present invention adopt the following technical solutions:

The first aspect provides a processing method, including: displaying a preview frame in a preview frame of the mobile device, the preview frame including at least two quadrilaterals; determining that the user's line of sight focus stays on the first quadrilateral (the first four sides) The duration in the shape of the at least two quadrilaterals is greater than a preset duration; the contour of the first quadrilateral is highlighted in the preview frame; after the first photographing instruction is detected, the first four sides are After performing the preset processing on the first image surrounded by the contour line, outputting the first image subjected to the preset processing; the preset processing includes: performing quadrilateral correction on the first image; or, the first image The text in the text performs OCR.

It can be seen that, in the processing method provided by the embodiment of the present invention, the mobile device determines, by detecting the focus of the user's line of sight, which quadrant in the preview frame the user's line of sight stays for a longer time, because the line of sight stays longer in the quadrilateral. The image is usually the target content that the user needs, then the mobile device highlights the edge line of the quadrilateral, thereby prompting the user that the user has already The first image in the quadrilateral is determined as the target content. Subsequently, after detecting the photographing instruction, the mobile device may directly output the first image after the preset processing. In this way, the mobile device can accurately and conveniently locate the target content in the preview frame for the user without manually adjusting the preview frame.

In a possible implementation, after the outline of the first quadrilateral is highlighted in the preview frame, the method further includes: determining that the duration of the user's line of sight staying in the second quadrilateral is greater than the preset duration, The second quadrilateral is any quadrilateral except the first quadrilateral of the at least two quadrilaterals; the outline of the first quadrilateral is normally displayed in the preview frame; the second is highlighted in the preview frame The outline of the quadrilateral. That is to say, when the user's line of sight changes, the mobile device can further display the contour of the second quadrilateral where the changed line of sight focus is displayed for the user.

In a possible implementation, after the outline of the second quadrilateral is highlighted in the preview frame, the method further includes: after detecting the second photographing instruction, enclosing the outline of the second quadrilateral After the second image performs the preset processing, the second image subjected to the preset processing is output.

In a possible implementation, after displaying the preview in the preview box of the mobile device, the method further includes: displaying a contour of a default quadrilateral in the preview frame, wherein the default quadrilateral is a quadrilateral having the largest area among the at least two quadrilaterals. That is to say, before detecting the user's line of sight focus, the mobile device may first use the quadrilateral with the largest area as the default quadrilateral, and present the image surrounded by the outline of the default quadrilateral as the target content to the user.

In a possible implementation manner, before displaying the preview frame in the preview frame of the mobile device, the method further includes: performing quadrilateral detection on the screen content in the preview frame, and storing the detected vertex coordinates of the at least two quadrilaterals.

In a possible implementation manner, determining that a user's line of sight focus stays within the first quadrilateral is greater than a preset duration includes: detecting a focus position of the line of sight focus within the preview frame; determining that the focus position is in a preview The duration in the first quadrilateral of the frame is greater than the preset duration.

In a possible implementation manner, performing OCR on the text in the first image includes: translating the text in the first image into a text in a preset language, and replacing the text in the first image with a translation And text of the preset language; or, the text in the first image is translated into the text of the preset language, and the translated text of the preset language is added to the first image, thereby implementing the document translation function.

In a possible implementation manner, the preset processing further includes: performing a cropping operation on the first image.

The second aspect provides a processing method, including: performing quadrilateral detection on a preview frame to be displayed, saving the detected vertex coordinates of at least two quadrilaterals; displaying the preview frame in a preview box of the mobile device; determining that the line of sight focus is located in the The first quadrilateral of the at least two quadrilaterals exceeds a preset duration, the outline of the first quadrilateral is presented in the preview frame; an instruction for photographing is detected; the first four in the preview frame After performing the preset processing on the first image surrounded by the outline of the edge, outputting the first image subjected to the preset processing, and the first image passing through the preset processing includes the contour line of the first quadrilateral The surrounding first content, the preset processing includes at least one of the following: quadrilateral correction; performing optical character recognition on the text in the first image, translating the recognized text into a text of a preset language, and the recognized text Replaced with the text of the preset language; in the first image The text performs optical character recognition, translates the recognized text into a text of a preset language, and adds the text of the preset language to the side of the recognized text.

In a possible implementation manner, the method further includes: determining that a line of sight focus is within a second quadrilateral of the at least two quadriles beyond a preset duration, and presenting the contour of the second quadrilateral in the preview frame a line; detecting an instruction for photographing; after performing the preset processing on the second image surrounded by the outline of the second quadrilateral in the preview frame, outputting the second image subjected to the preset processing, the passing The second image of the preset process includes a second content surrounded by the outline of the second quadrilateral.

In a third aspect, a processing method is provided, including: performing quadrilateral detection on a preview frame to be displayed, and storing the detected vertex coordinates of at least two quadrilaterals; displaying the preview frame in a preview frame of the mobile device, where the preview frame is presented a contour line of the first quadrilateral of the at least two quadrilaterals; determining that the line of sight focus is within the second quadrilateral of the at least two quadrilaterals for more than a preset duration, and presenting the second quadrilateral in the preview frame a contour that hides a contour of the first quadrilateral, the second quadrilateral being any one of the at least two quadrilaterals except the first quadrilateral; an instruction for photographing is detected; After performing a preset process on the second image surrounded by the outline of the second quadrilateral in the preview frame, outputting the second image that is subjected to the preset process, and the second image that is subjected to the preset process includes the first image a second content surrounded by a contour of the quadrilateral, the preset processing comprising at least one of the following: quadrilateral correction; performing optical character recognition on the text in the second image, and translating the recognized text into a preset language Text, and replacing the recognized text with the text of the preset language; performing optical character recognition on the text in the second image, translating the recognized text into a text of a preset language, and The text is added next to the recognized text.

A fourth aspect provides a processing method, comprising: displaying a preview frame in a preview frame of the mobile device, the preview frame including at least two quadrilaterals; detecting an operation acting on the first quadrilateral, the first four sides Forming one of the at least two quadrilaterals; in response to the operation, after performing a preset process on the first image surrounded by the outline of the first quadrilateral, outputting the first image subjected to the preset processing; The preset process includes: performing quadrilateral correction on the first image; or performing optical character recognition on the text in the first image. It should be understood that the operation includes, but is not limited to, clicking, double clicking, tapping, long pressing or repressing.

In a possible implementation manner, performing optical character recognition on the text in the first image includes: translating the text in the first image into a text in a preset language, and replacing the text in the first image The translated text of the preset language; or, the text in the first image is translated into the text of the preset language, and the translated text of the preset language is added to the first image.

In a possible implementation manner, the preset processing further includes: performing a cropping operation on the first image.

A fifth aspect provides a processing method, including: displaying a preview frame in a preview frame of the mobile device, the preview frame including at least two quadrilaterals; detecting a first operation acting on the first quadrilateral, the first a quadrilateral is one of the at least two quadrilaterals; in response to the first operation, highlighting the contour of the first quadrilateral; detecting a second operation acting on the camera key; in response to the second operation, After performing a preset process on the first image surrounded by the outline of the first quadrilateral, outputting the first image subjected to the preset process; the preset process includes: performing quadrilateral correction on the first image; Alternatively, optical character recognition is performed on the text in the first image. It should be understood that the first operation and the second operation may be the same or different, and are not limited herein. The first operation includes but is not limited to: clicking, double clicking, tapping, long pressing or re-pressing. The second operation includes but is not limited to: clicking, tapping, long pressing or re-pressing.

In a possible implementation manner, the preset processing further includes: performing a cropping operation on the first image.

A sixth aspect provides a mobile device, comprising: a display unit, configured to display a preview frame in a preview frame of the mobile device, the preview frame includes at least two quadrilaterals; and a determining unit, configured to determine that a user's line of sight focus stays at The duration of the first quadrilateral is greater than a preset duration, the first quadrilateral being one of the at least two quadrilaterals; the display unit is further configured to highlight the first quadrilateral in the preview frame a contour processing unit, configured to: after detecting the first photographing instruction, performing a preset process on the first image surrounded by the outline of the first quadrilateral, and outputting the first image subjected to the preset processing; The preset process includes: performing quadrilateral correction on the first image; or performing optical character recognition on the text in the first image.

In a possible implementation manner, the method further includes determining that a duration in which the user's line of sight focus stays in the second quadrilateral is greater than the preset duration, and the second quadrilateral is the first one of the at least two quadrilaterals Any one of the quadrilaterals outside the quadrilateral; the display unit is further configured to normally display the outline of the first quadrilateral in the preview frame; and highlight the outline of the second quadrilateral in the preview frame.

In a possible implementation, the processing unit is further configured to: after detecting the second photographing instruction, perform the preset processing on the second image surrounded by the outline of the second quadrilateral, and output the passing through The second image of the preset process.

In a possible implementation manner, the display unit is further configured to highlight a contour of a default quadrilateral in the preview frame, where the default quadrilateral is the largest quadrilateral of the at least two quadrilaterals.

In a possible implementation, the mobile device further includes: a detecting unit, configured to perform quadrilateral detection on the preview frame, and save the detected vertex coordinates of the at least two quadrilaterals.

In a possible implementation, the determining unit is specifically configured to detect a focus position of the line of sight focus in the preview frame, and determine that the focus is located in the first quadrilateral for a longer duration than the preset duration.

In a possible implementation, the processing unit is specifically configured to translate the text in the first image into a text in a preset language, and replace the text in the first image with the translated language in the preset language. Text; or, the text in the first image is translated into the text of the preset language, and the translated text of the preset language is added to the first image.

A seventh aspect provides a mobile device, comprising: a detecting unit, configured to perform quadrilateral detection on a preview frame to be displayed, save coordinates of at least two quadrilateral vertices detected; and display unit, in a preview box of the mobile device And displaying, in the preview frame, the first quadrilateral a detection unit configured to detect an instruction for photographing; a processing unit configured to perform a first image surrounded by the outline of the first quadrilateral in the preview frame After the line preset processing, the first image subjected to the preset processing is output, and the first image that is subjected to the preset processing includes the first content surrounded by the outline of the first quadrilateral, The preset processing includes at least one of the following: quadrilateral correction; performing optical character recognition on the text in the first image, translating the recognized text into a text of a preset language, and replacing the recognized text with the preset language a text; performing optical character recognition on the text in the first image, translating the recognized text into a text of a preset language, and adding the text of the preset language to the side of the recognized text.

An eighth aspect provides a mobile device, comprising: a detecting unit, configured to perform quadrilateral detection on a preview frame to be displayed, save coordinates of the detected at least two quadrilateral vertices; and display unit, in a preview box of the mobile device The preview frame is displayed, wherein the preview frame presents an outline of the first quadrilateral of the at least two quadrilaterals; and the determining unit is configured to determine that the line of sight focus is located in the second four of the at least two quadrilaterals The contour of the second quadrilateral is presented in the preview frame to hide the contour line of the first quadrilateral, and the second quadrilateral is the at least two Any one of the quadrilaterals except the first quadrilateral; the detecting unit is further configured to detect an instruction for photographing; and the processing unit is configured to: contour the second quadrilateral in the preview frame After performing the preset processing on the second image surrounded by the line, the second image subjected to the preset processing is output, and the second image subjected to the preset processing includes the contour line surrounded by the second quadrilateral Second content, The preset processing includes at least one of the following: quadrilateral correction; performing optical character recognition on the text in the second image, translating the recognized text into a text of a preset language, and replacing the recognized text with the preset language a text; performing optical character recognition on the text in the second image, translating the recognized text into a text of a preset language, and adding the text of the preset language to the side of the recognized text.

A ninth aspect provides a mobile device, including: a display unit, configured to display a preview frame in a preview frame of the mobile device, the preview frame includes at least two quadrilaterals; and a detecting unit configured to detect a role a quadrilateral operation, the first quadrilateral being one of the at least two quadrilaterals; a processing unit responsive to the operation, surrounding the outline of the first quadrilateral After the first image performs the preset processing, outputting the first image subjected to the preset processing; the preset processing includes: performing quadrilateral correction on the first image; or performing execution on the text in the first image Optical character recognition.

A tenth aspect provides a mobile device, comprising: a display unit, configured to display a preview frame in a preview frame of the mobile device, the preview frame includes at least two quadrilaterals; and a detecting unit, configured to detect a role a first operation of a quadrilateral, the first quadrilateral being one of the at least two quadrilaterals; the display unit further configured to highlight the first quadrilateral in response to the first operation a contouring unit; the detecting unit is further configured to detect a second operation acting on the camera key; the processing unit, configured to, in response to the second operation, the first surrounded by the contour line of the first quadrilateral After the image performs the preset processing, outputting the first image subjected to the preset processing; the preset processing includes: performing quadrilateral correction on the first image; or performing optical characters on the text in the first image Identification.

The eleventh aspect provides a mobile device comprising: a processor, a memory, a bus, and a communication An interface for storing a computer execution instruction, the processor being coupled to the memory via the bus, the processor executing the computer-executable instruction stored by the memory when the mobile device is running, to cause the mobile device to perform the arbitrary A treatment method.

A twelfth aspect provides a computer readable storage medium for storing computer software instructions for use in the mobile device described above, comprising a program designed to perform the above aspects for the mobile device.

A thirteenth aspect provides a computer program comprising instructions which, when executed by a computer, cause a computer to perform the processing method of any of the above.

The names of the above mobile devices are not limited to the devices or the functional modules themselves. In actual implementation, these devices or functional modules may appear under other names. As long as the functions of the respective devices or functional modules are similar to the present invention, they are within the scope of the claims and equivalents thereof.

In addition, the technical effects brought by the design method of any one of the second aspect to the thirteenth aspect can be referred to the technical effects brought by different design modes in the first aspect, and details are not described herein again.

These and other aspects of the present application will be more readily apparent from the following description of the embodiments.

DRAWINGS

FIG. 1 is a schematic diagram of an application scenario of a preview frame according to an embodiment of the present disclosure;

2 is a schematic diagram of an application scenario of a built-in camera according to an embodiment of the present invention;

FIG. 3 is a schematic structural diagram 1 of a mobile device according to an embodiment of the present disclosure;

FIG. 4 is a schematic diagram 1 of an application scenario of a processing method according to an embodiment of the present disclosure;

FIG. 5 is a schematic flowchart 1 of a processing method according to an embodiment of the present disclosure;

FIG. 6 is a schematic diagram 2 of an application scenario of a processing method according to an embodiment of the present disclosure;

FIG. 7 is a schematic diagram 3 of an application scenario of a processing method according to an embodiment of the present disclosure;

FIG. 8 is a schematic diagram 4 of an application scenario of a processing method according to an embodiment of the present disclosure;

FIG. 9 is a schematic diagram 5 of an application scenario of a processing method according to an embodiment of the present disclosure;

FIG. 10 is a schematic diagram 6 of an application scenario of a processing method according to an embodiment of the present disclosure;

FIG. 11 is a schematic diagram 7 of an application scenario of a processing method according to an embodiment of the present disclosure;

FIG. 12 is a schematic diagram 8 of an application scenario of a processing method according to an embodiment of the present disclosure;

FIG. 13 is a second schematic flowchart of a processing method according to an embodiment of the present disclosure;

FIG. 14 is a schematic diagram 9 of an application scenario of a processing method according to an embodiment of the present disclosure;

FIG. 15 is a schematic flowchart 3 of a processing method according to an embodiment of the present disclosure;

FIG. 16 is a schematic diagram of an application scenario of a processing method according to an embodiment of the present disclosure;

FIG. 17 is a schematic diagram of an application scenario of a processing method according to an embodiment of the present invention;

FIG. 18 is a schematic diagram of an application scenario of a processing method according to an embodiment of the present invention;

FIG. 19 is a schematic flowchart 4 of a processing method according to an embodiment of the present disclosure;

FIG. 20 is a schematic diagram of an application scenario of a processing method according to an embodiment of the present disclosure;

FIG. 21 is a schematic diagram of an application scenario of a processing method according to an embodiment of the present invention; FIG.

FIG. 22 is a schematic diagram of an application scenario of a processing method according to an embodiment of the present invention;

FIG. 23 is a schematic diagram of an application scenario of a processing method according to an embodiment of the present invention;

FIG. 24 is a schematic structural diagram 2 of a mobile device according to an embodiment of the present disclosure;

FIG. 25 is a schematic structural diagram 3 of a mobile device according to an embodiment of the present invention.

detailed description

The technical solutions in the embodiments of the present invention are described in detail below with reference to the accompanying drawings in the embodiments of the present invention.

In addition, the terms "first" and "second" are used for descriptive purposes only, and are not to be construed as indicating or implying a relative importance or implicitly indicating the number of technical features indicated. Thus, features defining "first" and "second" may include one or more of the features either explicitly or implicitly. In the description of the embodiments of the present invention, the meaning of "a plurality" is two or more unless otherwise specified.

The term "and/or" in this context is merely an association describing the associated object, indicating that there may be three relationships, for example, A and / or B, which may indicate that A exists separately, and both A and B exist, respectively. B these three situations. In addition, the character "/" in this article generally indicates that the contextual object is an "or" relationship.

In order to facilitate the understanding of the embodiments of the present invention, several terms introduced in the description of the embodiments of the present invention are first introduced herein:

The preview frame refers to a single-frame shooting picture of a minimum unit captured by the mobile device during the shooting process, and the shooting picture can be displayed in the display interface of the mobile device, for example, using the mobile phone as the mobile device as an example. After the user triggers the camera application in the mobile phone, as shown in FIG. 1 , a preview frame 12 can be displayed in the display interface of the mobile phone, and then the mobile phone captures the captured image captured by the camera in the preview frame 12 in real time. Each shot in the preview frame 12 can be used as a preview frame 11. Of course, the above-mentioned mobile device may also be an Augmented Reality (AR) device or a portable device (for example, a mobile phone) such as a wearable eyeglass, a wearable helmet, and the like.

The built-in camera 13 of the mobile device refers to a camera that is opposite to the user's line of sight when the user views the display screen of the mobile device. The built-in camera 13 can generally capture the state of the user's eyeball during work, for example, the front camera of the mobile phone (as shown in the figure) 2 (a)), or, in a wearable eyeglass (shown as (b) in FIG. 2) or a camera provided in a wearable helmet, the built-in camera 13 may specifically be an infrared camera, R (red, The red (green) green (green) B (blue, blue) camera or the like is not limited in this embodiment of the present invention.

The external camera of the mobile device refers to the camera set on the side facing away from the display when the user views the display screen of the mobile device, for example, the rear camera of the mobile phone or the camera set outside the wearable glasses or the wearable helmet. The external camera may specifically be an RGB camera, a Mono (black and white) camera, or a depth camera, etc., which is not limited in this embodiment of the present invention.

Embodiments of the present invention provide a processing method, which can be applied to any head mounted electronic device, for example, Augmented Reality (AR) glasses, Virtual Reality (VR) helmets, etc., and can also be applied to Any handheld electronic device, such as a mobile phone, a tablet computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), etc., in the embodiment of the present invention There are no restrictions on this.

The structure of the mobile device is shown in Figure 3. The mobile device 300 includes components such as a communication interface 320, a memory 330, an input unit 340, a display unit 350, a built-in camera 13, an external camera 14, an audio circuit 370, and a processor 380. It should be understood that the structure of the mobile device 300 illustrated in FIG. 3 does not constitute a limitation to the mobile device, which may include more or fewer components than those illustrated, or some components, or other different components.

The components of the mobile device 300 will be specifically described below with reference to FIG. 3:

The communication interface 320 is configured to receive and transmit signals during and after receiving or transmitting information or during a call. Typically, communication interfaces include, but are not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier (LNA), a duplexer, and the like.

The memory 330 can be used to store software programs and modules, and the processor 380 executes various functional applications and data processing of the mobile device 300 by running software programs and modules stored in the memory 330. The memory 330 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application required for at least one function (such as a camera function, an image playing function, etc.), and the like; the storage data area may be stored according to the movement. The data created by the use of the device 300 (such as audio data, image data, phone book, etc.) and the like.

The input unit 340 can be configured to receive input numeric or character information and to generate key signal inputs related to user settings and function control of the mobile device 300. In particular, the input unit 340 can include a touch screen 341 as well as other input devices 342. The touch screen 341, also referred to as a touch panel, can detect a touch operation on or near the user and drive the corresponding connection device according to a preset program. In addition to the touch screen 341, the input unit 340 may also include other input devices 342. In particular, other input devices 342 may include, but are not limited to, one or more of a physical keyboard, function keys (such as volume control buttons, power switch buttons, etc.), trackballs, mice, joysticks, and the like.

The display unit 350 can be used to display information input by the user or information provided to the user and various menus of the mobile device 300. The display unit 350 may include a display panel 351. Further, when the touch screen 341 detects a touch operation on or near it, it can be transmitted to the processor 380 to determine the type of the touch event, and then the processor 380 provides a corresponding visual output on the display panel 351 according to the type of the touch event. .

Audio circuitry 370, speaker 371, microphone 372 can provide an audio interface between the user and mobile device 300. The audio circuit 370 can transmit the converted electrical data of the received audio data to the speaker 371 for conversion to the sound signal output by the speaker 371; on the other hand, the microphone 372 converts the detected sound signal into an electrical signal by the audio circuit. The 370 is converted to audio data after reception, and the audio data is output to the communication interface 320 for transmission to, for example, another mobile device, or the audio data is output to the memory 330 for further processing.

Processor 380 is the control center of mobile device 300, which connects various portions of the entire mobile device using various interfaces and lines, by running or executing software programs and/or modules stored in memory 330, and by calling stored in memory 330. Data, performing various functions and processing data of the mobile device 300, thereby performing overall monitoring of the mobile device 300. Alternatively, processor 380 can include one or more processing units.

The mobile device 300 can also include at least one built-in camera 13 and at least one external camera 14. The built-in camera 13 or the external camera 14 can transmit the captured preview frame to the processor 380. After the image processing is performed on the preview frame by the processor 380, the processed image is sent to the display unit 350 for display.

Specifically, in the processing method provided by the embodiment of the present invention, as shown in FIG. 4, when the user triggers the target content positioning function in the mobile device (taking the head mounted electronic device in FIG. 4 as an example), for example, user setting The processor 380 can call the external camera 14 to acquire the current preview frame 11 and display the preview frame 11 in the display unit 350. Further, the processor 380 can call the preset in the memory 330. The quadrilateral detection algorithm acquires vertex coordinates of at least two quadrilaterals in the preview frame 11, for example, the vertex coordinates of the quadrilateral A in FIG. 4, the vertex coordinates of the quadrilateral B, the vertex coordinates of the quadrilateral C, and the vertex coordinates of the quadrilateral D.

At this time, the processor 380 can call the built-in camera 13 to detect the user's line of sight, determine the duration of the user's line of sight focus staying in the preview frame 11 and the position of the stay. If the user's line of sight focus stays in the quadrilateral B for longer than the preset duration, Then, as shown in FIG. 4, the processor 380 highlights the outline of the quadrilateral B in the preview frame 11 through the display unit 350, that is, the processor 380 uses the first image surrounded by the outline of the quadrilateral B as the preview image. Target content.

Then, if the processor 380 detects the first photographing instruction, the processor 380 performs preset processing on the first image, and outputs the preset processed first image in the display unit 350.

If the first photographing instruction is triggered when the user starts the document correction function, the preset processing may specifically be: performing quadrilateral correction on the first image, for example, performing tilt correction on the first image; The instruction is triggered when the user initiates the document translation function, and the preset processing may be: performing optical character recognition (OCR) on the text in the first image, for example, translating the Chinese characters in the first image into English.

It can be seen that, in the processing method provided by the embodiment of the present invention, the mobile device determines, by detecting the focus of the user's line of sight, which quadrant in the preview frame the user's line of sight stays for a longer time, because the line of sight stays longer in the quadrilateral. The image is usually the target content that the user needs. Then, the mobile device highlights the edge line of the quadrilateral, thereby prompting the user that the first image in the quadrilateral has been determined as the target content for the user, and subsequently, the mobile device detects the photo. After the instruction, the first image subjected to the preset processing may be directly output. In this way, the mobile device can accurately and conveniently locate the target content in the preview frame without the user manually adjusting the preview frame, and the mobile device only needs to perform the preset processing on the first image in the preview frame, thereby avoiding Processing of invalid data outside the first image, thereby improving image processing efficiency.

It should be noted that the foregoing processing method can be applied to various application scenarios such as document correction and document translation, and can also be applied to various application scenarios based on target content positioning, for example, business card recognition, bank card identification, ID card recognition, and the like. The embodiment of the present invention does not impose any limitation on this.

It should be understood that although not shown in FIG. 3, the mobile device 300 may further include a power source, a gravity sensor, a light sensor, a gyroscope, a barometer, a hygrometer, a thermometer, an infrared sensor, and a wireless fidelity ( Wireless fidelity, Wi-Fi module or Bluetooth module, etc., will not be described here.

Hereinafter, a processing method provided by an embodiment of the present invention is described in detail in conjunction with a specific embodiment. As shown in FIG. 5, the method includes:

401. The mobile device performs a quadrilateral detection operation on the content in the preview frame.

402. The mobile device saves vertex coordinates of each of the detected at least two quadrilaterals.

When the user turns on the function of positioning the target content, such as document correction or document recognition, the mobile device can call the external camera to enter the preview state. At this time, the mobile device displays the image captured by the external camera in the form of a preview frame on the mobile device. Within the display interface.

Specifically, in step 401, the mobile device performs a quadrilateral detection operation on the picture content in the preview frame, thereby determining one or more quadrilaterals included in the preview frame.

For example, the edge of the preview frame may be edge-detected, and then the straight line included in the preview frame may be extracted, and finally, according to a preset quadrilateral detection algorithm, for example, based on an open source computer vision library (Open Source Computer Vision) Library, OpenCV) Rectangular recognition algorithm, etc., determines one or more quadrilaterals composed of these straight lines.

Of course, since there may be many quads in the preview frame, and the content of the quadrilateral in the area or the circumference is too small may not be the target content required by the user, the quadrilateral detection operation may also be performed according to the size of the quadrilateral. Screening, discarding quadrilaterals with an area or perimeter less than the preset value, avoiding these quadrilaterals with smaller areas or perimeters from interfering with subsequent image processing. In addition, it is also possible to perform noise filtering and the like on the screen content in the preview frame to improve the accuracy of the quadrilateral detection operation.

In step 402, the mobile device saves the vertex coordinates of each of the detected at least two quadrilaterals to determine the position of each quadrilateral detected within the preview frame.

As shown in FIG. 6, the mobile device displays the preview frame in the preview frame 11 of the display unit. If the captured image in the preview frame 11 is an A4 paper, it can be determined that the preview frame is included by the quadrilateral detection operation. Five quadrangles, that is, a quadrilateral 1 formed by the edge of the A4 paper, and a quadrilateral 2-quadrant 5 included in the A4 paper, then the mobile device can detect the vertices of each of the five quadrilaterals by performing quadrilateral detection. Coordinates to determine the position of the five quadrilaterals in the preview frame 11.

403 (optional), the mobile device highlights a contour of the default quadrilateral in the preview frame; the default quadrilateral is the quadrilateral having the largest area among the at least two quadrilaterals.

Optionally, in step 403, the mobile device may determine the quadrilateral with the largest area from the at least two quadrilaterals detected above, and use the quadrilateral as the default quadrilateral to highlight the outline of the default quadrilateral in the preview frame.

Among them, highlighting refers to the function of highlighting text or image using lines of different colors or different thicknesses; highlighting can be clearly distinguished by visual observation compared with normal display.

As shown in FIG. 7, in the quadrilateral 1-quadrant 5, the largest quadrilateral is a quadrilateral 1, that is, a quadrangle formed by the edge of the A4 paper, then the mobile device can highlight the outline of the quadrilateral 1 (in FIG. 7). The bold display is indicated as a highlight, so that the user is prompted to display the quadrilateral with the largest area in the current preview frame as the quadrilateral 1, and the content surrounded by the outline of the quadrilateral 1 is the current shift. The default device waiting for the preset processing image.

The method for displaying the outline of the default quadrilateral may be various. In the above embodiment, the highlighting is taken as an example, and the display is also performed by using a dotted line or the like, which is not limited in this embodiment of the present invention.

404. The mobile device detects, according to vertex coordinates of each quadrilateral, a focus position of the user's line of sight focus within the preview frame.

405. If it is determined that the duration of the line of sight focus staying in the first quadrilateral is greater than the preset duration, the mobile device highlights the contour of the first quadrilateral in the preview frame, where the first quadrilateral is at least two quadrilaterals. one of.

In steps 404 and 405, the mobile device can capture the motion state of the user's eyeball by calling the built-in camera, and then use the eyeball tracking technology to detect the position of the user's line of sight focus within the preview frame, that is, the focus position.

Illustratively, the user and the eyeball and the image around the eyeball can be captured by the built-in camera. Further, the mobile device can track the eyeball and the feature changes around the eyeball to determine the user's line of sight focus position; or, the mobile device can be based on the iris The angle change is tracked to determine the user's line of sight focus position, or the mobile device can actively project a beam of infrared light to the iris to extract feature information for tracking to determine the user's line of sight focus position.

In a possible design method, after the mobile device determines that the focus position or the focus position changes, two threads can be started in parallel, and one thread is used to determine which current focus position is located according to the vertex coordinates of each quadrature described above. Within the quadrilateral, another thread can be used to record how long the user's line of sight focus stays at that focus position.

Since the user's line of sight focus may change slightly within a certain period of time, the user may be considered to be the user when the line of sight focus falls within the area centered on the focus position and the radius is R (R is the preset value). The line of sight focus stays at the above focus position.

Then, if the mobile device determines that the focus position is in the first quadrilateral (the first quadrilateral is one of the at least two quadrilaterals), and the user's line of sight focus stays at the focus position for longer than the preset duration (for example, 2 seconds), indicating that the user's current attention is concentrated in the first quadrilateral. At this time, the triggering mobile device displays the outline of the first quadrilateral in the preview frame to prompt the user to move. The current document determined by the device for the user is the first image surrounded by the outline of the first quadrilateral.

In another possible design method, after the mobile device determines the focus position, it may first determine, according to the vertex coordinates of each quadrilateral, which quadrilateral the focus position is located in, for example, located in the first quadrilateral, and further, determine Whether the duration of the user's line of sight focus staying within the first quadrilateral is greater than a preset duration.

Alternatively, after determining the focus position, the mobile device may first determine whether the duration of the user's line of sight focus staying at the focus position is greater than a preset duration. If the duration is greater than the preset duration, the focus position is further determined according to the coordinates of the vertices of each of the quadrilaterals. Which quadrilateral is located.

As shown in FIG. 8, the mobile device detects that the focus position of the user's line of sight focus in the preview frame is point A by calling the built-in camera. Then, the mobile device can determine the point A of each quadrilateral as described above, and determine that the point A is located in the quadrilateral 2, at this time, if the user's line of sight focus is at the focus position If the duration of the stay is greater than 2 seconds, the mobile device highlights the outline of the quadrilateral 2 in the preview frame (indicated by a bold display in FIG. 8 as a highlight), that is, prompting the user that the current mobile device is the current document determined by the user. An image enclosed by the outline of the quadrilateral 2.

At this time, the mobile device can also normally display the outline of the default quadrilateral (ie, quadrilateral 1) that has been displayed in step 403.

Of course, as shown in FIG. 9, if it is detected that the focus position of the user's line of sight focus in the preview frame is point B, since the point B is not in the quadrilateral 2-quadrant 5 but in the quadrilateral 1, the mobile device continues. The outline of the quadrilateral 1 that has been highlighted in step 403 is displayed.

406. After detecting the first photographing instruction, the mobile device performs a preset process on the first image surrounded by the outline of the first quadrilateral, and then outputs the first image subjected to the preset processing.

In summary, the “outputting the first image processed by the preset process” as referred to herein may be understood as an image obtained by the user after clicking the camera key, and the first image processed by the preset process is saved in the memory of the mobile device. For example, the first image subjected to the preset processing is as shown in FIG. 11, and it should be understood that the first image subjected to the preset processing may include all or part of the content in the quadrilateral 2 in FIG. 9; The first image subjected to the preset processing may include a part of the content in the quadrilateral 1 as shown in FIG. 9 in addition to all or part of the content in the quadrilateral 2 in FIG. 9; or, the It is assumed that the processed first image may include other content in addition to all or part of the content in the quadrilateral 2 in FIG.

In step 406, if the user determines that the first quadrilateral is the target content, the mobile device may be triggered to generate a first photographing instruction. For example, as shown in FIG. 10, after the user clicks the photographing button in the mobile device display interface, the mobile device A first photographing instruction is generated. Further, after the first photographing instruction is generated, the mobile device photographs the first image surrounded by the outline of the first quadrilateral, and performs preset processing on the first image obtained by photographing.

For example, the preview frame 11 shown in FIG. 10 is used as an example. If the user activates the document correction function and then triggers the camera button of the mobile device, the mobile device generates a first camera instruction in response to the operation, and further, according to the first camera instruction. Taking a picture of the first image enclosed by the outline of the quadrilateral 2, and performing quadrilateral correction on the obtained first image, for example, performing tilt correction on the first image, as shown in FIG. 11, obtaining the first image after tilt correction, Then, for the user to display the tilt-corrected first image in the mobile device, the document correction function is completed.

Or, if the user initiates the document translation function and triggers the camera button of the mobile device, at this time, the mobile device generates a first camera instruction in response to the operation, and further, according to the first camera instruction, the outline surrounded by the outline of the quadrilateral 2 An image is taken. Unlike the above method, the preset processing at this time is: performing OCR on the text in the first image.

Specifically, when the OCR is executed, the text in the first image may be first recognized by a preset character recognition algorithm to obtain computer characters recognizable by the mobile device, and further, the computer text may be obtained through a preset translation software or program. Translating into the text of the default language, for example, translating the Chinese in the first image into English, and subsequently, as shown in (a) of FIG. 12, the translated English can be directly used in the first image to replace the pre-translation. Chinese, or, as shown in (b) of FIG. 12, the translated English may also be added to the first image in the first image, for example, the translated image English is added next to the pre-translation Chinese to complete the document translation function.

It can be seen that when performing the above document translation function, only the characters in the first quadrilateral need to be recognized and translated, which can effectively avoid the interference of useless data in the outer area of the first quadrilateral, thereby improving the text. Speed and accuracy of recognition and translation.

Further, the foregoing preset processing may further include performing a cropping operation or the like on the first image, which is not limited in this embodiment of the present invention. In addition, before performing the above document translation function, quadrilateral correction can also be performed on the first image, thereby improving the accuracy of subsequent character recognition and translation.

Further, after performing the above step 404, if the mobile device detects that the user's line of sight focus position changes, the mobile device may perform the following steps 501-504.

Exemplarily, the mobile device may periodically determine the position of the user's line of sight focus. When the distance between the two determined focus positions is greater than the preset distance (the preset distance is greater than the radius R), the user's line of sight may be determined. The focus position changes. At this time, the mobile device may perform the following steps 501-504. When the distance between the focus positions determined twice consecutive times is less than or equal to the preset distance, it may be determined that the user's line of sight focus position has not changed. At this point, the mobile device can continue to perform steps 405-406 above.

Specifically, as shown in FIG. 13, steps 501-504 include:

501. The mobile device determines that a duration in which the user's line of sight stays in the second quadrilateral is greater than a preset duration, and the second quadrilateral is any one of the at least two quadrilaterals except the first quadrilateral.

Exemplarily, similar to step 405, after the user's line of sight focus position changes, the mobile device can determine that the new focus position is within the second quadrilateral according to the vertex coordinates of each of the above quadrilaterals. For example, as shown in (a) of FIG. 14, the new focus position C is located in the quadrilateral 5, and further, the mobile device can determine whether the duration of the user's line of sight focus staying within the quadrilateral 5 is greater than a preset duration, if greater than the pre- If the duration is set, the following steps 502-204 are continued.

502. The mobile device displays the outline of the first quadrilateral normally in the preview frame.

503. The mobile device highlights a contour of the second quadrilateral in the preview frame.

Different from step 405, in steps 502-203, since the mobile device has highlighted the contour of the first quadrilateral in the preview frame in step 405, the user's line of sight focus position has changed, new The focus position is located within the second quadrilateral, so the mobile device can normally display the outline of the first quadrilateral within the preview frame, ie, cancel or hide the outline of the first quadrilateral that has been highlighted in step 405, Restores the original display mode of the first quadrilateral in the preview frame; and highlights the outline of the second quadrilateral in the preview frame, thereby prompting the user that the current mobile device determines that the current document is the second quadrilateral The image enclosed by the outline.

Of course, the mobile device may also perform step 503 and then perform step 502. Alternatively, the mobile device may perform step 502 and step 503 at the same time.

Exemplarily, as shown in (a) of FIG. 14, the current new focus position is C, the mobile device determines that the C point is within the quadrilateral 5, and the line of sight focus stays within the quadrilateral 5 for longer than the preset duration. Then, the mobile device can cancel or hide the outline of the highlighted quadrilateral 2 in the preview frame 11 (ie, the outline of the normal display quadrilateral 2), and at the same time, highlight the outline of the quadrilateral 5 (in FIG. 14 (a) The bold display is indicated as a highlight), that is, the image that the current mobile device determines for the user that the current document is surrounded by the outline of the quadrilateral 5.

As a possible design, when the mobile device determines that the C point is within the quadrilateral 5, and the line of sight focus stays within the quadrilateral 5 for longer than the preset duration, the mobile device can also display only the content of the quadrilateral 5 in the preview frame 11. As shown in (b) of FIG. 14, the current document identified by the user for the current mobile device is an image surrounded by the outline of the quadrilateral 5. At this time, the image surrounded by the outline of the quadrilateral 5 may occupy the entire preview frame 11 or may be located in a certain area in the preview frame 11; the outline of the quadrilateral 5 may be highlighted or displayed normally, in the embodiment of the present invention. There are no restrictions on this.

504. After detecting the second photographing instruction, the mobile device performs a preset process on the second image surrounded by the outline of the second quadrilateral, and outputs a second image that passes the preset processing.

In summary, the “outputting the second image subjected to the preset processing” as referred to herein may be understood as an image obtained after the user clicks the camera key, and the first image processed through the preset process is saved in the memory of the mobile phone. It should be understood that the second image subjected to the preset processing may include all content or partial content in the quadrilateral 5 as shown in FIG. 14; or the second image subjected to the preset processing may be included in FIG. 14 except In addition to all or part of the content in the quadrilateral 5, part of the content in the quadrilateral 1 in FIG. 14 is included; or the second image subjected to the preset processing may include all the contents in the quadrilateral 5 as shown in FIG. In addition to or in part, it may include other content.

Similar to step 406, if the user determines that the second quadrilateral is the target content, the mobile device can be triggered to generate a second photographing instruction. Further, after the second photographing instruction is generated, the mobile device is triggered to take a photo of the second image surrounded by the outline of the second quadrilateral, and perform preset processing on the photographed second image.

For the specific preset processing method, refer to the related description in step 406, and therefore, details are not described herein.

In another possible design method, as shown in FIG. 15, the embodiment of the present invention further provides a processing method, including:

601. The mobile device displays a preview frame in its preview frame, the preview frame including at least two quadrilaterals.

Similar to step 401, when the user turns on the function of positioning the target content, such as document correction or document recognition, the mobile device can call the external camera to enter the preview state. At this time, the mobile device captures the image captured by the external camera as a preview frame. The form is displayed in the display interface of the mobile device.

Specifically, the mobile device may perform quadrilateral detection on the content of the picture in the preview frame, and save the coordinates of the vertices of each of the detected at least two quadrilaterals, thereby determining one or more quadrilaterals included in the preview frame.

Exemplarily, as shown in FIG. 16, the mobile device can determine that the preview frame 11 includes a quadrilateral 1-quadrant 5 by performing quadrilateral detection. At this time, the mobile device highlights the quadrilateral 1-quadrant 5 in the preview frame. Each quad, that is, reminds the user that the user detects a quadrilateral 1-quadrant 5 in the current preview frame. It should be understood that, as an alternative or alternative, the mobile device determines that the preview frame 11 includes a quadrilateral 1-quadrant 5 by performing quadrilateral detection, and the mobile device can also normally display the quadrilateral 1-quadrant 5 in the preview frame.

602. The mobile device detects an operation acting on the first quadrilateral, the first quadrilateral being one of the at least two quadrilaterals.

603. In response to the operation, after performing a preset process on the first image surrounded by the contour line of the first quadrilateral, the mobile device outputs the first image that passes the preset process.

Further, in steps 602-603, the user can click on any quadrilateral displayed in the preview frame. For example, as shown in FIG. 17, the user clicks on any position within the quadrilateral 2, and at this time, the mobile device can detect that the user acts on the quadrilateral. The click operation of 2 triggers the mobile device to perform a preset process on the first image surrounded by the outline of the first quadrilateral, and outputs the first image subjected to the preset process.

In summary, the “outputting the first image processed by the preset process” as referred to herein may be understood as an image obtained by the user after clicking the camera key, and the first image processed by the preset process is saved in the memory of the mobile phone. For example, the first image subjected to the preset processing is as shown in FIG. 18, and it should be understood that the first image subjected to the preset processing may include all or part of the content in the quadrilateral 2 in FIG. 17; The first image subjected to the preset processing may include, in addition to all contents or partial contents in the quadrilateral 2 in FIG. 17, a partial content in the quadrilateral 1 in FIG. 17; or, the It is assumed that the processed first image may include other content in addition to all or part of the content in the quadrilateral 2 in FIG.

Exemplarily, the foregoing preset processing includes: performing quadrilateral correction on the first image; or performing optical character recognition on the text in the first image. For the specific preset processing method, refer to the related description in step 406, and therefore, details are not described herein.

In another possible design method, as shown in FIG. 19, an embodiment of the present invention further provides a processing method, including:

701. The mobile device displays a preview frame in its preview frame, where the preview frame includes at least two quadrilaterals.

Similar to step 401, when the user turns on the function of positioning the target content, such as document correction or document recognition, the mobile device can call the external camera to enter the preview state. At this time, the mobile device can perform quadrilateral detection on the screen content in the preview frame. Determines one or more quads contained within the preview frame and displays the preview frame in its preview frame.

702. The mobile device detects a first operation acting on the first quadrilateral, the first quadrilateral being one of the at least two quadrilaterals.

703. In response to the first operation, the mobile device highlights the outline of the first quadrilateral.

In steps 702-703, the user can click on any quadrilateral displayed in the preview frame. For example, as shown in FIG. 20, the user clicks on any position within the quadrilateral 2. At this time, as shown in FIG. 20, the mobile device can detect The user acts on the click operation of the quadrilateral 2, triggering the mobile device to highlight the outline of the quadrilateral 2 (indicated by the bold display as a highlight in FIG. 21), thereby prompting the user that the current mobile device is the current document determined by the user. An image enclosed by the outline of the quadrilateral 2.

704. The mobile device detects a second operation that acts on the camera button.

705. After performing a preset process on the first image surrounded by the outline of the first quadrilateral, in response to the second operation, outputting the first image that passes the preset process.

The camera button may be a virtual button for triggering the photographing in the display interface, or may be a physical button for triggering the photographing on the mobile device, which is not limited in this embodiment of the present invention.

In steps 704-705, if the user determines that the first quadrilateral is the target content, as shown in FIG. Then, the camera button on the mobile device can be triggered, that is, the second operation, and further, the mobile device takes a photo of the first image surrounded by the outline of the first quadrilateral in response to the second operation, and the first image obtained by photographing After the preset process is performed, the first image subjected to the preset process is output.

In summary, the “outputting the first image processed by the preset process” as referred to herein may be understood as an image obtained by the user after clicking the camera key, and the first image processed by the preset process is saved in the memory of the mobile phone. For example, the first image subjected to the preset processing is as shown in FIG. 23, and it should be understood that the first image subjected to the preset processing may include all or part of the content in the quadrilateral 2 in FIG. 22; The first image subjected to the preset processing may include a part of the content in the quadrilateral 1 as shown in FIG. 22 in addition to all or part of the content in the quadrilateral 2 in FIG. 22; or, the It is assumed that the processed first image may include other content in addition to all or part of the content in the quadrilateral 2 in FIG.

Exemplarily, the foregoing preset processing includes: performing quadrilateral correction on the first image; or performing optical character recognition on the text in the first image. For the specific preset processing method, refer to the related description in step 406, and therefore, details are not described herein.

It can be understood that, in order to implement the above functions, the above mobile device or the like includes a hardware structure and/or a software module corresponding to each function. Those skilled in the art will readily appreciate that the embodiments of the present invention can be implemented in a combination of hardware or hardware and computer software in combination with the elements and algorithm steps of the various examples described in the embodiments disclosed herein. Whether a function is implemented in hardware or computer software to drive hardware depends on the specific application and design constraints of the solution. A person skilled in the art can use different methods to implement the described functions for each particular application, but such implementation should not be considered to be beyond the scope of the embodiments of the invention.

The embodiment of the present invention may divide a function module into a mobile device or the like according to the foregoing method example. For example, each function module may be divided according to each function, or two or more functions may be integrated into one processing module. The above integrated modules can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of the module in the embodiment of the present invention is schematic, and is only a logical function division, and the actual implementation may have another division manner.

In the case where the respective functional modules are divided by corresponding functions, FIG. 24 shows a possible structural diagram of the mobile device involved in the above embodiment.

The mobile device includes a detecting unit 1101, a display unit 1102, a determining unit 1103, and a processing unit 1104. The detecting unit 1101 is configured to support the mobile device to perform the processes 401-402 in FIG. 4, or the process 602 in FIG. 15, or the processes 702, 704 in FIG. 18; the display unit 1102 is configured to support the mobile device to perform the process in FIG. Process 403, 405, or process 502-503 in FIG. 10, or process 601 in FIG. 15, or process 701, 703 in FIG. 18; determination unit 1103 is used to support the mobile device to perform the process in FIG. Process 404, or process 501 in FIG. 10; processing unit 1104 is configured to support the mobile device to perform 406 in FIG. 4, or process 504 in FIG. 10, or process 603 in FIG. 15, or, in FIG. Process 705. All the related content of the steps involved in the foregoing method embodiments may be referred to the functional descriptions of the corresponding functional modules, and details are not described herein again.

In the case of employing an integrated unit, FIG. 25 shows a possible structural diagram of the mobile device involved in the above embodiment. The mobile device includes a processing module 1302 and a communication module 1303. The processing module 1302 is configured to control and manage the actions of the mobile device. For example, the processing module 1302 is configured to support the mobile device to perform the processes 401-406 in FIG. 5, or the processes 501-504 in FIG. 13, or, in FIG. Processes 601-603, or processes 701-705 in Figure 19, and/or other processes for the techniques described herein. The communication module 1303 is configured to support communication of the mobile device with other network entities. The mobile device can also include a storage module 1301 for storing program code and data of the mobile device.

The processing module 1302 may be a processor or a controller, for example, may be a central processing unit (CPU), a general-purpose processor, a digital signal processor (DSP), and an application specific integrated circuit (Application-Specific Integrated Circuit (ASIC), Field Programmable Gate Array (FPGA) or other programmable logic device, transistor logic device, hardware component, or any combination thereof. It is possible to implement or carry out various exemplary logical blocks, modules and circuits described in connection with the disclosure of the embodiments of the invention. The processor may also be a combination of computing functions, for example, including one or more microprocessor combinations, a combination of a DSP and a microprocessor, and the like. The communication module 1303 may be a transceiver, a transceiver circuit (eg, an RF circuit), a communication interface, or the like. The storage module 1301 may be a memory.

When the processing module 1302 is a processor, the communication module 1303 is a communication interface, and the storage module 1301 is a memory, the specific hardware structure of the mobile device according to the embodiment of the present invention may be the mobile device 300 shown in FIG.

Further, an embodiment of the present invention further provides a computer program, where the computer program includes instructions, when the computer program is executed by a computer, may cause the computer to perform the above steps 401-406, or steps 501-504, or steps 601-603, or the related processing methods in steps 701-705.

Further, an embodiment of the present invention further provides a computer readable storage medium for storing computer software instructions used by the mobile device, including any program designed to execute the mobile device.

Through the description of the above embodiments, those skilled in the art can clearly understand that for the convenience and brevity of the description, only the division of the above functional modules is illustrated. In practical applications, the above functions can be allocated according to needs. It is completed by different functional modules, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. For the specific working process of the system, the device and the unit described above, reference may be made to the corresponding process in the foregoing method embodiments, and details are not described herein again.

In the several embodiments provided by the present application, it should be understood that the disclosed system, apparatus, and method may be implemented in other manners. For example, the device embodiments described above are merely illustrative. For example, the division of the modules or units is only a logical function division. In actual implementation, there may be another division manner, for example, multiple units or components may be used. Combinations can be integrated into another system, or some features can be ignored or not executed. Alternatively, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interface, device or unit. The coupling or communication connection can be in electrical, mechanical or other form.

The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

The integrated unit, if implemented in the form of a software functional unit and sold or used as a standalone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application, in essence or the contribution to the prior art, or all or part of the technical solution may be embodied in the form of a software product stored in a storage medium. A number of instructions are included to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to perform all or part of the steps of the methods described in various embodiments of the present application. The foregoing storage medium includes: a flash memory, a mobile hard disk, a read only memory, a random access memory, a magnetic disk, or an optical disk, and the like, which can store program codes.

In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented in software, it may occur in whole or in part in the form of a computer program product. The computer program product therefore includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in accordance with embodiments of the present invention are generated in whole or in part. The computer can be a general purpose computer, a special purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer readable storage medium or transferred from one computer readable storage medium to another computer readable storage medium, for example, the computer instructions can be from a website site, computer, server or data center Transfer to another website site, computer, server, or data center by wire (eg, coaxial cable, fiber optic, digital subscriber line (DSL), or wireless (eg, infrared, wireless, microwave, etc.). The computer readable storage medium can be any available media that can be accessed by a computer or a data storage device such as a server, data center, or the like that includes one or more available media. The usable medium can be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a DVD), or a semiconductor medium (eg, a Solid State Disk (SSD)).

The foregoing is only a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto, and any changes or substitutions within the technical scope of the present application should be covered by the scope of the present application. . Therefore, the scope of protection of the present application should be determined by the scope of the claims.

Claims (22)

A processing method, the method being applied to a mobile device, comprising: Displaying a preview frame in a preview box of the mobile device, the preview frame including at least two quadrilaterals; Determining that the duration of the line of sight of the user staying within the first quadrilateral is greater than a preset duration, the first quadrilateral being one of the at least two quadrilaterals; Highlighting a contour line of the first quadrilateral in the preview frame; After the first photographing instruction is detected, after the preset processing is performed on the first image surrounded by the outline of the first quadrilateral, the first image subjected to the preset processing is output; the preset processing includes: Performing quadrilateral correction on the first image; or performing optical character recognition on the text in the first image. The method according to claim 1, wherein after highlighting the outline of the first quadrilateral in the preview frame, the method further comprises: Determining that a duration in which the user's line of sight focus stays within the second quadrilateral is greater than the predetermined length of time, and the second quadrilateral is any one of the at least two quadrilaterals except the first quadrilateral ; Displaying the outline of the first quadrilateral normally in the preview frame; An outline of the second quadrilateral is highlighted within the preview frame. The method according to claim 2, further comprising: after highlighting the outline of the second quadrilateral in the preview frame frame, further comprising: After detecting the second photographing instruction, after performing the preset processing on the second image surrounded by the outline of the second quadrilateral, outputting the second image subjected to the preset processing. The method according to any one of claims 1 to 3, further comprising: after displaying the preview frame in the preview box of the mobile device, A contour of a default quadrilateral is highlighted within the preview frame, and the default quadrilateral is a quadrilateral having the largest area among the at least two quadrilaterals. The method according to any one of claims 1 to 4, further comprising: before displaying the preview frame in the preview box of the mobile device, Quadrilateral detection is performed on the preview frame, and the detected vertex coordinates of at least two quadrilaterals are saved. The method according to any one of claims 1 to 5, wherein determining that the user's line of sight focus stays within the first quadrilateral is greater than a preset duration includes: Detecting a focus position of the line of sight focus within the preview frame; Determining that the focus position is within the first quadrilateral of the preview frame is longer than the preset duration. The method of any of claims 1-6, wherein performing optical character recognition on the text in the first image comprises: Translating the text in the first image into a text of a preset language, and replacing the text in the first image with the translated text in the preset language; or Translating the text in the first image into a text of a preset language, and adding the translated text of the preset language to the first image. The method according to any one of claims 1 to 7, wherein the preset processing The method further includes performing a cropping operation on the first image. A processing method, the method being applied to a mobile device, comprising: Performing a quadrilateral detection on the display preview frame, and storing the detected vertex coordinates of at least two quadrilaterals; Displaying the preview frame in a preview box of the mobile device; Determining that the line of sight focus is within the first quadrilateral of the at least two quadriles for more than a preset duration, and presenting the outline of the first quadrilateral in the preview frame; An instruction for photographing is detected; After performing a preset process on the first image surrounded by the contour line of the first quadrilateral in the preview frame, outputting the first image subjected to the preset process, the first passing the preset process The image includes a first content surrounded by the contour line of the first quadrilateral, and the preset processing includes at least one of the following: quadrilateral correction; performing optical character recognition on the text in the first image, and identifying the recognized text Translating into a text of a preset language, and replacing the recognized text with the text of the preset language; performing optical character recognition on the text in the first image, and translating the recognized text into a text of a preset language And adding the text of the preset language to the side of the recognized text. A processing method, the method being applied to a mobile device, comprising: Performing a quadrilateral detection on the display preview frame, and storing the detected vertex coordinates of at least two quadrilaterals; Displaying the preview frame in a preview box of the mobile device, and presenting a contour line of the first quadrilateral of the at least two quadrilaterals in the preview frame; Determining that the line of sight focus is within a second quadrilateral of the at least two quadriles for more than a preset duration, presenting the outline of the second quadrilateral in the preview frame, hiding the first quadrilateral a contour line, wherein the second quadrilateral is any one of the at least two quadrilaterals except the first quadrilateral; An instruction for photographing is detected; After performing a preset process on the second image surrounded by the outline of the second quadrilateral in the preview frame, outputting a second image that passes the preset process, and the second process that passes the preset process The image includes a second content surrounded by the contour line of the second quadrilateral, the preset processing includes at least one of the following: quadrilateral correction; performing optical character recognition on the text in the second image, and identifying the recognized text Translating into a text of a preset language, and replacing the recognized text with the text of the preset language; performing optical character recognition on the text in the second image, and translating the recognized text into a text of a preset language And adding the text of the preset language to the side of the recognized text. A processing method, the method being applied to a mobile device, comprising: Displaying a preview frame in a preview box of the mobile device, the preview frame including at least two quadrilaterals; Detecting an operation acting on the first quadrilateral, the first quadrilateral being one of the at least two quadrilaterals; In response to the operation, after performing a preset process on the first image surrounded by the outline of the first quadrilateral, outputting the first image subjected to the preset process; the preset process includes: The first image performs quadrilateral correction; or, optical character recognition is performed on the text in the first image. A processing method, the method being applied to a mobile device, comprising: Displaying a preview frame in a preview box of the mobile device, the preview frame including at least two quadrilaterals; Detecting a first operation acting on the first quadrilateral, the first quadrilateral being one of the at least two quadrilaterals; Highlighting the outline of the first quadrilateral in response to the first operation; A second operation acting on the camera button is detected; After performing the preset processing on the first image surrounded by the outline of the first quadrilateral, outputting the first image subjected to the preset processing in response to the second operation; the preset processing includes: Performing quadrilateral correction on the first image; or performing optical character recognition on the text in the first image. A mobile device, comprising: a display unit, configured to display a preview frame in a preview frame of the mobile device, where the preview frame includes at least two quadrangles; a determining unit, configured to determine that a duration in which the user's line of sight focus stays within the first quadrilateral is greater than a preset duration, the first quadrilateral being one of the at least two quadrilaterals; The display unit is further configured to highlight an outline of the first quadrilateral in the preview frame; a processing unit, configured to perform a preset process on the first image surrounded by the outline of the first quadrilateral after detecting the first photographing instruction, and output a first image that passes the preset processing; The preset processing includes: performing quadrilateral correction on the first image; or performing optical character recognition on the text in the first image. The mobile device of claim 13 wherein: The determining unit is further configured to determine that a duration of the line of sight of the user staying in the second quadrilateral is greater than the preset duration, and the second quadrilateral is the first of the at least two quadrilaterals Any quadrilateral outside the quadrilateral; The display unit is further configured to normally display an outline of the first quadrilateral in the preview frame; and highlight an outline of the second quadrilateral in the preview frame. The mobile device of claim 14 wherein: The processing unit is further configured to: after detecting the second photographing instruction, performing the preset processing on the second image surrounded by the contour line of the second quadrilateral, and outputting the Two images. Mobile device according to any of the claims 13-15, characterized in that The display unit is further configured to highlight a contour of a default quadrilateral in the preview frame, wherein the default quadrilateral is a quadrilateral having the largest area among the at least two quadrilaterals. The mobile device according to any one of claims 13 to 16, wherein the mobile device further comprises: And a detecting unit, configured to perform quadrilateral detection on the preview frame, and save the detected vertex coordinates of at least two quadrilaterals. A mobile device according to any of claims 13-17, wherein The determining unit is specifically configured to detect a focus position of the user's line of sight focus in the preview frame, and determine that the focus position is within the first quadrilateral of the preview frame is greater than the preset duration . Mobile device according to any of the claims 13-18, characterized in that The processing unit is configured to translate the text in the first image into a text in a preset language, and replace the text in the first image with the translated text in the preset language; or Translating the text in the first image into a text of a preset language, and adding the translated text of the preset language to the first image. Mobile device according to any of the claims 13-19, characterized in that The processing unit is specifically configured to translate the text in the first image into a text in a preset language, and replace the text in the first image with the text in the preset language after translation; or The text in the first image is translated into a text of a preset language, and the translated text of the preset language is added to the first image. A mobile device, comprising: one or more processors, a memory, a bus system, and one or more programs, the processor and the memory being connected by the bus system; Wherein the one or more programs are stored in the memory, the one or more programs comprising instructions that, when executed by the mobile device, cause the mobile device to perform as claimed in claims 1-12 The processing method according to any one of the preceding claims. A computer readable storage medium, comprising: instructions for causing the mobile device to perform the method of any of claims 1-12 when the instructions are executed by a mobile device Approach.

Images