CN105975566A - Image-based information searching method and device - Google Patents

Abstract

The invention is suitable for the technical field of electronic terminals, and provides an image-based information searching method and device, wherein the image-based information searching method and device comprise the following steps: starting a camera of the mobile terminal and starting a light emitting device; acquiring a first image shot by the camera, wherein in the first image, a light projection point of the light emitting device is positioned at a first vertex of a rectangular area where a search object is positioned; acquiring a second image shot by the camera, wherein in the second image, a light projection point of the light emitting device is positioned at a second vertex of the rectangular area, and the second vertex and the first vertex are opposite vertices of the rectangular area; constructing a rectangular frame based on the positions of the light projection points in the first image and the second image; and performing content recognition on the image in the rectangular frame to determine a search object, and acquiring a search result. The invention reduces the difficulty of intercepting the searched object in the image and improves the information matching efficiency in the searching process.

Description

Image-based information search method and device

technical field

The invention belongs to the technical field of electronic terminals, and in particular relates to an image-based information search method and device.

Background technique

At present, many electronic education products on the market can realize the function of searching for questions by taking photos, searching for corresponding definitions by taking photos of words or phrases, or searching for corresponding answers by taking photos of questions. The realization of the photo search function needs to go through the following steps: start the camera - focus and take a picture - intercept the search object in the image - convert the search object into text through OCR technology - search and match the search result by the server - feed back the search result to the electronic terminal.

In the process of realizing the photo search function, the distance between the search object and the non-search object on the page may be very narrow, or the search object captured by the camera is not parallel to the page, which may cause the image to be tilted. The difficulty of intercepting the search object increases the difficulty of identifying the search object in the background of the system, and reduces the information matching efficiency in the process of taking pictures and searching for questions.

Contents of the invention

In view of this, the embodiments of the present invention provide an image-based information search method and device to solve the problem that the prior art increases the difficulty of intercepting search objects in images and reduces the efficiency of information matching in the search process.

In the first aspect, an image-based information search method is provided, including:

Start the camera of the mobile terminal, and start the light emitting device;

Acquiring a first image taken by the camera, in the first image, the light projection point of the light emitting device is located at the first vertex of the rectangular area where the search object is located;

Acquiring a second image taken by the camera, in the second image, the light projection point of the light emitting device is located at a second vertex of the rectangular area, and the second vertex and the first vertex are the same The diagonal vertices of the rectangular area;

constructing a rectangular frame based on positions of light projection points in the first image and the second image;

Perform content recognition on the images in the rectangular frame to determine the search object, and obtain search results.

In a second aspect, an image-based information search device is provided, including:

The starting unit is used to start the camera of the mobile terminal and start the light emitting device;

A first acquiring unit, configured to acquire a first image captured by the camera, in the first image, the light projection point of the light emitting device is located at the first vertex of the rectangular area where the search object is located;

The second acquiring unit is configured to acquire a second image captured by the camera, in the second image, the light projection point of the light emitting device is located at a second vertex of the rectangular area, and the second vertex is in the same position as the second vertex. The first vertex is a diagonal vertex of the rectangular area;

a construction unit, configured to construct a rectangular frame based on positions of light projection points in the first image and the second image;

The search unit is configured to identify the content of the images in the rectangular frame to determine the search object, and obtain the search result.

In the embodiment of the present invention, the search object is positioned through the light projection point generated by the light emitting device, so as to efficiently intercept the rectangular area where the search object is located, reducing the interference of non-search objects on the search results, thereby reducing the The difficulty of intercepting the search object in the image improves the information matching efficiency in the search process.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the accompanying drawings that need to be used in the descriptions of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only of the present invention. For some embodiments, those of ordinary skill in the art can also obtain other drawings based on these drawings without paying creative efforts.

Fig. 1 is the implementation flowchart of the image-based information search method provided by the embodiment of the present invention;

Fig. 2 is an example diagram of light projection points provided by an embodiment of the present invention;

Fig. 3 is a specific implementation flowchart of the image-based information search method S104 provided by the embodiment of the present invention;

FIG. 4 is a flow chart of an image-based information search method provided by another embodiment of the present invention;

Fig. 5 is a structural block diagram of an image-based information search device provided by an embodiment of the present invention.

detailed description

In the following description, specific details such as specific system structures and technologies are presented for the purpose of illustration rather than limitation, so as to thoroughly understand the embodiments of the present invention. It will be apparent, however, to one skilled in the art that the invention may be practiced in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present invention with unnecessary detail.

In the embodiment of the present invention, start the camera of the mobile terminal, and start the light emitting device; obtain the first image taken by the camera, in the first image, the light projection point of the light emitting device is located at the location of the search object The first vertex of the rectangular area; acquire the second image taken by the camera, in the second image, the light projection point of the light emitting device is located at the second vertex of the rectangular area, and the second vertex and The first vertex is a diagonal vertex of the rectangular area; a rectangular frame is constructed based on the positions of light projection points in the first image and the second image; content recognition is performed on images in the rectangular frame to determine Search for objects, and get search results.

In order to illustrate the technical solutions of the present invention, specific examples are used below to illustrate.

Fig. 1 shows the implementation process of the image-based information search method provided by the embodiment of the present invention. In the embodiment of the present invention, the image-based information search includes images captured by a camera that contain search objects. The implementation process of the embodiment corresponding to Fig. 1 is described in detail as follows:

In S101, start the camera of the mobile terminal, and start the light emitting device.

The implementation of the embodiment of the present invention first needs to take pictures of the search objects through the camera, and the search objects include but not limited to words, phrases, titles, text blocks, and the like. In the embodiment of the present invention, the camera is activated so that the user can take pictures of the search object on paper or other planes by manipulating the camera. In addition, in the embodiment of the present invention, a light emitting device will be set in the vicinity of the camera , the light emitted by the light emitting device includes but not limited to infrared light, laser light, etc. In this way, the light emitted by the light emitting device will be projected on the subject of the camera, so that the image captured by the camera will display the light The light projection point of the emitter on the paper.

In S102, acquire a first image captured by the camera, in the first image, the light projection point of the light emitting device is located at a first vertex of the rectangular area where the search object is located.

In S103, acquire a second image taken by the camera, in the second image, the light projection point of the light emitting device is located at a second vertex of the rectangular area, and the second vertex is the same as the first A vertex is a diagonal vertex of the rectangular area.

Usually, the search object in the plane is printed or displayed in the format of text block or image block, therefore, based on the area occupied by the search object in the plane, a rectangular area can be determined, in which only the Search objects. Therefore, based on the above situation, in the embodiment of the present invention, the light emitted by the light-emitting device is respectively projected on a pair of diagonal vertices of the rectangular area, and the two projections are respectively photographed, so that it can be recorded The position of the light cast point in the capture plane for these two casts.

As shown in Figure 2, topic 2 is the search object, and the light projection point of the light emitting device is the cross cursor in Figure 2, then the cursor can be projected twice to position 1 and position 2 in Figure 2, and taken separately to obtain For the pages projected twice, in fact, position 1 is the upper left vertex of the rectangular area where the text block of title 2 is located, and position 2 is the lower right vertex of the rectangular area where the text block of title 2 is located. Position 1 and position 2 constitute A pair of diagonal vertices of the rectangular area.

As an embodiment of the present invention, before the camera captures the first image and the second image, the mobile terminal prompts the projection position of the light projection point, so that the light projection point of the emitting device are respectively projected to the diagonal vertices in the rectangular area.

In this embodiment, text prompts are given on the screen of the mobile terminal for the projection position of the light emitted by the light emitting device, or voice prompts are given through the mobile terminal. In this way, the user can be more clear about the projection position of the light projection point and speed up. The acquisition speed of the first image and the second image, thereby improving the operational efficiency of the entire information search process.

In S104, construct a rectangular frame based on positions of light projection points in the first image and the second image.

In the background of the application, using the position of the light projection point in the above first image and the second image as a pair of diagonal vertices of the unknown rectangular area, a unique rectangular frame can be constructed, which only contains the image content of the search object .

As an implementation of the embodiment of the present invention, S104 can be implemented as shown in Figure 3:

S301. Mark the same position in the second image according to the position of the light projection point in the first image.

S302. The position of the light projection point and the marked position in the second image are respectively used as diagonal vertices of the rectangular frame, and the rectangular frame is constructed.

Based on the related technology of image recognition, in the embodiment of the present invention, by analyzing the pixels in the first image, a continuous pixel area with specific pixel characteristics can be found in the first image, and this area is the light projection point in the image The presented light block, and through shape matching, determine that the shape of the light block is the same as or similar to the shape of the preset cursor graphic, thereby determining that the light block is the light projection point, and completing the positioning of the light projection point, and Mark the same position in the second image, so as to complete the positioning of the light projection points generated by the two projection operations on the second image, and then use these two positioning points as a pair of diagonal vertices of the unknown rectangular area to complete Construction of rectangular boxes.

In S105, content recognition is performed on the images in the rectangular frame to determine a search object, and a search result is obtained.

Based on the image in the rectangular frame, the application background performs content recognition on the search object in it, for example, using Optical Character Recognition (Optical Character Recognition, OCR) technology to determine the shape of the search object by detecting dark and bright patterns, and then use the character Recognition methods translate shapes into computer words, resulting in search objects in text format.

Specifically, the mobile terminal uploads the search object in text format to the server. After receiving the search object, the server performs data correction and analysis, searches and matches the parsed content in the database, and obtains the search object corresponding to the search object. result, and return the search result to the mobile terminal, and the mobile terminal displays the search result, thereby completing the entire image-based information search process.

Further, as an embodiment of the present invention, as shown in FIG. 4, after performing content identification on the image in the rectangular frame to determine the search object and before obtaining the search result, the method further includes:

S401. Display the result of the content recognition, so that the user can input an instruction according to the result of the content recognition.

S402, if it is detected that the input by the user is a confirmation instruction, perform a search based on the result of the content identification.

In the embodiment shown in Figure 4, after the content recognition of the image in the rectangular frame is completed, the recognized content is displayed, so that the user can confirm the correctness of the content recognition result, and only after the user confirms and enters the confirmation instruction, is it connected with The server background communicates and uploads search objects to ensure the correctness of search results.

S403. If it is detected that the user input is a re-photographing instruction, return to the operation of starting the camera of the mobile terminal and starting the operation of the light emitting device.

And if the user input is a re-photographing command, it means that the previous light projection point positioning is incorrect, resulting in content recognition errors. At this time, return to S101 to perform the related operations shown in Figure 1 to ensure the accuracy of information search.

In the embodiment of the present invention, the search object is positioned by the light projection point generated by the light emitting device, so as to efficiently intercept the rectangular area where the search object is located, reducing the interference of non-search objects on the search results. At the same time, when the light When the projection point is a cross cursor, it can also provide horizontal and vertical reference lines for image tilt correction, thereby reducing the difficulty of intercepting the search object in the image and improving the information matching efficiency in the search process.

It should be understood that the sequence numbers of the steps in the above embodiments do not mean the order of execution, and the execution order of each process should be determined by its functions and internal logic, and should not constitute any limitation to the implementation process of the embodiment of the present invention.

Corresponding to the image-based information search method described in the above embodiments, Fig. 5 shows a structural block diagram of an image-based information search device provided by an embodiment of the present invention, and the image-based information search device can be built into a mobile terminal Or in the application system of the mobile terminal. For ease of description, only the parts related to this embodiment are shown.

Referring to Figure 5, the device includes:

The starting unit 51 starts the camera of the mobile terminal and starts the light emitting device;

The first acquisition unit 52 is configured to acquire a first image captured by the camera, in which the light projection point of the light emitting device is located at the first vertex of the rectangular area where the search object is located;

The second acquisition unit 53 is configured to acquire a second image captured by the camera, in the second image, the light projection point of the light emitting device is located at the second vertex of the rectangular area, and The first vertex is a diagonal vertex of the rectangular area;

A construction unit 54, configured to construct a rectangular frame based on the positions of light projection points in the first image and the second image;

The search unit 55 performs content recognition on the images in the rectangular frame to determine the search object, and obtains the search result.

Optionally, the device also includes:

A prompting unit, before the camera captures the first image and the second image, prompts the projection position of the light projection point through the mobile terminal, so that the light projection point of the emitting device is respectively projected to the rectangular Diagonal vertices in the region.

Optionally, the construction unit 54 includes:

The marking subunit is used to mark the same position of the second image according to the position of the light projection point in the first image;

The construction subunit uses the position of the light projection point and the marked position in the second image as diagonal vertices of the rectangular frame to construct the rectangular frame.

Optionally, the device also includes:

a display unit, displaying the result of the content recognition, so that the user can input an instruction according to the result of the content recognition;

The search unit 55 is further configured to, if it is detected that the input by the user is a confirmation instruction, perform a search based on the result of the content identification.

Optionally, the device also includes;

The return unit, if it is detected that the user input is a re-photographing instruction, returns to execute the operation of starting the camera of the mobile terminal and starting the light emitting device.

Those skilled in the art can clearly understand that for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used for illustration. In practical applications, the above-mentioned functions can be assigned to different functional units, Completion of modules means that the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment may be integrated into one processing unit, or each unit may exist separately physically, or two or more units may be integrated into one unit, and the above-mentioned integrated units may adopt hardware It can also be implemented in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the protection scope of the present application. For the specific working process of the units and modules in the above system, reference may be made to the corresponding process in the foregoing method embodiments, and details will not be repeated here.

Those skilled in the art can appreciate that the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed by hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions for each specific application, but such implementation should not be regarded as exceeding the scope of the present invention.

In the embodiments provided in the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the system embodiments described above are only illustrative. For example, the division of the modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be Incorporation may either be integrated into another system, or some features may be omitted, or not implemented. In another point, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

If the integrated unit is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiment of the present invention is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage In the medium, several instructions are included to make a computer device (which may be a personal computer, server, or network device, etc.) or a processor (processor) execute all or part of the steps of the methods described in the various embodiments of the embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disk and other media that can store program codes.

The above-described embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: it can still carry out the foregoing embodiments Modifications to the technical solutions recorded in the examples, or equivalent replacement of some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the various embodiments of the present invention, and should be included in within the protection scope of the present invention.

Claims (10)

1. An image-based information search method, characterized in that, comprising: Start the camera of the mobile terminal, and start the light emitting device; Acquiring a first image taken by the camera, in the first image, the light projection point of the light emitting device is located at the first vertex of the rectangular area where the search object is located; Acquiring a second image taken by the camera, in the second image, the light projection point of the light emitting device is located at a second vertex of the rectangular area, and the second vertex and the first vertex are the same The diagonal vertices of the rectangular area; constructing a rectangular frame based on positions of light projection points in the first image and the second image; Perform content recognition on the images in the rectangular frame to determine the search object, and obtain search results. 2. The method of claim 1, further comprising: Before the camera captures the first image and the second image, the mobile terminal prompts the projection position of the light projection point, so that the light projection point of the emitting device is respectively projected to the rectangular area. Diagonal vertices. 3. The method according to claim 1, wherein said building a rectangular frame based on the positions of the light projection points in the first image and the second image comprises: Marking at the same position in the second image according to the position of the light projection point in the first image; The position of the light projection point and the marked position in the second image are respectively used as diagonal vertices of the rectangular frame to construct the rectangular frame. 4. The method according to claim 1, characterized in that, after performing content recognition on the image in the rectangular frame to determine the search object and before obtaining the search result, the method further comprises: Displaying the result of the content recognition, so that the user can input an instruction according to the result of the content recognition; If it is detected that the input by the user is a confirmation instruction, a search is performed based on the result of the content identification. 5. The method of claim 4, further comprising; If it is detected that the user input is a re-photographing instruction, return to the operation of starting the camera of the mobile terminal and starting the light emitting device. 6. An image-based information search device, characterized in that it comprises: The starting unit is used to start the camera of the mobile terminal and start the light emitting device; A first acquiring unit, configured to acquire a first image captured by the camera, in the first image, the light projection point of the light emitting device is located at the first vertex of the rectangular area where the search object is located; The second acquiring unit is configured to acquire a second image captured by the camera, in the second image, the light projection point of the light emitting device is located at a second vertex of the rectangular area, and the second vertex is in the same position as the second vertex. The first vertex is a diagonal vertex of the rectangular area; a construction unit, configured to construct a rectangular frame based on positions of light projection points in the first image and the second image; The search unit is configured to identify the content of the images in the rectangular frame to determine the search object, and obtain the search result. 7. The device of claim 6, further comprising: A prompting unit, configured to prompt the projection position of the light projection point through the mobile terminal before the camera captures the first image and the second image, so that the light projection point of the emitting device is respectively projected to the Diagonal vertices in the rectangular area described above. 8. The device according to claim 6, wherein the construction unit comprises: A labeling subunit, configured to label the same position in the second image according to the position of the light projection point in the first image; A construction subunit is configured to use the position of the light projection point and the marked position in the second image as diagonal vertices of the rectangular frame to construct the rectangular frame. 9. The device of claim 6, further comprising: a display unit, configured to display the result of the content recognition, so that the user can input an instruction according to the result of the content recognition; The search unit is further configured to, if it is detected that the input by the user is a confirmation instruction, perform a search based on the result of the content identification. 10. The device of claim 9, further comprising; The return unit is configured to return to execute the operation of starting the camera of the mobile terminal and starting the operation of the light emitting device if it is detected that the user input is a re-photographing instruction.