CN111783775B - Image acquisition method, device, equipment and computer readable storage medium - Google Patents
Image acquisition method, device, equipment and computer readable storage medium Download PDFInfo
- Publication number
- CN111783775B CN111783775B CN202010618086.XA CN202010618086A CN111783775B CN 111783775 B CN111783775 B CN 111783775B CN 202010618086 A CN202010618086 A CN 202010618086A CN 111783775 B CN111783775 B CN 111783775B
- Authority
- CN
- China
- Prior art keywords
- image
- card
- confidence
- target image
- frame
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
- G06V10/751—Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Databases & Information Systems (AREA)
- Computing Systems (AREA)
- Artificial Intelligence (AREA)
- Health & Medical Sciences (AREA)
- Image Analysis (AREA)
Abstract
本发明公开了一种图像采集方法、装置、设备和计算机可读存储介质。该方法包括：获取目标图像；将所述目标图像输入预先训练的目标检测模型中，并获取所述目标检测模型输出的检测结果；所述检测结果包括：所述目标检测模型在所述目标图像中检测到的卡片图像的卡片边框置信度和特征边框置信度；在所述卡片边框置信度大于第一置信度阈值，并且所述特征边框置信度大于第二置信度阈值的情况下，执行针对所述目标图像的图像采集操作。本发明可以避免出现采集错误种类卡片的卡片图像的问题，降低后续图像识别的运算量和运算时长，提升图像识别的效率。
The present invention discloses an image acquisition method, device, equipment and computer-readable storage medium. The method comprises: acquiring a target image; inputting the target image into a pre-trained target detection model, and acquiring a detection result output by the target detection model; the detection result comprises: a card border confidence and a feature border confidence of a card image detected by the target detection model in the target image; when the card border confidence is greater than a first confidence threshold, and the feature border confidence is greater than a second confidence threshold, performing an image acquisition operation for the target image. The present invention can avoid the problem of acquiring card images of the wrong type of card, reduce the amount of computation and computation time of subsequent image recognition, and improve the efficiency of image recognition.
Description
Technical Field
The present invention relates to the field of computer technologies, and in particular, to an image acquisition method, apparatus, device, and computer readable storage medium.
Background
Card image recognition is involved in many applications to obtain information in a card image. For example, identifying a borrowing card number in the borrowing card image, and identifying a bank card account number in the bank card image.
The card image acquisition is the first step of card image recognition, and the acquisition quality of the card image has a larger influence on the result of card image recognition.
Currently, card image acquisition is performed at the front end and card image recognition is performed at the back end.
In the card image acquisition stage, a mode of photographing the card is generally adopted to finish the acquisition of the card image, but the existing acquisition mode lacks an auditing mechanism, and the problem of acquiring the card image of the wrong type of card is easy to occur. For example, in some scenarios it is desirable to capture images of a user's borrowing card, while the user uses other cards of the same size to complete the image capture in order to preserve personal privacy.
Although the type of the card can be identified as being wrong in the image identification stage, the background has huge number of images to be identified, and the unverified images increase the operation amount and operation duration of image identification and reduce the efficiency of image identification.
Disclosure of Invention
The invention mainly aims to provide an image acquisition method, an image acquisition device, image acquisition equipment and a computer readable storage medium, so as to solve the problem that an existing card image acquisition mode lacks an auditing mechanism.
Aiming at the technical problems, the embodiment of the invention is solved by the following technical scheme:
The embodiment of the invention provides an image acquisition method, which comprises the steps of acquiring a target image, inputting the target image into a pre-trained target detection model, and acquiring a detection result output by the target detection model, wherein the detection result comprises card frame confidence coefficient and feature frame confidence coefficient of a card image detected by the target detection model in the target image, and executing image acquisition operation for the target image under the condition that the card frame confidence coefficient is larger than a first confidence coefficient threshold value and the feature frame confidence coefficient is larger than a second confidence coefficient threshold value.
The detection result further comprises card frame opposite vertex angle coordinates of the card image, the image acquisition operation for the target image is performed, the image acquisition operation comprises the steps of determining whether the card image is in a preset area range of the target image according to the card frame opposite vertex angle coordinates of the card image, acquiring the target image if the card image is in the preset area range of the target image, otherwise, discarding the target image, and acquiring a next target image in a video stream to which the target image belongs, so that the next target image is input into the target detection model.
The method further comprises the step of displaying preset card position prompt information before the next target image is acquired in the video stream to which the target image belongs if the card image is not in the preset area range of the target image under the condition that the video stream to which the target image belongs is a video stream shot in real time, so as to prompt the position of a moving card, and the card image in the next target image is in the preset area of the next target image.
The method further comprises the steps of discarding the target image and acquiring a next target image in a video stream to which the target image belongs under the condition that the confidence of the card frame is smaller than or equal to the first confidence threshold and/or the confidence of the characteristic frame is smaller than or equal to the second confidence threshold, so that the next target image is input into the target detection model.
The method further comprises the step of displaying preset card correction prompt information before the next target image is acquired in the video stream to which the target image belongs so as to prompt the type of the replacement card, and if the card frame confidence is smaller than or equal to the first confidence threshold and/or the characteristic frame confidence is smaller than or equal to the second confidence threshold, the card frame confidence of the card image in the next target image is larger than the first confidence threshold and the characteristic frame confidence is larger than the second confidence threshold.
The method further comprises the steps of determining image acquisition time from the moment when the first target image is acquired by the video stream to the current moment when the confidence coefficient of the card frame is smaller than or equal to the first confidence coefficient threshold value and/or the confidence coefficient of the characteristic frame is smaller than or equal to the second confidence coefficient threshold value, and updating the first confidence coefficient threshold value and the second confidence coefficient threshold value when the image acquisition time is longer than a preset time threshold value, wherein the updated first confidence coefficient threshold value is smaller than the first confidence coefficient threshold value before updating, and the updated second confidence coefficient threshold value is smaller than the second confidence coefficient threshold value before updating.
The method comprises the steps that if a first confidence coefficient threshold value, which is larger than the card frame confidence coefficient of a card image in a target image, is an updated first confidence coefficient threshold value and a second confidence coefficient threshold value, which is larger than the characteristic frame confidence coefficient of the card image in the target image, is an updated second confidence coefficient threshold value before the target image is acquired, a corresponding confidence coefficient threshold degradation identifier is set for the target image when the target image is acquired.
The number of parameters of the target detection model is smaller than a preset parameter number threshold, and/or the number FLOPs of floating point operations executed by the target detection model is smaller than a preset floating point operation number threshold, and/or the frame rate of the target detection model is larger than a preset frame rate threshold.
The embodiment of the invention also provides an image acquisition device which comprises an acquisition module, a detection module and an execution module, wherein the acquisition module is used for acquiring a target image, the detection module is used for inputting the target image into a pre-trained target detection model and acquiring a detection result output by the target detection model, the detection result comprises card frame confidence coefficient and characteristic frame confidence coefficient of a card image detected by the target detection model in the target image, and the execution module is used for executing image acquisition operation for the target image under the condition that the card frame confidence coefficient is larger than a first confidence coefficient threshold value and the characteristic frame confidence coefficient is larger than a second confidence coefficient threshold value.
The embodiment of the invention also provides image acquisition equipment which comprises a processor and a memory, wherein the processor is used for executing an image acquisition program stored in the memory so as to realize the image acquisition method of any one of the above.
Embodiments of the present invention also provide a computer-readable storage medium storing one or more programs executable by one or more processors to implement the image acquisition method described in any one of the above.
The embodiment of the invention has the following beneficial effects:
According to the embodiment of the invention, the card image in the target image can be audited, the card frame confidence coefficient and the characteristic frame confidence coefficient of the card image are determined, whether the image acquisition operation is executed is determined according to the card frame confidence coefficient and the characteristic frame confidence coefficient, if the card frame confidence coefficient is larger than the first confidence coefficient threshold value and the characteristic frame confidence coefficient is larger than the second confidence coefficient threshold value, the image acquisition operation is executed, the problem of acquiring the card image of the wrong type card is avoided, the operation amount and the operation duration of the subsequent image identification are reduced, and the image identification efficiency is improved.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description serve to explain the application and do not constitute a limitation on the application. In the drawings:
FIG. 1 is a flow chart of an image acquisition method according to an embodiment of the invention;
FIG. 2 is a schematic diagram of sample labeling according to an embodiment of the invention;
FIG. 3 is a flowchart of steps of an image acquisition operation according to an embodiment of the present invention;
Fig. 4 is a block diagram of an image pickup apparatus according to an embodiment of the present invention;
fig. 5 is a block diagram of an image capturing apparatus according to an embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the drawings and the embodiments, in order to make the objects, technical solutions and advantages of the present invention more apparent.
According to an embodiment of the present invention, there is provided an image acquisition method. Fig. 1 is a flowchart of an image acquisition method according to an embodiment of the invention.
Step S110, a target image is acquired.
The target image is the target image to be audited. Further, a card image of the target card may be included in the target image.
Specifically, in the received video stream, one of the video images is acquired, and the video image is taken as a target image. The video stream is a video stream photographed in real time or a video stream photographed in advance.
For example, a camera can be called to shoot a video stream in real time, the video stream returned by the camera is received, and a first frame video image of the video stream is obtained as a target image.
And step S120, inputting the target image into a pre-trained target detection model, and obtaining a detection result output by the target detection model, wherein the detection result comprises card frame confidence coefficient and feature frame confidence coefficient of a card image detected by the target detection model in the target image.
The target detection model is used for detecting whether a card image of a suspected target type card is contained in the target image, if the card image of the suspected target type card is contained in the target image, extracting a card frame and a feature frame of the suspected target type image, and determining a card frame confidence coefficient corresponding to the card frame and a feature frame confidence coefficient corresponding to the feature frame.
The target type card refers to the type of card to be checked. Such as a borrowing card, a banking card, etc.
The confidence of the card frame refers to the credibility of the image area surrounded by the card frame as the target type card.
The confidence of the feature frame refers to the credibility of the image area surrounded by the feature frame as the target type. The characteristic frame is the frame of the region where the characteristic information in the target type card is located.
The number of feature information in the target class card may be one or more. For example, the characteristic information in the borrowing card comprises a borrowing card number and a borrowing card photo. As another example, the characteristic information in the bank card includes a bank card number. The target detection model can be used for detecting the confidence level of the characteristic frame corresponding to the frame of the area where each characteristic information is located.
Step S130, executing an image acquisition operation for the target image when the card frame confidence is greater than a first confidence threshold and the feature frame confidence is greater than a second confidence threshold.
And the first confidence threshold is used for measuring whether the card frame of the card image is the card frame of the target type card. If the confidence coefficient of the card frame of the card image is larger than the first confidence coefficient threshold value, the card frame of the card image is the card frame of the target type card, otherwise, the card frame of the card image is not the card frame of the target type card.
And the second confidence threshold is used for measuring whether the characteristic frame of the card image is the characteristic frame of the target type card. If the confidence coefficient of the characteristic frame of the card image is larger than the second confidence coefficient threshold, the characteristic frame of the card image is the characteristic frame of the target type card, otherwise, the characteristic frame of the card image is not the characteristic frame of the target type card.
In this embodiment, since the card frame is easier to extract, and the feature frame is more difficult to extract, the first confidence threshold is greater than the second confidence threshold. For example, the first confidence threshold is set to 0.8, the second confidence threshold is set to 0.6, the card frame confidence is 0.9 (0.9 > 0.8), the feature frame confidence is 0.7 (0.7 > 0.6), the image acquisition operation is performed, the card frame confidence is 0.85 (0.85 > 0.8), the feature frame confidence is 0.5 (0.5 < 0.6), and the image acquisition operation is not performed.
The target image may be directly acquired when an image acquisition operation for the target image is performed, or may be acquired when the target image meets a preset image acquisition condition. The image acquisition condition is, for example, that the card image in the target image is in the central region of the target image.
After the target image is acquired, the target image is transmitted to the back end so that the back end performs image recognition processing on the target image.
According to the embodiment, the card image in the target image can be audited, the card frame confidence coefficient and the characteristic frame confidence coefficient of the card image are determined, whether the image acquisition operation is executed is determined according to the card frame confidence coefficient and the characteristic frame confidence coefficient, if the card frame confidence coefficient is larger than a first confidence coefficient threshold value and the characteristic frame confidence coefficient is larger than a second confidence coefficient threshold value, the image acquisition operation is executed, the problem of acquiring the card image of the wrong type card is avoided, the operation amount and operation duration of subsequent image recognition are reduced, and the image recognition efficiency is improved.
The training process of the object detection model is described below.
In this embodiment, the target detection model is a lightweight model, so that the image acquisition method of this embodiment can be integrated into the front end, and occupies a smaller storage space, so that the front end can operate smoothly.
Further, the number of parameters of the target detection model is smaller than a preset parameter number threshold, and/or the number of Floating Point Operations (FLOPs) performed by the target detection model is smaller than a preset Floating Point operation number threshold, and/or the frame rate (FRAME RATE) of the target detection model is larger than a preset frame rate threshold. The parameter quantity threshold value, the floating point operation number threshold value and the frame rate threshold value are all empirical values.
Further, the object detection model may optionally be constructed using SSD (Single Shot MultiBox Detector) algorithm or YOLO (You Only Look Once) algorithm. The SSD algorithm is, for example, mobilenet-SSD algorithm.
In the sample labeling, the card frame of the target type card can be labeled as a card, and the feature frame of the target type card can be labeled as a number. When the card frame is marked, the area where the card frame can contain the whole target type card is required to be ensured, and when the characteristic frame is marked, the area where the characteristic frame can contain the whole characteristic information is required to be ensured. For example, fig. 2 is a schematic diagram of sample labeling, where a bank card is a target card, a card frame of the bank card is a card frame of the bank card, and a number is a frame of an area where a bank card number is located according to an embodiment of the present invention.
The target detection model is trained by using the marked plurality of samples until the target detection model converges.
The card frame and the characteristic frame of the card image of the target type card are detected by using the trained target detection model, the accuracy is high, the generalization capability is high, the card face pattern of the target type card is designed to be complex even if the design sense of the card is improved, the target type card is placed in a complex background environment when the target type card is shot, the target detection model can accurately detect the card frame and the characteristic frame of the card image, and the card frame confidence and the characteristic frame confidence are determined.
Further, since the sample used when training the target detection model is an image of a real target class card, the target detection model has a characteristic of discriminating the authenticity of the target class card. For example, in some scenes, a user's bank card image needs to be acquired, but in order to be beneficial, an illegal user steals a photocopy of another bank card to acquire the bank card image, but the photocopy cannot have larger differences with a real bank card image in terms of color, brightness or texture, so that the difference between the photocopy and the real bank card image is easily detected by a target detection model, and further, the confidence of a card frame and the confidence of a characteristic frame output by the target detection model are smaller, so that the image acquisition operation is not executed on the photocopy.
Another case of the embodiment of the present invention will be described below.
If there is only one target image, the image acquisition process of the embodiment may be ended and image acquisition failure prompt information may be displayed when the confidence level of the card frame is less than or equal to the first confidence level threshold and/or the confidence level of the feature frame is less than or equal to the second confidence level threshold.
If the target image is a video image obtained from a video stream, discarding the target image and obtaining a next target image in the video stream to which the target image belongs in order to input the next target image into the target detection model when the card frame confidence is smaller than or equal to the first confidence threshold and/or the feature frame confidence is smaller than or equal to the second confidence threshold. That is, the next video image is acquired from the video stream, and the newly acquired video image is taken as a new target image. If no target image capable of executing image acquisition operation exists, the image acquisition flow of the embodiment of the invention can be circularly executed on a plurality of video images in the video stream, so that the target image meeting the confidence requirement can be dynamically found in the video stream.
Further, if the video stream to which the target image belongs is a video stream photographed in real time, and/or if the card frame confidence is less than or equal to the first confidence threshold and/or the feature frame confidence is less than or equal to the second confidence threshold, a preset card correction prompt message is displayed before the next target image is acquired in the video stream to which the target image belongs, so as to prompt the type of the replacement card, so that the card frame confidence of the card image in the next target image is greater than the first confidence threshold and the feature frame confidence is greater than the second confidence threshold. If the card frame confidence and the feature frame confidence of the card image are both lower than the corresponding threshold values, there is a possibility that an error occurs in the type of the photographed card, for example, a card borrowing image needs to be collected, if the user photographs a membership card image, the sizes of the card frames are different, and the frame (feature frame) of the area where the card borrowing number is located cannot be extracted from the membership card image.
In one embodiment, to avoid the problem of unsuccessful acquisition for a long period of time, a first confidence threshold and a second confidence threshold may be set to a plurality of gradients. Each gradient corresponds to a group of first confidence threshold value and second confidence threshold value, the first confidence threshold value among the gradients is set to be different values, the second confidence threshold value among the gradients is set to be different values, the change trend of the first confidence threshold value and the second confidence threshold value among the gradients is the same, the first confidence threshold value is ranked from large to small, and the second confidence threshold value is ranked from large to small from the 1 st gradient to the N th gradient. In order to collect a clearer target image, initial values of the first confidence coefficient threshold value and the second confidence coefficient threshold value can be set to be maximum values in a plurality of gradients, and degradation processing can be performed on the first confidence coefficient threshold value and the second confidence coefficient threshold value in order to avoid the problem that the target image of the target type card cannot be collected normally.
The method comprises the steps of determining image acquisition time from the moment when a first target image is acquired by a video stream to the current moment when the confidence of a card frame is smaller than or equal to a first confidence threshold value and/or the confidence of a characteristic frame is smaller than or equal to a second confidence threshold value, and updating the first confidence threshold value and the second confidence threshold value when the image acquisition time is longer than a preset time threshold value, wherein the updated first confidence threshold value is smaller than the first confidence threshold value before updating, and the updated second confidence threshold value is smaller than the second confidence threshold value before updating.
The image acquisition duration may be an empirical value or a value obtained through experimentation. Further, a plurality of time length thresholds can be set, when the time length of the image acquisition is greater than the minimum time length threshold, the first confidence coefficient threshold and the second confidence coefficient threshold are updated to be the first confidence coefficient threshold and the second confidence coefficient threshold in the 2 nd gradient according to the sequence from the large value to the small value, when the time length of the image acquisition is greater than the next minimum time length threshold, the first confidence coefficient threshold and the second confidence coefficient threshold are updated to be the first confidence coefficient threshold and the second confidence coefficient threshold in the 3 rd gradient according to the sequence from the large value to the small value, and so on, if the first confidence coefficient threshold and the second confidence coefficient threshold in all gradients are used up, the image acquisition process of the embodiment of the invention is ended and the image acquisition failure prompt information is displayed.
If the updated first confidence threshold value and the updated second confidence threshold value are respectively compared with the card frame confidence value and the feature frame confidence value, a corresponding confidence threshold degradation mark is required to be set for the target image, so that the confidence threshold value used by the target image is represented by the confidence threshold degradation mark as not being the largest confidence threshold value. In other words, if, before the target image is acquired, a first confidence threshold with greater card frame confidence of a card image in the target image is an updated first confidence threshold, and a second confidence threshold with greater card frame confidence of a card image in the target image is an updated second confidence threshold, then a corresponding confidence threshold degradation identifier is set for the target image when the target image is acquired.
In another embodiment, to further increase the efficiency of image recognition, the integrity of the card may also be determined. The determination of the integrity of the card may be achieved by determining the location of the area in the target image where the card image is located.
Further, the target detection model is also used for determining the coordinates of the opposite vertex angles of the card frame of the card image. Thus, the detection result of the target detection model also comprises the coordinates of the opposite vertex angles of the card frame of the card image. The card frame opposite vertex angle coordinates comprise vertex coordinates of two opposite corners of the card image. For example, the top left corner of the card image and the bottom right corner.
As shown in fig. 3, a flowchart of steps of an image acquisition operation according to an embodiment of the present invention is shown.
Step S310, determining whether the card image is in the preset area range of the target image according to the card frame opposite vertex angle coordinates of the card image, if so, executing step S320, and if not, executing step S330.
The preset region range may be a central region of the target image, and the central region does not overlap with an edge of the target image.
For example, the vertex coordinates of the upper left corner (x 1,y1) and the vertex coordinates of the lower right corner (x 2,y2) of the card image. The width w and the height h of the target image. The boundary threshold t is the closest distance from the edge of the card image to the edge of the target image, which may demarcate the centered area of the target image.
When the vertex coordinates (x 1,y1) of the upper left corner and the vertex coordinates (x 2,y2) of the lower right corner of the card image meet the following conditions, the card image is indicated to be in the central area of the target image, and the card image is complete and can be acquired:
the upper left corner of the target image is taken as the origin of the coordinate system, the direction of the vertical axis is downward along the edge passing through the origin in the target image, and the direction of the horizontal axis is rightward along the edge passing through the origin in the target image.
Step S320, if the card image is within the preset area range of the target image, acquiring the target image.
Step S330, if the card image is not in the preset area range of the target image, discarding the target image, and acquiring a next target image in the video stream to which the target image belongs, so as to input the next target image into the target detection model.
And if the video stream to which the target image belongs is a video stream shot in real time, displaying preset card position prompt information before acquiring the next target image in the video stream to which the target image belongs, so as to prompt the position of a moving card, and enabling the card image in the next target image to be in the preset area of the next target image.
If the card image is not in the preset area range of the target image, the card image may not be complete enough, and then the user adjusts the position of the card through the card position prompt information, so that the card is in the preset area range when appearing in the target image again.
The embodiment of the invention also provides an image acquisition device. Fig. 4 is a block diagram of an image capturing device according to an embodiment of the present invention.
The image acquisition device comprises an acquisition module 410, a detection module 420 and an execution module 430.
An acquisition module 410 is configured to acquire a target image.
The detection module 420 is configured to input the target image into a pre-trained target detection model, and obtain a detection result output by the target detection model, where the detection result includes a card frame confidence coefficient and a feature frame confidence coefficient of a card image detected by the target detection model in the target image.
An execution module 430 is configured to execute an image acquisition operation for the target image if the card border confidence is greater than a first confidence threshold and the feature border confidence is greater than a second confidence threshold.
The functions of the apparatus according to the embodiments of the present invention have been described in the foregoing method embodiments, so that the descriptions of the embodiments are not exhaustive, and reference may be made to the related descriptions in the foregoing embodiments, which are not repeated herein.
The embodiment also provides image acquisition equipment. As shown in fig. 5, a block diagram of an image capturing apparatus according to an embodiment of the present invention.
In this embodiment, the image acquisition device includes, but is not limited to, a processor 510, a memory 520.
The processor 510 is configured to execute an image acquisition program stored in the memory 520 to implement the image acquisition method described above.
Specifically, the processor 510 is configured to execute an image acquisition program stored in the memory 520, so as to obtain a target image, input the target image into a pre-trained target detection model, and obtain a detection result output by the target detection model, where the detection result includes a card frame confidence level and a feature frame confidence level of a card image detected by the target detection model in the target image, and execute an image acquisition operation for the target image when the card frame confidence level is greater than a first confidence level threshold and the feature frame confidence level is greater than a second confidence level threshold.
The detection result further comprises card frame opposite vertex angle coordinates of the card image, the image acquisition operation for the target image is performed, the image acquisition operation comprises the steps of determining whether the card image is in a preset area range of the target image according to the card frame opposite vertex angle coordinates of the card image, acquiring the target image if the card image is in the preset area range of the target image, otherwise, discarding the target image, and acquiring a next target image in a video stream to which the target image belongs, so that the next target image is input into the target detection model.
The method further comprises the step of displaying preset card position prompt information before the next target image is acquired in the video stream to which the target image belongs if the card image is not in the preset area range of the target image under the condition that the video stream to which the target image belongs is a video stream shot in real time, so as to prompt the position of a moving card, and the card image in the next target image is in the preset area of the next target image.
The method further comprises the steps of discarding the target image and acquiring a next target image in a video stream to which the target image belongs under the condition that the confidence of the card frame is smaller than or equal to the first confidence threshold and/or the confidence of the characteristic frame is smaller than or equal to the second confidence threshold, so that the next target image is input into the target detection model.
The method further comprises the step of displaying preset card correction prompt information before the next target image is acquired in the video stream to which the target image belongs so as to prompt the type of the replacement card, and if the card frame confidence is smaller than or equal to the first confidence threshold and/or the characteristic frame confidence is smaller than or equal to the second confidence threshold, the card frame confidence of the card image in the next target image is larger than the first confidence threshold and the characteristic frame confidence is larger than the second confidence threshold.
The method further comprises the steps of determining image acquisition time from the moment when the first target image is acquired by the video stream to the current moment when the confidence coefficient of the card frame is smaller than or equal to the first confidence coefficient threshold value and/or the confidence coefficient of the characteristic frame is smaller than or equal to the second confidence coefficient threshold value, and updating the first confidence coefficient threshold value and the second confidence coefficient threshold value when the image acquisition time is longer than a preset time threshold value, wherein the updated first confidence coefficient threshold value is smaller than the first confidence coefficient threshold value before updating, and the updated second confidence coefficient threshold value is smaller than the second confidence coefficient threshold value before updating.
The method comprises the steps that if a first confidence coefficient threshold value, which is larger than the card frame confidence coefficient of a card image in a target image, is an updated first confidence coefficient threshold value and a second confidence coefficient threshold value, which is larger than the characteristic frame confidence coefficient of the card image in the target image, is an updated second confidence coefficient threshold value before the target image is acquired, a corresponding confidence coefficient threshold degradation identifier is set for the target image when the target image is acquired.
The number of parameters of the target detection model is smaller than a preset parameter number threshold, and/or the number FLOPs of floating point operations executed by the target detection model is smaller than a preset floating point operation number threshold, and/or the frame rate of the target detection model is larger than a preset frame rate threshold.
The embodiment of the invention also provides a computer readable storage medium. The computer-readable storage medium here stores one or more programs. The computer readable storage medium may include volatile memory, such as random access memory, or nonvolatile memory, such as read only memory, flash memory, hard disk, or solid state disk, or a combination of the foregoing.
The one or more programs in the computer-readable storage medium may be executed by the one or more processors to implement the image acquisition methods described above. Since the image acquisition method has been described in detail above, a detailed description thereof is omitted here.
The above description is only an example of the present invention and is not intended to limit the present invention, but various modifications and variations can be made to the present invention by those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the scope of the claims of the present invention.
Claims (9)
1. An image acquisition method, comprising:
Receiving a video stream shot by image acquisition equipment, and determining any frame of video image in the video stream as a target image;
Inputting the target image into a pre-trained target detection model, and acquiring a detection result output by the target detection model, wherein the detection result comprises card frame confidence coefficient and characteristic frame confidence coefficient of a card image detected by the target detection model in the target image, wherein the characteristic frame confidence coefficient is used for representing the credibility of a target type of an image area surrounded by a characteristic frame, and the characteristic frame refers to the frame of an area where one or more pieces of characteristic information in a target type card are located;
Executing image acquisition operation aiming at the target image under the condition that the confidence coefficient of the card frame is larger than a first confidence coefficient threshold value and the confidence coefficient of the characteristic frame is larger than a second confidence coefficient threshold value;
Discarding the target image and acquiring a next target image in a video stream to which the target image belongs so as to input the next target image into the target detection model when the card frame confidence is smaller than or equal to the first confidence threshold and/or the feature frame confidence is smaller than or equal to the second confidence threshold;
determining the image acquisition time length from the moment when the first target image is acquired by the video stream to the current moment;
And updating the first confidence coefficient threshold value and the second confidence coefficient threshold value when the image acquisition time length is larger than a preset time length threshold value, wherein the updated first confidence coefficient threshold value is smaller than the first confidence coefficient threshold value before updating, and the updated second confidence coefficient threshold value is smaller than the second confidence coefficient threshold value before updating.
2. The method of claim 1, wherein the step of determining the position of the substrate comprises,
The detection result also comprises the coordinates of the opposite vertex angles of the card frame of the card image;
the performing an image acquisition operation for the target image includes:
determining whether the card image is in a preset area range of the target image according to the card frame opposite vertex angle coordinates of the card image;
And if the card image is in the preset area range of the target image, acquiring the target image, otherwise, discarding the target image, and acquiring a next target image in the video stream to which the target image belongs so as to input the next target image into the target detection model.
3. The method according to claim 2, wherein the method further comprises:
And if the video stream to which the target image belongs is a video stream shot in real time, displaying preset card position prompt information before acquiring the next target image in the video stream to which the target image belongs, so as to prompt the position of a moving card, and enabling the card image in the next target image to be in the preset area of the next target image.
4. The method according to claim 1, wherein the method further comprises:
And under the condition that the video stream to which the target image belongs is a video stream shot in real time, if the card frame confidence is smaller than or equal to the first confidence threshold and/or the feature frame confidence is smaller than or equal to the second confidence threshold, displaying preset card correction prompt information before the next target image is acquired in the video stream to which the target image belongs, so as to prompt the type of the replacement card, wherein the card frame confidence of the card image in the next target image is larger than the first confidence threshold and the feature frame confidence is larger than the second confidence threshold.
5. The method of claim 1, wherein the acquiring the target image comprises:
If a first confidence threshold value with the greater confidence coefficient of the card frame of the card image in the target image is an updated first confidence coefficient threshold value and a second confidence coefficient threshold value with the greater confidence coefficient of the characteristic frame of the card image in the target image is an updated second confidence coefficient threshold value before the target image is acquired, setting a corresponding confidence coefficient threshold degradation identifier for the target image when the target image is acquired.
6. The method according to any one of claim 1 to 5, wherein,
The number of parameters of the object detection model is less than a preset parameter number threshold, and/or,
The number FLOPs of floating point operations performed by the object detection model is less than a predetermined floating point operation number threshold, and/or,
The frame rate of the target detection model is greater than a preset frame rate threshold.
7. An image acquisition device, comprising:
The acquisition module is used for acquiring a target image and comprises the steps of receiving a video stream shot by image acquisition equipment and determining any frame of video image as the target image;
The detection module is used for inputting the target image into a pre-trained target detection model and obtaining a detection result output by the target detection model, wherein the detection result comprises card frame confidence coefficient and characteristic frame confidence coefficient of a card image detected by the target detection model in the target image, the characteristic frame confidence coefficient is used for representing the credibility degree of a target type of an image area surrounded by a characteristic frame, and the characteristic frame refers to the frame of an area where one or more pieces of characteristic information in a target type of card are located;
the execution module is used for executing image acquisition operation aiming at the target image under the condition that the confidence coefficient of the card frame is larger than a first confidence coefficient threshold value and the confidence coefficient of the characteristic frame is larger than a second confidence coefficient threshold value;
Discarding the target image and acquiring a next target image in a video stream to which the target image belongs so as to input the next target image into the target detection model when the card frame confidence is smaller than or equal to the first confidence threshold and/or the feature frame confidence is smaller than or equal to the second confidence threshold;
determining the image acquisition time length from the moment when the first target image is acquired by the video stream to the current moment;
And updating the first confidence coefficient threshold value and the second confidence coefficient threshold value when the image acquisition time length is larger than a preset time length threshold value, wherein the updated first confidence coefficient threshold value is smaller than the first confidence coefficient threshold value before updating, and the updated second confidence coefficient threshold value is smaller than the second confidence coefficient threshold value before updating.
8. An image acquisition device, characterized in that the image acquisition device comprises a processor and a memory, wherein the processor is used for executing an image acquisition program stored in the memory to realize the image acquisition method according to any one of claims 1-6.
9. A computer-readable storage medium, characterized in that the computer-readable storage medium stores one or more programs, the one or more programs are executable by one or more processors to implement the image acquisition method of any one of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010618086.XA CN111783775B (en) | 2020-06-30 | 2020-06-30 | Image acquisition method, device, equipment and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010618086.XA CN111783775B (en) | 2020-06-30 | 2020-06-30 | Image acquisition method, device, equipment and computer readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111783775A CN111783775A (en) | 2020-10-16 |
| CN111783775B true CN111783775B (en) | 2025-02-21 |
Family
ID=72760500
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010618086.XA Active CN111783775B (en) | 2020-06-30 | 2020-06-30 | Image acquisition method, device, equipment and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111783775B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114299037B (en) * | 2021-12-30 | 2023-09-01 | 广州极飞科技股份有限公司 | Quality evaluation method and device for object detection result, electronic equipment and computer readable storage medium |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111027450A (en) * | 2019-12-04 | 2020-04-17 | 深圳市新国都金服技术有限公司 | Bank card information identification method and device, computer equipment and storage medium |
| CN111126394A (en) * | 2019-12-25 | 2020-05-08 | 上海肇观电子科技有限公司 | Character recognition method, reading aid, circuit and medium |
| CN111199240A (en) * | 2018-11-16 | 2020-05-26 | 马上消费金融股份有限公司 | Training method of bank card identification model, and bank card identification method and device |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150006361A1 (en) * | 2013-06-28 | 2015-01-01 | Google Inc. | Extracting Card Data Using Three-Dimensional Models |
| US9430766B1 (en) * | 2014-12-09 | 2016-08-30 | A9.Com, Inc. | Gift card recognition using a camera |
| KR101824600B1 (en) * | 2015-09-08 | 2018-03-14 | 에스케이플래닛 주식회사 | A system and method credit card number and expiration date recognition using a terminal device |
| CN105095919A (en) * | 2015-09-08 | 2015-11-25 | 北京百度网讯科技有限公司 | Image recognition method and image recognition device |
| CN110427909B (en) * | 2019-08-09 | 2023-04-28 | 连连银加信息技术有限公司 | Mobile terminal driving license detection method and system, electronic equipment and storage medium |
| CN110378328B (en) * | 2019-09-16 | 2019-12-13 | 图谱未来(南京)人工智能研究院有限公司 | certificate image processing method and device |
-
2020
- 2020-06-30 CN CN202010618086.XA patent/CN111783775B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111199240A (en) * | 2018-11-16 | 2020-05-26 | 马上消费金融股份有限公司 | Training method of bank card identification model, and bank card identification method and device |
| CN111027450A (en) * | 2019-12-04 | 2020-04-17 | 深圳市新国都金服技术有限公司 | Bank card information identification method and device, computer equipment and storage medium |
| CN111126394A (en) * | 2019-12-25 | 2020-05-08 | 上海肇观电子科技有限公司 | Character recognition method, reading aid, circuit and medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111783775A (en) | 2020-10-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110189285B (en) | Method and device for fusion of multi-frame images | |
| US20200160040A1 (en) | Three-dimensional living-body face detection method, face authentication recognition method, and apparatuses | |
| JP4772839B2 (en) | Image identification method and imaging apparatus | |
| KR101333871B1 (en) | Method and arrangement for multi-camera calibration | |
| CN112287868B (en) | Human body action recognition method and device | |
| CN103916603B (en) | Backlighting detecting and equipment | |
| CN112287866A (en) | Human body action recognition method and device based on human body key points | |
| CN112287867B (en) | Multi-camera human body action recognition method and device | |
| CN102999750B (en) | A kind of fingerprint on site Enhancement Method removing background interference | |
| WO2021195873A1 (en) | Method and device for identifying region of interest in sfr test chart image, and medium | |
| WO2023098045A1 (en) | Image alignment method and apparatus, and computer device and storage medium | |
| JP2019021297A (en) | Image processing device and method, and electronic apparatus | |
| CN112634298A (en) | Image processing method and device, storage medium and terminal | |
| CN109753886B (en) | Face image evaluation method, device and equipment | |
| CN111783775B (en) | Image acquisition method, device, equipment and computer readable storage medium | |
| CN115205113A (en) | Image splicing method, device, equipment and storage medium | |
| CN113822927A (en) | Face detection method, device, medium and equipment suitable for weak-quality images | |
| TWI620147B (en) | Image synthesis method for synthesizing people | |
| CN115424181A (en) | Target object detection method and device | |
| CN114332981A (en) | Face living body detection method and device, electronic equipment and storage medium | |
| CN114298952A (en) | Label image generation method, device, equipment and storage medium | |
| CN115004245A (en) | Target detection method, target detection device, electronic equipment and computer storage medium | |
| JP2009098867A (en) | Character string recognition method, computer program, and storage medium | |
| CN113705285B (en) | Principal identification method, apparatus, and computer-readable storage medium | |
| WO2025189466A1 (en) | Detection method and apparatus, and computer device and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: Room 221, 2 / F, block C, 18 Kechuang 11th Street, Daxing District, Beijing, 100176 Applicant after: Jingdong Technology Holding Co.,Ltd. Address before: Room 221, 2 / F, block C, 18 Kechuang 11th Street, Daxing District, Beijing, 100176 Applicant before: Jingdong Digital Technology Holding Co.,Ltd. Address after: Room 221, 2 / F, block C, 18 Kechuang 11th Street, Daxing District, Beijing, 100176 Applicant after: Jingdong Digital Technology Holding Co.,Ltd. Address before: Room 221, 2 / F, block C, 18 Kechuang 11th Street, Daxing District, Beijing, 100176 Applicant before: JINGDONG DIGITAL TECHNOLOGY HOLDINGS Co.,Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |