CN115809814A - Image processing method - Google Patents

Abstract

The invention provides an image processing method. In this method, a plurality of image quality evaluation items are received, and the total running time of the image quality evaluation items executed on the cloud server is calculated, and the individual running times of the image quality evaluation items executed on the image processing device are respectively calculated. The image quality evaluation items are divided into a first type of image quality evaluation item and a second type of image quality evaluation item according to the individual running time and the total running time. The image processing device executes the first type of image quality evaluation items, and the cloud server executes the second type of image quality evaluation items. Therefore, the image quality evaluation items are classified by an image processing method, and the processing efficiency of image identification is further improved.

Description

image processing method

technical field

The present invention relates to the field of image processing, in particular to an image processing method for image quality assessment (Image Quality Assessment).

Background technique

In general image recognition, the image to be tested is first subjected to Image Quality Assessment (IQA) to filter out images whose image quality is lower than the threshold. The current method is to upload the image to be tested to the cloud and make the cloud server execute a plurality of selected image quality evaluation items.

The user equipment and the current network environment will affect the required transmission time for the image processing device (local end) to upload/download images and recognition results from the cloud server. If the current network environment is not good, it will take too long for the image processing device to upload the image and receive the image recognition result, which will reduce and affect the efficiency of image processing. Moreover, if the image to be tested fails the image quality evaluation item executed by the cloud server, and the user still has to wait for the time from uploading the image to receiving the image quality failure. In this way, the user must retake the image or select a new image, and sometimes the efficiency of using the cloud server to perform image quality evaluation item recognition is lower than that of the user judging the image quality by naked eyes. Accordingly, it is inconvenient to use.

Contents of the invention

The present invention proposes an image processing method, wherein the image processing method can divide the image quality evaluation items into the first type of image quality evaluation items and The second category of image quality evaluation items is to dynamically adjust the device for executing the image quality evaluation items according to the user equipment and network environment, so that it is closer to the actual conditions of the user's equipment device and network environment.

The image processing method of the present invention is applicable to an image processing device. The image processing device has a processor and a storage unit, and is communicatively connected to a cloud server. The image processing method executes the following steps according to the instructions of the image processing device: receiving image quality evaluation items. Calculate the total running time of the image quality evaluation item executed on the cloud server, and calculate the individual running time of the image quality evaluation item executed on the image processing device. According to the individual running time and the total running time, the image quality evaluation items are divided into the first type of image quality evaluation items and the second type of image quality evaluation items. The first type of image quality evaluation item is executed by the image processing device, and the second type of image quality evaluation item is executed by the cloud server.

Based on the above, according to the image processing method of the embodiment of the present invention, according to the device status of the image processing device and the network status of the user, the execution device of the image quality evaluation item is adaptively and dynamically adjusted. In this way, when the user's equipment and network environment conditions make the required time too long, part or all of the image quality evaluation items can be assigned to the local image processing device or processor for image quality evaluation during image processing.

Description of drawings

The accompanying drawings are included to provide a further understanding of the invention, and are incorporated in and constitute a part of this specification. The drawings illustrate embodiments of the invention and, together with the description, serve to explain principles of the invention.

1 is a block diagram of an image processing device according to an embodiment of the present invention;

2 is a flowchart of an image processing method according to an embodiment of the present invention;

3 is a schematic flowchart of an image processing method according to an embodiment of the present invention;

Fig. 4 is a flowchart of image recognition according to an embodiment of the present invention;

5A is a schematic diagram of an image processing method;

5B is a schematic diagram of an image processing method according to an embodiment of the present invention

Explanation of reference numbers

100: image processing device;

110: processor;

120: storage unit;

130: display screen;

2: Image recognition model;

RL: image recognition result;

IQA_CR: distribution result;

L: local terminal;

C: cloud server;

Cloud_Time: total running time;

IQA A_Time: the running time of image quality evaluation item A;

IQA B_Time: the running time of image quality assessment item B;

IQA C_Time: the running time of image quality evaluation item C;

IQA D_Time: the running time of image quality evaluation item D;

IQA_G1: The first category of image quality evaluation items;

IQA_G2: The second category of image quality evaluation items;

IQA: image quality assessment project;

IQA A: image quality evaluation item A;

IQA B: image quality evaluation item B;

IQA C: image quality evaluation item C;

IQA D: image quality evaluation item D;

Img_m: image;

S210, S220, S230, S240, S211: steps.

Detailed ways

Reference will now be made in detail to the exemplary embodiments of the present invention, examples of which are illustrated in the accompanying drawings. Wherever possible, the same reference numbers will be used in the drawings and description to refer to the same or like parts.

FIG. 1 is a block diagram of an image processing device according to an embodiment of the present invention, but it is only for convenience of description and not intended to limit the present invention. Referring to FIG. 1 , the image processing method is applicable to an image processing device. The image processing device 100 may include a storage unit 120 , a processor 110 and a display screen 130 . In some embodiments, the image processing device 100 may be implemented as a notebook computer, a desktop computer, a tablet computer, an industrial computer, a server or other types of computing devices or image processing devices, and the present invention is not limited thereto.

The storage unit 120 is used for storing images, instructions, program codes, software modules and other data. The memory unit may include volatile memory circuits and non-volatile memory circuits. The volatile storage circuit is used to store data volatilely. For example, the volatile storage circuit may include random access memory (Random Access Memory, RAM) or similar volatile storage media. The non-volatile storage circuit is used for non-volatile storage of data. For example, the nonvolatile storage circuit may include a read only memory (Read Only Memory, ROM), a solid state disk (solid state disk, SSD) and/or a traditional hard disk (Hard diskdrive, HDD) or similar nonvolatile storage media .

The display screen 130 can be realized by a liquid crystal display (Liquid Crystal Display, LCD), a plasma display (Plasma Display), or a touch screen with a touch module can also be used as the display screen 130 .

The processor 110 is coupled to the storage unit 120 to control the whole or part of the operation of the image processing device, such as a central processing unit (Central Processing Unit, CPU), or other programmable general-purpose or special-purpose microprocessors Microprocessor, Digital Signal Processor (DSP), Programmable Controller, Application Specific Integrated Circuits (ASIC), Programmable Logic Device (PLD), graphics Processor (Graphics Processing Unit, GPU or other similar devices or a combination of these devices. The processor 110 can execute program codes, software modules, instructions, image quality evaluation items, default values for image processing, image recognition software recorded in the storage unit and so on, so as to realize the image processing method in the embodiment of the present invention.

FIG. 2 is a flowchart of an image processing method according to an embodiment of the invention. Please refer to FIG. 2 , the method of this embodiment is applicable to the above-mentioned image processing device 100. In this embodiment, the image processing method of the present invention at least performs the following steps according to the instructions of the image processing device 100: the processor 110 receives a plurality of image quality Evaluation items (step S210). For example, the image quality evaluation item may be entropy of the calculated image, average gradient of the calculated image, standard deviation of the calculated image, edge intensity of the calculated image, etc., that is to say, A plurality of different image quality evaluation items are sent to the processor 110 according to user presets or setting values of image quality evaluation items built in the image processing device.

FIG. 3 is a schematic flowchart of an image processing method according to an embodiment of the present invention. The processor 110 calculates the total running time of the image quality evaluation item executed on the cloud server (Cloud Server), and calculates the individual running time of the image quality evaluation item executed on the image processing device (step S220 ). Specifically, the processor 110 receives the test image and executes each image quality evaluation item on the test image and calculates the individual running time of each image quality evaluation item. Please refer to Fig. 3, in one embodiment, the default value of image quality evaluation item is four kinds, respectively is image quality evaluation item A (namely, IQA A), image quality evaluation item B (namely, IQA B), image quality An evaluation item C (ie, IQA C) and an image quality evaluation item D (ie, IQA D). Moreover, the running time of the image quality evaluation items calculated by the processor 110 in the processor 110 is 7 seconds (IQA A), 2 seconds (IQA B), 3 seconds (IQA C) and 6 seconds (IQA D). At the same time, the processor 110 uploads the test image to the cloud server to execute each image quality evaluation item through the cloud server. Next, the processor 110 calculates the total running time Cloud_Time spent to receive the result of executing the image quality evaluation item from the cloud server. For example, the processor 110 calculates the time between when the processor 110 uploads the test image to the cloud server and when the processor 110 receives the results of the image quality evaluation items (A, B, C, D) from the cloud server. The total running time of Cloud_Time is 9 seconds.

The processor 110 divides the image quality evaluation items into a first type of image quality evaluation item and a second type of image quality evaluation item according to the individual running time and the total running time (step S240 ). FIG. 4 is a flowchart of image recognition according to an embodiment of the present invention. Please refer to Figure 2, Figure 3 and Figure 4. Specifically, the processor 110 calculates the classification whose running time is less than the total running time according to the individual running time of executing the image quality evaluation item on the processor 110 and the total running time of executing the image quality evaluating item on the cloud server (Cloud), Furthermore, the image quality evaluation items are classified into the first type of image quality evaluation items and the second type of image quality evaluation items. In one embodiment, step S240 further includes: the processor 110 classifies the image quality evaluation items whose sum of individual running times is less than the total running time as the first type of image quality evaluation items IQA_G1, and classifies the rest of the image quality evaluation items as The second category of image quality assessment items IQA_G2. For example, in one embodiment, the total running time Cloud_Time is 9 seconds, and the processor 110 classifies the image quality evaluation item B with an individual running time of 2 seconds and the image quality evaluation item C with an individual running time of 3 seconds as the first A class of image quality assessment items IQA_G1. Next, the processor 110 classifies the remaining image quality evaluation items A and D into the second type of image quality evaluation item IQA_G2.

In another embodiment, the processor 110 divides the image quality evaluation items into the first type image quality evaluation item IQA_G1 and the second type image quality evaluation item IQA_G2 according to the individual running time and the total running time (step S240) further includes: processing The device 110 arranges the image quality evaluation items sequentially according to their individual running times. For example, image quality evaluation item A, image quality evaluation item B, image quality evaluation item C, and image quality evaluation item D are arranged in order according to their individual running times of 7, 2, 3, and 6 seconds as image quality evaluation items B, Image quality evaluation item C, image quality evaluation item D, image quality evaluation item A. Next, the processor 110 classifies the first N image quality evaluation items into the first type of image quality evaluation items IQA_G1, and classifies the rest of the image quality evaluation items into the second type of image quality evaluation items IQA_G2. For example, the N value can be set as one-third or one-half of the total number of image quality evaluation items. On the other hand, the N value may be that the sum of the first N individual running times of the sequentially arranged image quality evaluation items is less than the total running time. For example, taking the first two image quality evaluation items B and image quality evaluation item C, the individual running time of the sum is 5 seconds, which is less than 9 seconds of the total running time of the image quality evaluation items executed by the cloud. In this embodiment, N is a positive integer.

Processor 110 divides multiple image quality evaluation items into first-type image quality evaluation items and second-type image quality evaluation items according to individual running time and total running time. After the step (step S220), the present invention The image processing method further includes: the processor 110 classifies the image quality evaluation items into a first type of image quality evaluation item and a second type of image quality evaluation item, and outputs a distribution result IQA_CR. Next, the processor 110 stores the distribution result IQA_CR in the storage unit 120 , and sets the distribution result IQA_CR as the image processing default value, and the image processing device 100 and the cloud server execute subsequent image quality evaluation items according to the distribution result IQA_CR. Specifically, the processor 110 records the result classified in step S220 as an allocation result IQA_CR and stores it in the storage unit 120 of the image processing device 100 . Next, the processor 100 sets the allocation result IQA_CR as the default value of the influencing processing device. That is to say, the subsequent image processing of the image processing device 100 is performed by the image processing device 100 (ie, the local end) and the cloud server respectively according to the allocation result IQA_CR for subsequent image quality evaluation items. In other words, in the subsequent image processing process, the image processing device executes the first type of image quality evaluation item IQA_G1 by the processor 110 according to the allocation result IQA_CR, and the second type of image quality evaluation item IQA_G2 is executed by the cloud server.

Before the step of calculating the total running time of multiple image quality evaluation items executed on the cloud server (step S220), the image processing method of the present invention further includes: testing whether the image processing device 100 supports each of the multiple image quality evaluation items respectively an image quality evaluation item (step S211 ); and classify the image quality evaluation item not supported by the image processing device 100 into the second type of image quality evaluation item IQA_G2. Specifically, the processor 110 tests whether the image processing device 100 supports each image quality evaluation item, and classifies the image quality evaluation items not supported by the image processing device into the second type of image quality evaluation items, so as to improve the quality of the image quality evaluation items. classification efficiency. For example, some image quality evaluation items require a specific peripheral device or a specific display adapter level. The image quality evaluation items of the display adapter are classified into the second type of image quality evaluation items. In another embodiment, the processor 110 may classify individual image quality assessment items as the second type of image quality assessment item IQA_G2 according to the running time of the individual image quality assessment item being greater than a threshold. For example, the processor 110 classifies the image quality assessment items whose running time is longer than 5 seconds into the second type of image quality assessment items IQA_G2, so as to speed up the processing speed of the image quality assessment by the image processing device 100 .

After the image processing device 100 executes the first type of image quality evaluation item, and the cloud server executes the step of the second type of image quality evaluation item (step S240), the image processing method of the present invention further includes: the cloud server converts the second type The images that all pass the image quality evaluation item IQA_G2 are used in the image recognition model 2, and the image recognition result RL is output; and the image processing device 100 executes the first type of image quality evaluation item to output the execution result, and if the execution result is passed, the image The recognition result RL is displayed on the display screen 130; if the execution result is not passed, the image recognition result RL is not used. Specifically, according to the distribution result IQA_CR of the processor 110 , the cloud server executes the second type of image quality evaluation item IQA_G2 on the subsequent image Img_m to be evaluated. Next, the images Img_m that all pass the second type of image quality assessment item IQA_G2 are used in image recognition to output an image recognition result RL of the image.

It is worth noting that the processor 110 of the image processing device 100 executes the first type of image quality evaluation item IQA_G1 on subsequent images that need quality evaluation to output the execution result, and the processor 110 executes the first type of image quality evaluation item IQA_G1 according to the execution result. The result is correspondingly displayed or not using the image recognition result RL output by the cloud server. Specifically, if the execution result of the first type of image quality evaluation item IQA_G1 is passed, and the execution result of the second type of image quality evaluation item IQA_G2 executed by the cloud server is also passed and the image recognition result RL is output, then the processor 110 The image recognition result RL is received and output and displayed on the display screen 130 coupled to the processor 110 . On the other hand, if the execution result of the first type of image quality evaluation item IQA_G1 executed by the processor 110 is not passed, the processor 110 does not use (ie, ignore) the image recognition result RL output by the cloud server. In this way, the user can use the image processing method of the present invention to improve the execution efficiency of image quality evaluation in image processing and shorten the time required for image quality evaluation.

FIG. 5A is a schematic diagram of an image processing method. FIG. 5B is a schematic diagram of an image processing method according to an embodiment of the present invention. Please refer to FIG. 5A and FIG. 5B. FIG. 5A is an existing method that does not use the image processing method of the present invention. The image is uploaded to the cloud server C and the image quality is evaluated through the cloud server C. Because the image quality is lower than the image quality The set value of the evaluation item is therefore returned to the image by the cloud server C and rejected for image recognition. Then, the user must upload a new image again, and then perform image quality evaluation and image recognition (Image Recognition AI) through the cloud server C, and then transmit the image recognition result to the user's image processing device (ie, the local terminal C). It can be known from FIG. 5A that the existing method takes 20 seconds for image recognition. However, FIG. 5B is a schematic flow chart of the image processing method of the present invention. It can be known from FIG. 5B that when the image processing device 100 uploads an image to the cloud server, the image processing device 100 can simultaneously execute the image quality evaluation item B and Image quality evaluation item C. Therefore, when the image processing device 100 executes the image quality evaluation item B, the quality of the output image (same, image) does not meet the set value of the image quality evaluation item B, that is, upload another image to the cloud server C again, so, By using the image processing method of the present invention, it takes 6 seconds to upload the image, 1 second to perform IQA A & IQA D, image recognition and 2 Before the recognition result is returned within seconds, the image can be uploaded for the second time through the image processing device 100 at the local end, thereby shortening the time required for image recognition to 13 seconds. It should be added that, as is well known to those skilled in the art, existing image quality assessment items (ie, image quality assessment methods (Image Quality Assessment Method)) are not limited to the four types in this embodiment, and the images The setting value of the quality evaluation is determined according to the default value of the selected image quality evaluation method, and since it belongs to the content and technology well-known to those skilled in the art, so this case will not repeat it here. On the other hand, the image recognition processing may be image recognition processing for recognizing diabetic retinopathy through an image scanned by eyes, iris recognition processing, character recognition, pattern recognition, object detection image recognition, face recognition, and the like.

To sum up, in the embodiment of the present invention, when the user uses the image processing method to assist the image quality evaluation in the image processing, the image quality evaluation items can be divided into the first type of image quality evaluation according to the individual running time and the total running time Items and the second type of image quality evaluation items, the image processing device at the local end executes the first type of image quality evaluation items by classification, and the cloud server executes the second type of image quality evaluation items, and then according to the local end image processing device According to the status of the equipment and the connection status between the local end and the cloud server, some image quality evaluation items are dynamically assigned to be executed on the local end (that is, the image processing device), and the other part of the image quality evaluation items are executed on the cloud server to achieve improvement. Processing time for image quality assessment and improved detection efficiency of image processing methods (ie, image recognition).

Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present invention, rather than limiting them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: It is still possible to modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the technical solutions of the various embodiments of the present invention. scope.

Claims (8)

1. An image processing method applied to an image processing apparatus having a processor and a storage unit, the image processing method performing the following steps according to an instruction of the image processing apparatus

Receiving a plurality of image quality evaluation items;

calculating the total running time of the image quality evaluation items executed on the cloud server, and calculating the individual running time of the image quality evaluation items executed on the image processing device;

classifying a plurality of image quality evaluation items into a first type of image quality evaluation item and a second type of image quality evaluation item according to the individual running time and the total running time; and

executing, by the image processing device, a first type of image quality evaluation item, and executing, by the cloud server, a second type of image quality evaluation item.

2. The image processing method according to claim 1, wherein the step of classifying the image quality evaluation items into a first type of image quality evaluation item and a second type of image quality evaluation item according to the individual running time and the total running time further comprises:

at least one image quality evaluation item, the individual running times of which are added to be less than the total running time, is classified as a first type of image quality evaluation item, and the remaining image quality evaluation items in the plurality of image quality evaluation items are classified as a second type of image quality evaluation item.

3. The image processing method according to claim 1, wherein the step of classifying the plurality of image quality evaluation items into a first type of image quality evaluation item and a second type of image quality evaluation item according to the individual running time and the total running time further comprises:

arranging the image quality evaluation items in sequence according to respective running times; and

classifying the first N image quality evaluation items into a first type of image quality evaluation items, and classifying the rest image quality evaluation items into a second type of image quality evaluation items;

wherein a sum of individual running times of the image quality evaluation items of the first category is less than the total running time of the image quality evaluation items.

4. The image processing method according to claim 1, wherein the processor further comprises, after the step of dividing the plurality of image quality evaluation items into a first type of image quality evaluation item and a second type of image quality evaluation item according to the individual running time and the total running time:

and generating the distribution result of the image quality evaluation items by using the first type of image quality evaluation items and the second type of image quality evaluation items.

5. The image processing method according to claim 1, wherein the step of calculating the total runtime of the plurality of image quality evaluation items performed by the cloud server further comprises:

testing whether the image processing apparatus supports each of the plurality of image quality evaluation items; and

classifying the image quality evaluation items not supported by the image processing apparatus as the second type of image quality evaluation items.

6. The image processing method according to claim 1, wherein the step of classifying the plurality of image quality evaluation items into a first type of image quality evaluation item and a second type of image quality evaluation item according to the individual running time and the total running time further comprises:

classifying the image quality evaluation items whose individual run times are greater than a threshold value as the second type of image quality evaluation items.

7. The image processing method according to claim 1, wherein after the steps of executing a first type of image quality evaluation item by the image processing apparatus and executing a second type of image quality evaluation item by the cloud server, further comprising:

the cloud server uses the images which pass through the second type of image quality evaluation items for image recognition and outputs image recognition results; and

the image processing device executes the first type of image quality evaluation item to output an execution result, and if the execution result is passed, the image recognition result is displayed on a display screen; and if the execution result is not passed, the image recognition result is not adopted.

8. The image processing method according to claim 4, wherein said step of generating the assignment result of the image quality evaluation item further comprises, after said step of:

storing the allocation result in the storage unit; and

and setting the distribution result as an image processing default value.

Images