TW202139075A - Deep learning model training system, deep learning model training method, and non-transitory computer readable storage medium - Google Patents

Abstract

A deep learning model training system, deep learning model training method, and non-transitory computer readable storage medium are provided. The method includes: adjusting an original image set to obtain testing image sets according to testing values; inputting the testing training image sets to a first learning model for testing to obtain confidence values, wherein the confidence values correspond to the testing values in a one-to-one manner; classifying the test values into an invalid group or a valid group according to whether the confidence value corresponding to each of the test value is less than a threshold value; obtaining training values according to the test values in the invalid group; adjusting the original image set to obtain training image sets according to the training values; and inputting the original image set and the training image sets to the first deep learning model for training to obtain a second deep learning model.

Description

Deep learning model training system, deep learning model training method, and non-transient computer readable storage media

This case is about the field of artificial intelligence, especially a deep learning model training system and method.

In recent years, the research of artificial intelligence has been changing rapidly, especially the application of deep learning to various fields, including the field of image recognition. Today's deep learning systems in the field of image recognition usually require users to use a huge amount of images as training data to train the deep learning system, so that the deep learning system can operate normally, that is, the output result of the deep learning system has a higher Confidence (also known as Confidence). Although such a training method is easy to understand in theory, it is not easy to implement because it is time-consuming to process and obtain a huge amount of images and usually requires a high-performance processor to execute. In addition to collecting the original images, the method of obtaining a huge amount of images is to obtain similar images by adjusting the original images.

However, in the process of training a deep learning system with a huge amount of images as training data, you will find that the amount of training is not proportional to the increase in confidence. For example, a large amount of training only slightly increases the confidence. In other words, the current method of adjusting the original image to obtain a similar image can increase training data, but it is a time-consuming and ineffective training method.

In addition, in current meter research, although smart meters have been developed, smart meters use electronic measurement and report data to make measurement data online in real time. But in practice, smart meters are not common, and most users still use traditional meters. If the traditional meter is replaced with a smart meter, the water supply representative will need to stop water, and the electricity supply will need to be cut off. This not only costs the cost of the meter itself, but also requires a 24-hour operation of the factory. It bears a huge time cost, so it is undesirable for users to replace traditional meters with smart meters. However, traditional meters require manual meter reading to collect and count measurement data, which is not only inefficient but also prone to errors. Therefore, today's meters still need to be improved.

In view of the above, this case proposes a deep learning model training system and method.

According to some embodiments, the deep learning model training method includes: adjusting the original image set according to a plurality of test values to obtain a plurality of test image sets; inputting the test image set to the first deep learning model for testing, and obtaining a plurality of confidence levels. Correspond to the test value in a one-to-one manner; classify the test value into a valid group or an invalid group according to whether the confidence level corresponding to each test value is less than a threshold; obtain multiple training values according to the test value in the invalid group; The training value is adjusted to the original image set to obtain multiple training image sets for training; and the original image set and the training image set are input to the first deep learning model to obtain the second deep learning model.

According to some embodiments, the deep learning model training system includes a processor and a storage device. The processor is used to receive the original image set, multiple test values and threshold values. The processor is used to adjust the original image set according to the test value to obtain a plurality of test image sets, and input the test image set to the first deep learning model for testing to obtain a plurality of confidences, and the confidences correspond to the test values in a one-to-one manner. The processor classifies the test value into a valid group or an invalid group according to whether the confidence level corresponding to each test value is less than a threshold value, and obtains a plurality of training values according to the test value in the invalid group. The processor is used to adjust the original image set according to the training value to obtain a plurality of training image sets, and input the original image set and the training image set to the first deep learning model for training to obtain the second deep learning model.

According to some embodiments, a non-transitory computer-readable storage medium is used to store one or more software programs. The one or more software programs include a plurality of instructions, when these instructions are executed by one or more processors of the electronic device , Which will enable electronic devices to perform deep learning model training methods.

In summary, the deep learning model training system, the deep learning model training system method, and the non-transitory computer-readable storage medium of some embodiments of this case can adjust the original image set according to the test value to obtain the test image set, and input the test image set The first deep learning model is tested to obtain the confidence of the test image set, that is, the confidence of the test value. And according to the threshold value, the test value can be classified into the valid group or the invalid group, and then the training value can be obtained from the test value in the invalid group. And adjust the original image set according to the training value to obtain the training image set, and input the original image set and the training image set to the first deep learning model for training to obtain the second deep learning model.

Hereinafter, multiple embodiments of this case will be disclosed in schematic form. For the sake of clarity, many practical details will be described in the following description. However, it should be understood that these practical details should not be used to limit the case. In other words, in some embodiments of this case, these practical details are unnecessary. In addition, for the sake of simplifying the drawings, some conventionally used structures and elements will be drawn in a simple schematic manner in the drawings, and in all the drawings, the same reference numerals will be used to denote the same or similar elements. . And if it is possible in implementation, the features of different embodiments can be applied interactively.

FIG. 1 is a schematic diagram of a deep learning model training system 100 according to some embodiments of the present case. Please refer to FIG. 1. In some embodiments, the deep learning model training system 100 includes a processor 120 and a storage device 140. The processor 120 is configured to receive an original image set, a plurality of test values and threshold values, and adjust the original image set according to the test values to obtain a plurality of test image sets. The processor 120 is configured to input the test image set to the first deep learning model T1 for testing to obtain multiple confidence levels, and the confidence levels correspond to the test values in a one-to-one manner. The processor 120 is configured to classify the test value into a valid group or an invalid group according to whether the confidence level corresponding to the test value is less than a threshold value, and obtain a plurality of training values according to the test value in the invalid group. The processor 120 is configured to adjust the original image set according to the training value to obtain a plurality of training image sets. The processor 120 is configured to input the original image set and the training image set to the first deep learning model T1 for training to obtain the second deep learning model T2. The storage device 140 is used to store the first deep learning model T1 and the second deep learning model T2. In some embodiments, the storage device 140 is electrically connected to the processor 120.

In some embodiments, the deep learning model (ie, the first deep learning model T1 or the second deep learning model T2) is suitable for use in image recognition, such as recognizing the value of a digital image in the image. Taking the image with the digital image "0011" as an example, the deep learning model is used to identify the image to obtain the value "0011".

In some embodiments, when the deep learning model is tested, the deep learning model outputs the test answer (for example, the value "0011") and the confidence of the test answer according to the input test question (for example, an image with a digital image "0011") Spend. Specifically, the confidence level represents how confident the deep learning model is that the test answer is correct, or how confident the deep learning model is that the input test question can be answered correctly, that is, the correct rate of the test answer. Therefore, the confidence level is between 0 and 1. When the confidence level is 1, the correct rate of the test answer is 100%, the confidence level of 0.5 represents the correct rate of the test answer is 50%, and when the confidence level is 0, the test answer is correct. The correct rate is 0%.

In some embodiments, the deep learning model is trained by supervised learning. Specifically, supervised learning involves inputting corresponding training questions and training answers to the deep learning model for training. The deep learning model after training will be based on The received training question and training answer are used to process the test question, that is, the deep learning model can compare the test question with the training question, and output the test answer corresponding to the test question according to the training answer. Therefore, after the deep learning model is based on appropriate supervised learning, the deep learning model can improve the confidence of the output, that is, the correct rate of the test answer. When this manual refers to “input images (sets) to the first deep learning model T1 for training, and obtain the second deep learning model T2”, it means “input images (sets) and corresponding values (sets) to the first A deep learning model T1 is trained, and a second deep learning model T2 is obtained". The image (set) is the training question in supervised learning, and the corresponding value (set) is the training answer in supervised learning. The corresponding value (set) is not limited to be obtained in any form. For example, the image (set) file itself has a value (set), the value (set) is input from the outside to the deep learning model training system 100, or the processor 120 is based on other The recognition model recognizes the image (set) and generates a value (set).

Fig. 2 is a flowchart of a deep learning model training method according to some embodiments of the present case. In order to clearly explain the operation of the various components in FIG. 1, the following will be described in detail with the flowchart in FIG. 2 as follows. However, anyone with ordinary knowledge in the technical field to which this case belongs can understand that the deep learning model training method of the embodiment of this case is not limited to being applied to the deep learning model training system 100 of FIG. 1, nor is it limited to each of the flowcharts in FIG. Item step sequence. In some embodiments, a non-transitory computer-readable storage medium is used to store one or more software programs, and the one or more software programs include a plurality of instructions, when these instructions are executed by one or more processing circuits of the electronic device , Which will enable electronic devices to perform deep learning model training methods. Specifically, the electronic device includes one or more processing circuits (also referred to as control circuits). The deep learning model training method is implemented by, for example, one or more software programs. The software programs are stored on CDs, hard disks or other non-transitory computer-readable storage media, and the software programs include multiple instructions related to processing circuits. After these commands or software programs are loaded by the electronic device, the electronic device will execute the deep learning model training method. The detailed description of each step of the deep learning model training method is listed below.

Please refer to FIGS. 1 and 2 together. In some embodiments, the process of the deep learning model training method includes a test image set obtaining step (step S210), a model testing step (step S220), a grouping step (step S230), and training values. The obtaining step (step S240), the training image set obtaining step (step S250), and the model training step (step S260).

In some embodiments, the test image set obtaining step (step S210 in FIG. 2) includes: adjusting the original image set according to a plurality of test values to obtain a plurality of test image sets. Specifically, the original image set includes a plurality of original images, and each test image set includes a plurality of test images. The test value takes the angle rotation as an example, the unit of the test value is "degree", and each test value is "-10, -9.5, -9, -8.5, -8, -7.5, -7, -6.5, -6, -5.5, -5, -4.5, -4, -3.5, -3, -2.5, -2, -1.5, -1, -0.5, 0, 0.5, 1, 1.5, 2, 2.5, 3, 3.5, 4 , 4.5, 5, 5.5, 6, 6.5, 7, 7.5, 8, 8.5, 9, 9.5, 10". A positive test value indicates that the original image is rotated clockwise, a negative test value indicates that the original image is rotated counterclockwise, and a test value of "0" indicates that the original image is not rotated, but it is not limited to this. In some embodiments, the test image set corresponds to the test value in a one-to-one manner, for example, one test image set is rotated by "-10" degrees, another test image set is rotated by "-9.5" degrees, and so on. When there are "21" test values, the processor 120 adjusts the original image set according to the "21" test values to obtain corresponding "21" test image sets. In some embodiments, the number of original images in an original image set multiplied by the test scale factor is equal to the number of test images in a test image set, and the test scale factor is between 0 and 1. Specifically, the processor 120 selects a part of the original image set from the original image set according to the test scale factor, and adjusts the part of the original image set to obtain each test image set. In particular, "the number of original images in a part of the original image set" (or, "the number of test images in a test image set") divided by the "number of original images in the original image set" "Is equal to the test scale factor. For example, when the number of original images is "1000", the test value is "21", and the test scale factor is "0.1", the number of some original images is "1000*0.1=100", and the test images in each test image set The number of is "100", so the total number of test images in the "21" test image set is "21*100=2100".

In some embodiments, the test value is an arithmetic sequence. For example, in the above-mentioned angle rotation embodiment, the maximum value of the test value is "10" degrees, and the minimum value of the test value is "-10" degrees, which is the test value of the arithmetic sequence The tolerance is "0.5".

In some embodiments, when the processor 120 adjusts the original image set according to the test value to obtain multiple test image sets, each test value only corresponds to the same adjustment category, for example, the adjustment category is angle rotation, but it is not limited to this. In some embodiments, the adjustment categories include, but are not limited to, angle rotation, equal scaling, height enlargement, width enlargement, contrast adjustment, brightness adjustment, and gamma value adjustment. The "angle rotation" can rotate the original image in any direction. For example, in the Cartesian coordinate system, the original image includes a first axis and a second axis. The first axis is the Y axis and the second axis is the Z axis. . The original image can be rotated on the X axis, Y axis, Z axis or any axis as the test image. "Height magnification" means that the original image is stretched while maintaining the same width, and "width magnification" means that the original image is stretched while maintaining the same height.

In some embodiments, the test value includes the original value corresponding to the original image set. Specifically, the original value is one of the test values, usually the original value is the benchmark of the test value, for example, the original value is the minimum, maximum, or median of the test values. In terms of the adjustment category corresponding to the test value, the original value is the value of the original image set corresponding to the adjustment category. For example, when the adjustment category is angle rotation, the original value is "0" degrees, that is, the rotation angle of the original image set is "0" degrees (that is, not rotated). For example, when the adjustment category is equal scaling, the original value is "1", that is, the ratio of the original image set is "1" times (that is, the original size).

In some embodiments, the test image set corresponding to the original value is a partial original image set. Specifically, when the processor 120 adjusts the original image set according to the original value to obtain the test image set, the processor 120 uses the original image set as the test image set, that is, the processor 120 can obtain the test image without adjusting the original image set. set. For example, when the adjustment category corresponding to the test value is angle rotation, the original value is "0" degrees, so the processor 120 can use the original image set as the test image set. In some embodiments, the processor 120 determines part of the original image set as the test image set according to the test scale factor.

In some embodiments, the model testing step (step S220) includes: inputting a test image set to the first deep learning model T1 for testing to obtain a plurality of confidence levels, and the confidence levels correspond to the test value in a one-to-one manner. From the foregoing description of the deep learning model, it can be seen that the confidence level represents how confident the first deep learning model T1 is for the input test image set to correctly output the corresponding test answer. Therefore, when the processor 120 inputs the test image set to the first deep learning model T1 for testing, the processor 120 can obtain the confidence level corresponding to the test image set from the first deep learning model T1. Since the test image set is adjusted according to the test value, the confidence level corresponds to the test value in a one-to-one manner. In some embodiments, the confidence level corresponding to the test image set is an average of multiple sub-confidence levels. Specifically, the test image set includes a plurality of test images, the first deep learning model T1 corresponds to the output confidence level according to the input test image set, and the first deep learning model T1 outputs each sub-belief according to each input test image. Spend. Therefore, when the processor 120 inputs any test image to the first deep learning model T1 for testing, the first deep learning model T1 can output the sub-confidence level corresponding to this test image. Therefore, the processor 120 includes a certain test image set The average calculation of all sub-confidences corresponding to each test image can obtain the confidence of this test image set. In some embodiments, the confidence level corresponding to the test image set may be the median, maximum value, and minimum value of multiple sub-confidence levels, and is not limited thereto.

In some embodiments, the test value is angle rotation as an example, and the comparison table between the test value and the confidence level is shown in Table 1 below:

Table 1 test value Confidence test value Confidence test value Confidence -10 0 -2.5 0.5 3 0.4 -9.5 0 -2 0.6 3.5 0.3 -9 0 -1.5 0.7 4 0.2 -8.5 0 -1 0.8 4.5 0.1 -8 0 -0.5 0.9 5 0 -7.5 0 0 1 5.5 0 -7 0 0.5 0.9 6 0 -6.5 0 1 0.8 6.5 0 -6 0 1.5 0.7 7 0 -5.5 0 2 0.6 7.5 0 -5 0 2.5 0.5 8 0 -4.5 0.1 8.5 0 -4 0.2 9 0 -3.5 0.3 9.5 0 -3 0.4 10 0

In some embodiments, the grouping step (step S230 in FIG. 2) includes: judging whether the confidence level corresponding to each test value is less than a threshold value, and classifying the test value into an invalid group or a valid group. Specifically, the higher the confidence, the more accurately the first deep learning model T1 can recognize the input test image set, and the lower the confidence, the less the first deep learning model T1 can recognize correctly. Therefore, the processor 120 determines whether the confidence level corresponding to the test value is less than the threshold value, distinguishes the valid confidence level from the invalid confidence level through the threshold value, classifies the test value with the confidence level less than the threshold value into the invalid group, and classifies the confidence level greater than or The test value equal to the threshold is classified into the effective group. Taking the test value of angle rotation as an example and referring to Table 1, if the threshold value is "0.5", when the confidence level is less than "0.5", it means that the first deep learning model T1 cannot be effectively and correctly identified, so the confidence level is less than "0.5". Test value "-10, -9.5, -9, -8.5, -8, -7.5, -7, -6.5, -6, -5.5, -5, -4.5, -4, -3.5, -3, 3, "3.5, 4, 4.5, 5, 5.5, 6, 6.5, 7, 7.5, 8, 8.5, 9, 9.5, 10" are classified into the invalid group. When the confidence level is greater than or equal to "0.5", it means that the first deep learning model T1 can be effectively and correctly identified, so the confidence level is greater than or equal to the test value "-2.5, -2, -1.5, -1, -0.5" corresponding to "0.5" , 0, 0.5, 1, 1.5, 2, 2.5" are classified into the effective group.

In some embodiments, the test value takes angular rotation as an example, and the comparison table of the invalid group and the valid group is shown in Table 2 below:

Table 2 Invalid group Effective group Invalid group test value Confidence test value Confidence test value Confidence -10 0 -2.5 0.5 3 0.4 -9.5 0 -2 0.6 3.5 0.3 -9 0 -1.5 0.7 4 0.2 -8.5 0 -1 0.8 4.5 0.1 -8 0 -0.5 0.9 5 0 -7.5 0 0 1 5.5 0 -7 0 0.5 0.9 6 0 -6.5 0 1 0.8 6.5 0 -6 0 1.5 0.7 7 0 -5.5 0 2 0.6 7.5 0 -5 0 2.5 0.5 8 0 -4.5 0.1 8.5 0 -4 0.2 9 0 -3.5 0.3 9.5 0 -3 0.4 10 0

In some embodiments, the training value obtaining step (step S240 in FIG. 2) includes: obtaining a plurality of training values according to the test value in the invalid group. Specifically, the test image set corresponding to the test value in the invalid group is the test image set that cannot be effectively and correctly identified by the first deep learning model T1, so the test image set corresponding to the test value in the invalid group is the first one. The deep learning model T1 needs to strengthen the training part, so the processor 120 obtains the training value according to the test value in the invalid group. In some embodiments, the training value is all the test values in the invalid group. Taking the test value as angle rotation as an example and referring to Table 2, the training value is "-10, -9.5, -9, -8.5, -8, -7.5 , -7, -6.5, -6, -5.5, -5, -4.5, -4, -3.5, -3, 3, 3.5, 4, 4.5, 5, 5.5, 6, 6.5, 7, 7.5, 8, 8.5, 9, 9.5, 10".

In some embodiments, the training value obtaining step (step S240 in FIG. 2) further includes: obtaining a valid interval value according to the difference between the largest test value in the valid group and the original value; and according to multiple integers of the valid interval value Times, the training value is obtained from the test value in the invalid group. Specifically, the process of the processor 120 obtaining the training value according to the test value in the invalid group includes the above steps, wherein the processor 120 calculates the difference between the largest test value in the valid group and the original value, and then calculates the difference between the two values of the difference. Obtain the valid interval value (that is, the valid interval value is twice the difference between the largest test value in the valid group and the original value), and the processor 120 will invalidate the tests that meet multiple integer multiples of the valid interval value The value is used as the training value. Taking the test value of angle rotation as an example and referring to Table 2, the largest test value in the effective group is "2.5", the original value is "0", and the difference between the largest test value in the effective group and the original value is "| 2.5-0|=2.5", twice the difference is "2*2.5=5", so the valid interval value is "5". The multiple integer multiples of the valid interval value are "5*m, m=0, ±1, ±2,...", so the test values that meet multiple integer multiples of the valid interval value in the invalid group are "-10, -5" , 5, 10", so the training value is "-10, -5, 5, 10".

In some embodiments, the training image set obtaining step (step S250 in FIG. 2) includes: adjusting the original image set according to the training value to obtain a plurality of training image sets. Specifically, the training image set includes a plurality of training images. Taking the test value as angle rotation as an example and referring to Table 2, the unit of the training value is "degree", and the training value is "-10, -5, 5, 10". In some embodiments, the training image set corresponds to the training value in a one-to-one manner. For example, the original image set is rotated by "-10" degrees to obtain one training image set, and the original image set is rotated by "-5" degrees to obtain another A training image set, and so on, so when there are "4" training values, the processor 120 adjusts the original image set according to the "4" training values to obtain "4" training image sets. In some embodiments, the number of original images in an original image set multiplied by the training scale factor is equal to the number of training images in a training image set, and the training scale factor is between 0 and 1. Specifically, the training scale factor is similar to the test scale factor, and the difference is that the training scale factor is, for example, but not limited to, equal to the test scale factor. The processor 120 selects a part of the original image set from the original image set according to the training scale coefficient, and adjusts a part of the original image set to obtain each training image set. In particular, "the number of original images in a part of the original image set" (or, "the number of training images in a training image set") divided by the "number of original images in the original image set" "Is equal to the training ratio coefficient. For example, when the number of original images is "1000" and the training scale factor is "0.5", the number of training images in a training image set is "1000*0.5=500", the same is true for "4" training image sets The total number of training images is "4*500=2000".

In some embodiments, the model training step (step S260 in FIG. 2) includes: inputting the original image set and the training image set to the first deep learning model T1 for training, and obtaining the second deep learning model T2. Specifically, when the processor 120 inputs the original image set and the training image set to the first deep learning model T1 for training, the first deep learning model T1 uses the original image set and each training image set as training data. In other words, the second deep learning model T2 can improve the confidence of each test image set of the invalid group based on the original image set and each training image set. Therefore, compared with the first deep learning model T1, without all the test values in the invalid group as training values, the second deep learning model T2 can use each training image set as training data to improve the invalid group Each test image set corresponds to the confidence level, so the amount of training data for the deep learning model can be streamlined. Taking the test value of angle rotation as an example and referring to Table 2, the processor 120 only needs to input the original image set and the training image set corresponding to the training value "-10, -5, 5, 10" (a total of 4 training image sets) To the first deep learning model T1 for training.

In some embodiments, taking the training scale factor of 1 as an example, the number of training images in the training image set is equal to the number of original images in the original image set. For all test values in the invalid group (ie, -10, -9.5 , -9, -8.5, -8, -7.5, -7, -6.5, -6, -5.5, -5, -4.5, -4, -3.5, -3, 3, 3.5, 4, 4.5, 5, 5.5, 6, 6.5, 7, 7.5, 8, 8.5, 9, 9.5, 10) as the training value (a total of 30 training image sets, excluding the original image set) for comparison, the deep learning model training system 100 And the deep learning model training method only needs "(1+4)/(1+30) = 16.1%" training data volume. For comparison with all test values as training values (a total of 41 training image sets, including original image sets), the deep learning model training system 100 and the deep learning model training method only need "(1+4)/( 1+40) = 12.2%" training data volume. The foregoing embodiment of reducing the amount of training data is only used as an example and is not limited to this. The amount of training data reduced by the learning model training system 100 and the deep learning model training method will be based on different adjustment categories, different test values, and different Confidence and different thresholds have different results.

In some embodiments, the test value is angle rotation as an example, and the comparison table between the test value and the confidence level is shown in Table 3 below:

table 3 Effective group Invalid group test value Confidence test value Confidence 0 1 3 0.4 0.5 0.9 3.5 0.3 1 0.8 4 0.2 1.5 0.7 4.5 0.1 2 0.6 5 0 2.5 0.5 5.5 0 6 0 6.5 0 7 0 7.5 0 8 0 8.5 0 9 0 9.5 0 10 0

In some embodiments, each test value according to Table 3 is "0, 0.5, 1, 1.5, 2, 2.5, 3, 3.5, 4, 4.5, 5, 5.5, 6, 6.5, 7, 7.5, 8, 8.5, 9, 9.5, 10", the original value is "0" degrees. Since the threshold value is "0.5", the test value "3, 3.5, 4, 4.5, 5, 5.5, 6, 6.5, 7, 7.5, 8, 8.5, 9, 9.5, 10" classifications with confidence less than "0.5" In the invalid group, test values "0, 0.5, 1, 1.5, 2, 2.5" with a confidence level greater than or equal to "0.5" are classified into the valid group. In some embodiments, when the training value is all test values in the invalid group, the training value is "3, 3.5, 4, 4.5, 5, 5.5, 6, 6.5, 7, 7.5, 8, 8.5, 9, 9.5, 10. ". In some embodiments, the training value obtaining step (step S250 in FIG. 2) further includes: obtaining a valid interval value according to the difference between the largest test value in the valid group and the original value; and according to multiple integers of the valid interval value Times, the training value is obtained from the test value in the invalid group. The largest test value in the effective group is "2.5", the original value is "0", and the difference between the largest test value in the effective group and the original value is "|2.5-0|=2.5", which is twice the difference It is "2*2.5=5", so the valid interval value is "5". The multiple integer multiples of the valid interval value are "5*m, m=0, ±1, ±2,...", so the test value of multiple integer multiples of the valid interval value in the invalid group is "5, 10", So the training value is "5, 10".

In some embodiments, the deep learning model training method further includes an importance degree obtaining step, which is used to obtain the importance degree corresponding to the test value. In some embodiments, the processor 120 obtains the importance corresponding to the test value according to the importance obtaining step. Specifically, when the processor 120 adjusts the original image set to a plurality of test image sets according to the test value, each test value only corresponds to the same adjustment category, so the importance degree obtaining step is used to obtain the importance degree of the adjustment category. Therefore, the adjustment category corresponds to the importance in a one-to-one manner.

In some embodiments, the deep learning model training system 100 and the deep learning model training method can determine which adjustment category is used to train the first deep learning model T1 according to the importance of each adjustment category. For example, rotate the angle with the least importance as the adjustment category. In some embodiments, the deep learning model training system 100 and the deep learning model training method can train the first deep learning model T1 in each adjustment category in sequence according to the importance of each adjustment category.

In some embodiments, when the height of the original image is greater than the width of the original image, the importance of the adjustment category is arranged as follows: proportional scaling>angle rotation>contrast adjustment>brightness adjustment>gamma adjustment>height enlargement>width enlarge.

In some embodiments, when the height of the original image is smaller than the width of the original image, the importance of the adjustment category is arranged as follows: proportional scaling>angle rotation>contrast adjustment>brightness adjustment>gamma adjustment>width enlargement>height enlarge.

FIG. 3 is a flowchart of the steps of obtaining importance according to some embodiments of the present case. 3, in some embodiments, the flow of the importance degree obtaining step includes a range value obtaining step (step S310), a grouping step (step S320), an invalid boundary value obtaining step (step S330), and an importance degree determining step (step S340). ).

In some embodiments, the step of obtaining the range value (step S310 in FIG. 3) includes: obtaining the range value according to the difference between the maximum test value and the minimum test value. Specifically, the range value is the range of the test value, so the processor 120 can obtain the range value according to the difference between the maximum test value and the minimum test value. Taking the test value of angle rotation as an example and referring to Table 1, the maximum test value is "10" and the minimum test value is "-10", so the range value is "|10-(-10)|=20".

In some embodiments, the grouping step (step S320 in FIG. 3) includes: classifying the test value into an invalid group or a valid group according to whether the confidence level corresponding to each test value is less than a threshold value. The grouping step (step S320 in FIG. 3) is similar to the grouping step (step S230 in FIG. 2), which will not be repeated here. The difference between step S320 and step S230 is that the thresholds of the two can be the same (for example, the thresholds are both Is "0.5"), or it can be different (for example, the threshold of step S320 can be "0.6", and the threshold of step S230 can be "0.5"). Take the test value of angle rotation as an example and refer to Table 2. If the threshold value is "0.5", the confidence level is less than "0.5" and the test values classified into the invalid group are "-10, -9.5, -9, -8.5,- 8, -7.5, -7, -6.5, -6, -5.5, -5, -4.5, -4, -3.5, -3, 3, 3.5, 4, 4.5, 5, 5.5, 6, 6.5, 7, 7.5, 8, 8.5, 9, 9.5, 10". When the confidence level is greater than or equal to "0.5", the test value classified in the effective group is "-2.5, -2, -1.5, -1, -0.5, 0, 0.5, 1, 1.5, 2, 2.5".

In some embodiments, the invalid boundary value obtaining step (step S330 in FIG. 3) includes: obtaining the invalid boundary value according to the smallest difference between the test value in the invalid group and the original value. Specifically, the processor 120 can compare the difference between the test value in the invalid group and the original value, obtain respective differences between the test values in the invalid group and the original values, and compare the differences between the test values and the original values. The size obtains the smallest difference from it, and then obtains the invalid boundary value according to the test value corresponding to the smallest difference, that is, the invalid boundary value is the test value corresponding to the smallest difference. It should be noted that the difference between the test value and the original value is "|original value-test value|", that is, each difference is positive. The invalid boundary value represents the threshold at which the test value cannot be correctly identified by the first deep learning module T0. In other words, the invalid boundary value is the boundary between the invalid group and the valid group, and the invalid boundary value is the test value in the invalid group . In some embodiments, taking the test value of angle rotation as an example and referring to Table 2, if the original value is "0", the difference between the test value in the invalid group and the original value is "3, 3.5, 4, 4.5, 5, 5.5, 6, 6.5, 7, 7.5, 8, 8.5, 9, 9.5, 10", so the smallest difference is "3", and the smallest difference corresponds to a test value of "3" or "-3" ", and "3" or "-3" are invalid boundary values. It should be noted that using "3" or "-3" as an invalid boundary value does not affect the importance, because the difference between "3" and "-3" and the original value is the same (both are the smallest differences) Value "3").

In some embodiments, the importance determination step (step S340 in FIG. 3) includes: obtaining the importance according to the difference between the original value and the invalid boundary value divided by the range value. Specifically, the processor 120 can obtain the quotient according to the difference between the original value and the invalid boundary value divided by the range value, and the quotient is the importance. The original value is the value of the original image set corresponding to the adjustment category, and the importance is the difference between the original value and the invalid boundary value divided by the range value, that is, the importance corresponds to the difference between the valid group (invalid group) and the range value Proportion. Therefore, when the importance is lower, the test value representing the valid group accounts for the lower proportion of all test values (the test value of the invalid group accounts for the higher proportion of all test values), that is, the adjustment category is more likely to affect the confidence level . Conversely, when the importance is higher, the test value representing the effective group accounts for the higher proportion of all test values (the test value of the invalid group accounts for the lower proportion of all test values), which means that the adjustment category is less likely to affect confidence Spend. Take the test value of angular rotation as an example and refer to Table 2. Take the test value of angular rotation as an example and refer to Table 2. The original value is "0", the difference between the invalid boundary value and the original value is "3", the range The value is "20" and the quotient is "3/20=0.15", so the importance is "0.15".

In some embodiments (for example, Table 2), the importance determination step (step S340 in FIG. 3) can be expressed by a first importance obtaining formula, and the first importance obtaining formula is as follows:

是代表在

與

之中取最小值，B₁ 、B₂ 是無效邊界值，R為範圍值。Among them, I is the importance, O is the original value,

Is on behalf of

and

Take the smallest value among them, B ₁ and B ₂ are invalid boundary values, and R is the range value.

In some embodiments (for example, Table 3), the importance determination step (step S340 in FIG. 3) can be expressed by a second importance obtaining formula, and the second importance obtaining formula is as follows:

Among them, I is the importance, O is the original value, B is the invalid boundary value, and R is the range value.

FIG. 4 is a schematic diagram of a deep learning model training system 100 according to some embodiments of the present application. Please refer to FIG. 4, in some embodiments, the deep learning model training system 100 can be used with a meter 400 and an image capturing device 500. The meter 400 is used to display values, and the image capturing device 500 is used to capture original images from the meter 400, such as capturing "the area where the meter 400 is used to display values" or "the display screen of the meter 400" as the original image. The deep learning model training system 100 then obtains the original image from the image capturing device 500, so the deep learning model training system 100 can obtain the original image set (ie, multiple original images). In some embodiments, the image capturing device 500 is connected to the deep learning model training system 100 via a wireless network. The meter 400 is, for example, but not limited to, mechanically rotating digital or electronic display values, and the meter 400 is, for example, but not limited to, used in a power transmission network or a tap water transmission network.

In some embodiments, in addition to the processor 120 and the storage device 140, the deep learning model training system 100 can further include a meter 400 and an image capture device 500 (the deep learning model is not shown in the figure). The training system 100 can further include a meter 400 and an image capturing device 500).

FIG. 5 is a partial schematic diagram of a meter 400 drawn according to some embodiments of the present application. Referring to FIG. 5, in some embodiments, the area used for displaying the value of the meter 400 can display 4 decimal digits, that is, the value displayed by the meter 400 is between "0000 and 9999". Taking Figure 5 as an example, the value displayed on the meter 400 is "0011".

In some embodiments, the architecture of the deep learning model used by the deep learning model training system 100 and the deep learning model training method is, for example, but not limited to, Convolutional Neural Network (CNN) and Recurrent Neural Network (Recurrent Neural Network, RNN), Deep Neural Network (DNN), or yolo (You Only Look Once). In other words, the deep learning model training system 100 and the deep learning model training method are, for example, but not limited to, obtaining the confidence level based on the above-listed architecture.

FIG. 6 is a schematic diagram of the object OB and the frame according to some embodiments of the present application. FIG. 7 is a schematic diagram of a combined ratio IOU according to some embodiments of the present case. 6 and 7, in some embodiments, the image IM input to the deep learning model has an object OB, and the deep learning model sets the first frame A1 and the second frame A2 according to the position of the object OB in the image IM. . The deep learning model obtains the intersection area A3 and the union area A4 according to the first frame A1 and the second frame A2, where the intersection area A3 is the intersection area between the first frame A1 and the second frame A2, the union area A4 is the combined area between the first frame A1 and the second frame A2. The deep learning model obtains the confidence of the image IM according to the image IM, the first frame A1 and the second frame A2. The formula for obtaining the confidence of the deep learning model is for example but not limited to the following:

C=Pr(ob)*IOU, IOU=A3/A4.

Among them, C is the confidence level, Pr(ob) is the probability that the object OB belongs to a category (take Figure 6 as an example, the category can be "0", so Pr(ob) is the probability that the object OB belongs to "0"), and And IOU is the intersection area A3 divided by the union area A4.

In some embodiments, specifically, the deep learning model divides the image IM into a plurality of array blocks (a total of S*S array blocks), calculates the probability that the object OB in each array block belongs to a category, and The maximum value is obtained from the probability of the object OB belonging to a category in each array block, and the maximum value is used as the probability of the object OB belonging to a category. The first frame A1 is a reference standard block (ground truth box), which is a block in which the deep learning model marks the object OB as a standard answer based on the results of previous training. The second frame A2 is a candidate block (bounding box), that is, the deep learning model marks whether the predicted object OB in the image IM belongs to a category according to the input image IM.

In summary, the deep learning model training system, deep learning model training method, and non-transitory computer-readable storage media of some embodiments of this case can adjust the original image set according to the test value to obtain the test image set, and input the test image set to The first deep learning model is tested to obtain the confidence of the test image set, that is, the confidence of the test value. And according to the threshold value, the test value can be classified into the valid group or the invalid group, and then the training value can be obtained from the test value in the invalid group. And adjust the original image set according to the training value to obtain the training image set, and input the original image set and the training image set to the first deep learning model for training to obtain the second deep learning model. In some embodiments, the deep learning model training system and method can use the training image set to strengthen the training of the first deep learning model without training all the test values corresponding to the confidence level below the threshold, so it can achieve The effect of simplifying the amount of training data of the deep learning model, and still obtaining a second deep learning model with high confidence. In some embodiments, the deep learning model training system and method can be adapted to identify the image of the area where the meter is used to display the value.

100: Deep learning model training system 120: processor 140: storage device 400: Meter 500: Image capture device T1: The first deep learning model T2: The second deep learning model IM: Image OB: Object A1: The first frame A2: The second frame A3: Intersection area A4: Union area S210-S260: steps S310-S340: steps

Fig. 1 is a schematic diagram of a deep learning model training system according to some embodiments of the present application. Fig. 2 is a flowchart of a deep learning model training method according to some embodiments of the present case. FIG. 3 is a flowchart of the steps of obtaining importance according to some embodiments of the present case. FIG. 4 is a schematic diagram of a deep learning model training system according to some embodiments of the present application. Fig. 5 is a partial schematic diagram of a meter according to some embodiments of the present application. Fig. 6 is a schematic diagram of objects and frames drawn according to some embodiments of the present case. FIG. 7 is a schematic diagram of the combined comparison according to some embodiments of the present case.

S210-S260: steps

Claims (15)

A deep learning model training method, including: A test image set obtaining step, adjusting an original image set according to a plurality of test values to obtain a plurality of test image sets; A model testing step: input the test image sets to a first deep learning model for testing to obtain a plurality of confidence levels, and the confidence levels correspond to the test values in a one-to-one manner; In a grouping step, the test values are classified into an invalid group or an effective group according to whether the confidence corresponding to each test value is less than a threshold; A training value obtaining step, obtaining a plurality of training values according to the test values in the invalid group; A training image set obtaining step, adjusting the original image set according to the training values to obtain a plurality of training image sets; and A model training step is to input the original image set and the training image sets to the first deep learning model for training to obtain a second deep learning model. The deep learning model training method according to claim 1, wherein the step of obtaining the training value further includes: Obtain an effective interval value according to the difference between the largest test value in the effective group and an original value; and According to multiple integer multiples of the valid interval value, the training values are obtained from the test values in the invalid group. The deep learning model training method according to claim 1, wherein the test image sets correspond to the test values in a one-to-one manner, and the training image sets correspond to the training values in a one-to-one manner. The deep learning model training method according to claim 1, wherein the test values are an arithmetic sequence. The deep learning model training method according to claim 1, wherein the test values include an original value corresponding to the original image set. The deep learning model training method according to claim 5, wherein the test image set corresponding to the original value is part of the original image set. The deep learning model training method according to claim 1, further includes an importance degree obtaining step, the importance degree obtaining step is used to obtain an importance degree corresponding to the test values, wherein the importance degree obtaining step includes: Obtain a range value according to the difference between the largest test value and the smallest test value; According to whether the confidence level corresponding to each test value is less than the threshold value, classify the test values into the invalid group or the valid group; Obtaining an invalid boundary value according to the smallest difference between the test values in the invalid group and an original value; and The importance is obtained according to the difference between the original value and the invalid boundary value divided by the range value. A deep learning model training system, including: A processor for receiving an original image set, a plurality of test values, and a threshold value, adjust the original image set according to the test values to obtain a plurality of test image sets, and input the test image sets to a first depth The learning model is tested to obtain multiple confidences, the confidences correspond to the test values in a one-to-one manner, and the test values are classified into one according to whether the confidence corresponding to each test value is less than the threshold Invalid group or a valid group, obtain multiple training values according to the test values in the invalid group, adjust the original image set according to the training values to obtain multiple training image sets, and input the original image set and the Training the image set to the first deep learning model for training to obtain a second deep learning model; and A storage device is used to store the first deep learning model and the second deep learning model. The deep learning model training system according to claim 8, wherein the processor obtains an effective interval value according to the difference between the largest test value in the effective group and an original value, and according to the number of effective interval values Integer multiples obtain the training values from the test values in the invalid group. The deep learning model training system according to claim 8, wherein the test image sets correspond to the test values in a one-to-one manner, and the training image sets correspond to the training values in a one-to-one manner. The deep learning model training system according to claim 8, wherein the test values are an arithmetic sequence. The deep learning model training system according to claim 8, wherein the test values include an original value corresponding to the original image set. The deep learning model training system according to claim 12, wherein the test image set corresponding to the original value is part of the original image set. For the deep learning model training system according to claim 8, the processor obtains a range value according to the difference between the largest test value and the smallest test value, and whether the confidence level corresponding to each test value is less than The threshold classifies the test values into the invalid group or the valid group, obtains an invalid boundary value according to the smallest difference between the test values in the invalid group and an original value, and obtains an invalid boundary value according to the original value and the original value. The difference between the invalid boundary values is divided by the range value to obtain the importance. A non-transitory computer-readable storage medium for storing one or more software programs, the one or more software programs including a plurality of instructions, when the instructions are executed by one or more processing circuits of an electronic device , The electronic device will be made to perform a deep learning model training method, and the deep learning model training method includes: A step of obtaining a test image set, adjusting an original image set to a plurality of test image sets according to a plurality of test values; A model testing step: input the test image sets to a first deep learning model for testing to obtain a plurality of confidence levels, and the confidence levels correspond to the test values in a one-to-one manner; In a grouping step, the test values are classified into an invalid group or an effective group according to whether the confidence corresponding to each test value is less than a threshold; A training value obtaining step, obtaining a plurality of training values according to the test values in the invalid group; A training image set obtaining step, adjusting the original image set according to the training values to obtain a plurality of training image sets; and A model training step is to input the original image set and the training image sets to the first deep learning model for training to obtain a second deep learning model.

Images