TWI783239B - Method of optimizing image data - Google Patents

Abstract

This invention discloses a method of optimizing image data including: marking objects in an image to calculate the ratio of the marked objects to the image; generating a comparison result; processing the image according to the comparison result; and then inputting the processed image into a neural network-like model; wherein if the comparison result is that the ratio is less than a threshold, the object is filtered from the image, otherwise, it is not filtered. The present invention uses an object threshold setting to improve the detection effect, which is suitable for setting a threshold according to different proportions of moving objects of an image in a multimedia image stream of different sizes for optimizing image training data.

Description

Method of Optimizing Image Data

The present invention relates to a method for optimizing image data, in particular, relates to a method for setting a size threshold of a moving object to optimize a training data set.

Supervised learning of artificial intelligence needs to collect data and label it for learning. Among the large amount of labeled data, not all data are beneficial to the training of neural networks, and even the detection effect may be caused by the diversity of data. decreased. In the past, in the training of similar neural networks, due to its black box characteristics, the characteristics of the data were ignored during training, so that all the data were put into the training. Therefore, how to screen out more excellent data from a large amount of data so that the model trained by the neural network is closer to the ideal is the goal conceived by those skilled in the art.

In addition, due to the different proportions of objects on the screen, when generating data sets, objects of the same type will collect data of various proportions, resulting in erratic size of detected objects, and there may be cases where too small moving objects cannot be detected question.

One purpose of the present invention is to provide a method for setting the size threshold of moving objects according to the ratio value of the object in the image (this article may be referred to as the ratio), so as to optimize the deep learning training set, which can Improve the accuracy rate and solve the problem of not being able to detect too small moving objects in the past. Another object of the present invention is to provide a training data set for optimizing moving objects, which can provide the proportion of the object in the image for reference and correction by the neural network, and solve the problem that the accuracy of the current model is difficult to further improve.

Therefore, the present invention provides a method for optimizing image data, which includes: marking the object in the image; calculating the ratio of the object to the image; comparing the ratio with a preset threshold to generate a comparison result; and processing according to the comparison result the image.

In addition, after obtaining a large amount of training data, the present invention utilizes the ratio value of the object and the image, and standardizes that the size of the object in this image should be greater than the threshold before it is considered a usable sample, and then puts the filtered training data into the neural network The road is trained so that it improves its model accuracy and meets the desired requirements.

Furthermore, the image is divided into general image and fisheye image. After collecting a large number of marked images, the general image is screened using the ratio value of the frame to the object, while the fisheye image is obtained by obtaining the center point and its largest circle, and Find the radius of the largest circle, divide the radius value into four equal parts, and divide it into four concentric circles. According to the ideal size of each type of object in each concentric circle and its distance from the center point, the object's position in the image is obtained. Ideal scale value.

S101, S102, S103a, S103b, S104, S105a, S105b, S106: steps

Please refer to the detailed description of the present invention and accompanying drawings, will further understand the technical content and purpose effect of the present invention; The relevant drawings are:

Fig. 1 is a schematic flowchart of the method for optimizing image data of the present invention.

FIG. 2 is a schematic diagram of an embodiment of the method for optimizing image data of the present invention, wherein objects are marked in the fisheye image.

FIG. 3 is a schematic diagram of an embodiment of the method for optimizing image data of the present invention, wherein the influence of a fixed threshold is shown.

FIG. 4 is a schematic diagram of an embodiment of the method for optimizing image data of the present invention, wherein the influence of the adaptive threshold is shown.

Please refer to FIG. 1 , in step S101 , mark moving objects of all images using a mark tool. The labeling information of the moving object includes the category, ( x 1, y 1) coordinates and ( x 2, y 2) coordinates of the object in the image. After obtaining the above labeling information, the covered area of the object can be calculated. As shown in Figure 2, there are multiple bounding boxes in the image. Except for the object in the blue bounding box to the left of the center of the fisheye image is a motorcycle, the objects in the other bounding boxes The category is a small car, and each bounding box has its own parameters such as center coordinates, rectangle length and width, and so on.

In step S102, it is determined whether the image is a normal image or a fisheye image according to different lenses. Since the output image size of each camera is different (for example: 1080p, 720p), it is not possible to filter only by the area covered by the object. As shown in Figure 3, if a fixed threshold is used for filtering, the identifiable objects in 720p will be filtered, or 1080p will retain the objects that originally need to be filtered. Therefore, after using the ratio method and setting the threshold, as shown in Figure 4, small objects can be filtered out more effectively at any input image size, which can optimize the data set to achieve better training results. In addition, the threshold is set by dividing the default object size in the image/image size (that is, dividing the default object length*width in the image by the image resolution).

The operation of the ratio method is the general image and the image with deformation (for example: fisheye image), wherein, the general image here refers to the one shot by the general camera, and the deformed image refers to the one shot by the fisheye, or It can be judged manually, the image type is pre-set by the system, or it can be judged automatically according to its characteristics. In step S103b, when it is a general image, the ratio of the same object to the entire image is calculated as formula (1). In step S104, the calculated ratio is compared with the threshold value. In one embodiment, the threshold is set to 0.0003 to 0.001, and all objects in the data set are filtered with these thresholds. For example, when the object ratio is greater than the threshold, it means no filtering, as in step S105a; otherwise, it is filtered, as in step S105b. It is better to use the filtered dataset for training and compare the results with which threshold. Different types of objects can set different thresholds. Different objects set different thresholds. For example, the length and width of pedestrians, motorcycles, small cars and large cars in the image are different. Among them, comparing the input images of two resolutions (such as 720p, 1080p), if a fixed threshold is used for filtering, the 720P will be recognizable 1080p retains the objects that need to be filtered (as shown in Figure 3).

Therefore, it is calculated whether the object ratio meets the threshold value, and if it is less than the threshold value, it is regarded as the background, and the image is re-labeled. For example, in Figure 4, assuming the threshold is 0.0006, the ratio of 1080P image objects is greater than the threshold and is retained, and the threshold is set according to the image ratio. Assuming that the threshold of 720P pedestrians in Figure 4 is 0.001, the pedestrians in the upper image above 720P Length*Width/(1280*720) is greater than 0.0005, so the pedestrians in the yellow frame are reserved. Assume that the threshold of the 720P small car in Figure 4 is 0.0005, and the small car length*width/(1280* below the 720P in Figure 4 720) is also greater than 0.0005, so the small cars in the yellow frame remain marked.

Formula (1) is as follows:

Threshold = default object size ( ObjectSize ) / image size ( VideoSize )

Ratio ε = actual object size ( ObjectSize ) / image size ( VideoSize )

閾值(threshold)，不過濾(reject object) Ratio ε

Threshold ( threshold ), do not filter ( reject object )

Ratio ε < threshold ( threshold ), filter ( reserve object )

In step 103a, when it is a fisheye image, the thresholds used are also different because the proportions of the images displayed by each category are different. Obtain the center point of the fisheye image and its maximum circle through the coordinates, and calculate the radius of the maximum circle. Divide the radius value into several equal parts, and divide it into multiple concentric circles. According to the ideal size of each type of object in each concentric circle and Its distance from the center point yields the ideal value of the object's aspect ratio in this image. It should be noted that on the fisheye image, the size of the object changes with the distance from the center point, and the closer the distance to the center point, the larger the size of the object, and the radius of the concentric circle is the distance from the center point, so the concentric circle can be used The radius of the circle shows the size of the object, and obtains the size of various objects at different distances, and the calculated size of the object is the ideal value. According to this method, the ratio obtained by each different distance is used to filter the data set, and the results are compared to find out the best result for objects whose distance should be filtered. As shown in step S105a, if the ratio is greater than or equal to threshold, then no filtering is performed, otherwise, as shown in step S105b, the object will be filtered and regarded as the background. Objects of different classes have different sizes in different concentric circles. Suppose there are two concentric circles, one with a distance of 10(arb.unit) from the center and the other with a distance of 15(arb.unit) from the center point. In the case of the same object, the size of the object measured at a distance of 10 is 150, Measure the size of the object at a distance of 15 to 60, and the ratio is distance/object length*width. Use this ratio as the threshold setting to determine which is the best ideal ratio, and the measured ratio will be different according to different categories. Assume that the size of the small car in the red frame is 150, assume that the intersection point of the crosshairs is the center point, and the distance from the small car in the red frame is 10, and assume that the size of the motorcycle in the blue frame is 60, use the ratio: distance/object length * width to do the calculation. As explained above, the general image directly sets the threshold value from 0.0003 to 0.001, while the fisheye is made in a concentric circle. Then find out which threshold corresponds to the category with the best results.

Table 1 is the experimental results of an embodiment of the general image. In Table 1, the first column is the threshold value, and the second to sixth columns are the average precision (AP (average precision) ) values, and the last column is the average value of all categories of AP.

Finally, in step S106, the filtered training data set is put into neural network training.

Therefore, the present invention can optimize the training data set by filtering objects. In general images and fisheye images, the calculation methods for object ratios are different. When manually selecting frames, the object size of each person frame is defined differently. Through this method, The size of each category is set to the most appropriate ratio, and this method can be used to input any image size, so that data and optimization can be achieved to achieve better training results and improve accuracy.

The method for optimizing the training data set according to the ratio value proposed by the present invention has the following advantages:

1. Since this method filters and optimizes the data set proportionally to moving objects in multimedia images, this method can be applied for optimization regardless of whether the output size of the image is different or the shape of the image is different.

2. Putting the optimized data set into a neural network for training, the accuracy rate that was previously difficult to improve has a clear upward trend.

The above detailed description is a concrete description of a feasible embodiment of the present invention, but this embodiment is not used to limit the patent scope of the present invention, and any equivalent implementation or change that does not depart from the technical spirit of the present invention shall include within the patent scope of this case. To sum up, this case is not only innovative in terms of technical thinking, but also can enhance the above-mentioned multiple functions compared with conventional products. It should fully meet the requirements of the statutory invention patent of novelty and advancement, and file an application in accordance with the law. I sincerely request your office to approve this document. Invention patent applications, to encourage inventions, to be grateful.

S101, S102, S103a, S103b, S104, S105a, S105b, S106: steps

Claims (9)

A method for optimizing image data using a computer, comprising: receiving an image from a camera for marking an object in the image; calculating the ratio of the object to the image by the computer; comparing the ratio with a preset threshold, generating a comparison result; and the computer processes the image according to the comparison result, wherein the processed image is a training data set to be input into a neural network-like model, wherein, if the image is a general image, the preset The threshold is the preset size of the object in the image divided by the size of the image, and different preset thresholds are set according to different types of objects, wherein the step of processing the image by the computer according to the comparison result includes: If the comparison result is that the ratio is smaller than the preset threshold, the object is filtered from the image, otherwise, the object is not filtered. The method described in item 1 of the scope of the patent application further includes: the computer inputs the processed image into the neural network model. The method described in item 1 of the scope of the patent application further includes: before calculating the ratio, the computer judges that the image is a normal image or a fisheye image. The method described in claim 3 of the patent application, wherein, if the image is judged to be a normal image, the computer divides the actual size of the object in the image by the size of the image to obtain the ratio. The method described in item 1 of the patent claims, wherein the preset threshold ranges from 0.0003 to 0.001. The method described in item 3 of the scope of the patent application, wherein, if the image is judged to be a fisheye image, the computer first obtains the center point of the fisheye image and its maximum circle to calculate the half of the maximum circle Then divide the largest circle into a plurality of concentric circles according to the radius, and then obtain the ratio according to the size of the object in each of the concentric circles and its distance from the center point. The method described in item 6 of the scope of patent application, wherein the size of the object on each of the concentric circles is related to the category of the object. The method described in item 1 of the scope of patent application, wherein, the step of receiving the image from the camera for labeling the object in the image includes: the computer obtains the category including the object and the object in the image Labeling information of the coordinates, wherein the labeling information is used for calculating the size of the labeled object. The method described in claim 1, wherein the object is a moving object.

Images