TWI734186B - Training image generating method and electronic device - Google Patents

Abstract

A training image generation method and an electronic device are provided. The method includes: obtaining a first image including a specific object; removing a portion of the first image that does not belong to the specific object to obtain a second image; determining a first location in a target image; applying the specific object in the second image to the target image to generate a training image according to the first location; and training a model by using the training image.

Description

Training image generation method and electronic device

The invention relates to a training image generation method and electronic device.

The Advanced Driver Assistant System (ADAS) uses a variety of sensors installed in the car (for example, it can detect variables such as light, heat, and pressure) to collect the inside and outside of the car at the first time The environmental data is processed by the identification, detection and tracking of static and dynamic objects, so that the driver can be aware of possible dangers in the fastest time, so as to attract the attention of the driver and improve the safety of the initiative safety technology.

The sensors used in ADAS are mainly cameras, radars, lasers and ultrasonics, etc., which can detect light, heat, pressure or other variables used to monitor the state of the car. The aforementioned sensors are usually located on the front and rear bumpers, rear mirrors, inside the steering column or on the windshield of the vehicle. Among them, the camera can obtain the image of the scene around the vehicle, and the obtained image can usually be used for target detection. For example, detecting objects such as vehicles, pedestrians, traffic signs and so on ahead. Currently, technologies such as deep learning algorithms, machine learning algorithms, or convolutional neural networks are often used to detect targets.

When using convolutional neural networks for target detection, a large number of training images are usually needed for training, but the number of traffic signs in the data collected by the camera (ie, the data of actual road conditions) is much smaller than that of vehicles or pedestrians. . This is because on the road, the number of traffic signs is relatively small relative to objects such as vehicles or pedestrians, which causes data imbalance. When encountering data imbalance, the traditional classification algorithm that takes the overall classification accuracy as the learning objective will pay too much attention to the majority of objects, so that the performance of the classification of the minority objects is reduced.

The present invention provides a training image generation method and electronic device, which can increase the number of a specific object (for example, a traffic sign) in the training image, so that after the training image is used to train a model for recognizing the specific object, it can achieve higher results. The accuracy rate of forecasting, and avoid the problems caused by data imbalance.

The present invention provides a method for generating training images for use in an electronic device. The method includes: obtaining a first image containing a specific object; removing parts of the first image that do not belong to the specific object to obtain a corresponding A second image of the specific object; determining a first position in a target image; according to the first position, pasting the specific object in the second image to the target image to generate a Training images; and using the training images to train a model for recognizing the specific object.

The present invention provides an electronic device including: a processor. The processor is configured to perform the following operations: obtain a first image including a specific object; remove parts of the first image that do not belong to the specific object to obtain a second image corresponding to the specific object; and determine a A first position in the target image; according to the first position, paste the specific object in the second image to the target image to generate a training image; and use the training image for training Identify a model of the specific object.

Based on the foregoing, the training image generation method and electronic device of the present invention can increase the number of a specific object (for example, a traffic sign) in the training image, so that after the training image is used to train a model for identifying the specific object, there may be Higher prediction accuracy rate, and avoid the problems caused by data imbalance.

In order to make the above-mentioned features and advantages of the present invention more comprehensible, the following specific embodiments are described in detail in conjunction with the accompanying drawings.

Now referring to the exemplary embodiments of the present invention in detail, examples of the exemplary embodiments are illustrated in the accompanying drawings. In addition, wherever possible, elements/components with the same reference numbers in the drawings and embodiments represent the same or similar parts.

FIG. 1 is a block diagram of an electronic device according to an embodiment of the invention.

Please refer to FIG. 1, the electronic device 100 includes a processor 20, an input/output circuit 22 and a storage circuit 24. Among them, the input/output circuit 22 and the storage circuit 24 are respectively coupled to the processor 20. The electronic device 100 is, for example, an electronic mobile device such as a desktop computer, a server, a mobile phone, a tablet computer, a notebook computer, etc., which is not limited herein.

The processor 20 may be a central processing unit (Central Processing Unit, CPU), or other programmable general-purpose or special-purpose microprocessors (Microprocessor), digital signal processors (Digital Signal Processor, DSP), programmable Integrated circuit (Application Specific Integrated Circuit, ASIC) or other similar components or a combination of the above components.

The input/output circuit 22 is, for example, an input interface or circuit for obtaining relevant data from outside the electronic device 100 or other sources. In addition, the input/output circuit 22 can also transmit the data generated by the electronic device 100 to the output interface or circuit of other electronic devices, which is not limited herein.

The storage circuit 24 can be any type of fixed or removable random access memory (RAM), read-only memory (ROM), flash memory (flash memory) or similar components Or a combination of the above elements.

In this exemplary embodiment, a plurality of code fragments are stored in the storage circuit 24 of the electronic device 100, and after the above-mentioned code fragments are installed, they will be executed by the processor 20. For example, the storage circuit 24 includes a plurality of modules, and each operation of the training image generation method applied to the electronic device 100 is executed by these modules, and each module is composed of one or more code fragments. However, the present invention is not limited to this, and various operations of the electronic device 100 may also be implemented in other hardware forms.

In the Advanced Driver Assistance System (ADAS), a model trained to recognize a specific object (for example, a traffic sign or a vehicle) in an image can be configured. Generally speaking, the training images used to train the aforementioned models are usually images collected from a driving recorder. However, in actual scenes, the number of vehicles is usually much larger than the number of traffic signs, which leads to the problem of unbalanced training data. In the case of fewer traffic signs in the training image, the model training result of the convolutional neural network will have a better recognition rate for the vehicle, but the accuracy of the traffic signs is indeed very low.

Therefore, the present invention proposes a training image generation method to increase the number of a specific object (for example, a traffic sign) in the training image, so that after the training image is used to train a model for recognizing the specific object, the model can have Higher accuracy rate.

The following is an example of a model trained to recognize a traffic sign in an image. In particular, in order to improve the prediction accuracy of the model, it is necessary to increase the number of traffic signs in the training image.

FIG. 2 is a schematic diagram of a first image drawn according to an embodiment of the invention. FIG. 3 is a schematic diagram of a target image drawn according to an embodiment of the invention.

Please refer to FIGS. 2 and 3 at the same time. Here, it is assumed that the processor 20 intends to increase the number of traffic signs in the target image 32 to generate training images. First, the processor 20 may obtain an image 30 (also referred to as a first image) containing a traffic sign through the input/output circuit 22, for example. For example, an image 30 containing traffic signs can be extracted from an original image through a tagging tool software. Alternatively, the image 30 can also be manually selected from the original image by the user, which is not limited here.

In this embodiment, when the image 30 is larger than the target image 32, the processor 20 will (for example, randomly) determine a reduction magnification, and reduce the image 30 according to the reduction magnification so that the size of the image 30 is smaller than that of the target image 32. size. In addition, when the image 30 is not larger than the target image 32, the processor 20 may not reduce the image 30.

In detail, it is assumed here that the image 30 has a width w ₀ and a height h ₀ , and the target image 32 has a width w and a height h. When the width w _{0 is} greater than the width w or the height h _{0 is} greater than the height h, the processor 20 will determine that the image 30 is larger than the target image 32. At this time, the processor 20 determines a reduction magnification, and reduces the image 30 according to the reduction magnification so that the size of the image 30 is smaller than the size of the target image 32.

Then, the processor 20 removes parts of the image 30 that are not traffic signs to obtain an image corresponding to the traffic sign (hereinafter referred to as the second image). For example, the processor 20 may multiply the image 30 by a mask matrix to obtain the aforementioned second image.

In more detail, FIGS. 4A and 4B are schematic diagrams of using a mask to remove parts of an image that are not traffic signs according to an embodiment of the present invention.

-------------------------公式(1)First, please refer to FIG. 4A. First, the processor 20 can determine a mask matrix 40 according to the size of the aforementioned image 30. For example, the processor 20 adjusts the size of the mask matrix 40 to match the size of the image 30 (or the reduced image 30). In this embodiment, the mask matrix 40 includes (2*r ₀ +1) columns and (2*r ₀ +1) rows, and r ₀ is a non-zero positive integer. In this embodiment, r ₀ is the radius of the traffic sign in the image 30. In addition, the processor 20 uses the following formula (1) to set the value of the i-th column and the j-th row in the mask matrix 40:

-------------------------Formula 1)

That is, when (ir ₀ ) ² +(jr ₀ ) ^{2 is} less than r ₀ ² , the processor 20 will set the value of the i-th column and the j-th row in the mask matrix 40 to 1. When (ir ₀ ) ² +(jr ₀ ) ^{2 is} not less than r ₀ ² , the processor 20 will set the value of the i-th column and the j-th row in the mask matrix 40 to 0. Among them, i and j are respectively positive integers greater than zero and less than or equal to (2*r ₀ +1).

After the mask matrix 40 is determined, referring to FIG. 4B, the processor 20 multiplies the image 30 by the mask matrix 40 to obtain the image 60. Here, the image 60 is the aforementioned second image. In particular, only the traffic signs are left in the image 60 and the parts that are not traffic signs have been removed.

After that, FIG. 5 is a schematic diagram of generating training images according to an embodiment of the present invention.

(亦稱為，第一位置)。處理器20會根據此第一位置，將影像60中的交通標誌(不含背景部份)貼至目標影像32中以產生一訓練影像90。需注意的是，前述的

需要滿足

，藉此避免影像60超越目標影像32的邊界。其中，s為前述的縮小倍率。Referring to FIG. 5, the processor 20 determines (for example, randomly) a position in the target image 32

(Also known as the first position). The processor 20 will paste the traffic sign (excluding the background part) in the image 60 to the target image 32 according to the first position to generate a training image 90. It should be noted that the aforementioned

Need to meet

This prevents the image 60 from exceeding the boundary of the target image 32. Among them, s is the aforementioned reduction magnification.

After that, the processor 20 can reselect the image and repeat the above steps to generate more traffic signs in the training image 90. After the training image 90 is generated, the processor 20 can use the training image 90 to train a model for recognizing traffic signs.

in training set 1.       Random choose a fake traffic sigin image as

2.       Get

‘s width and height

and

‘s width and height

3.       If

or

: Random set

which satisfy

and

else Set

4.       resize

with scale

5.       resize

with the same shape as

6.       Random choose

‘s position in

as

7.

8.       paste

to

: for

for

if

9.       Repeat step 1 to 8 to fake more traffic signIn particular, the aforementioned process of generating training images can be simply expressed in pseudo code as follows: For train image -

in training set 1. Random choose a fake traffic sigin image as

2. Get

's width and height

and

's width and height

3. If

or

: Random set

which satisfy

and

else Set

4. resize

with scale

5. resize

with the same shape as

6. Random choose

's position in

as

7.

8. paste

to

: for

for

if

9. Repeat step 1 to 8 to fake more traffic sign

FIG. 6 is a schematic diagram of a training image generation method according to an embodiment of the invention.

Referring to FIG. 6, in step S601, the processor 20 obtains a first image containing a specific object. In step S603, the processor 20 removes the part of the first image that does not belong to the specific object to obtain a second image corresponding to the specific object. In step S605, the processor 20 determines the first position in the target image. In step S607, the processor 20 pastes the specific object in the second image to the target image according to the first position to generate a training image. Finally, in step S609, the processor 20 uses the training images to train a model for identifying specific objects.

In summary, the training image generation method and electronic device of the present invention can increase the number of a specific object (for example, a traffic sign) in the training image, so that after the training image is used to train a model for recognizing the specific object, Have a higher prediction accuracy rate.

Although the present invention has been disclosed in the above embodiments, it is not intended to limit the present invention. Anyone with ordinary knowledge in the relevant technical field can make some changes and modifications without departing from the spirit and scope of the present invention. The protection scope of the present invention shall be subject to those defined by the attached patent application scope.

100: electronic device 20: processor 22: input and output circuit 24: storage circuit w, w ₀ : width h, h ₀ : height 30, 60: image 32: target image 40: matrix 90: training image S601~S609: step

FIG. 1 is a block diagram of an electronic device according to an embodiment of the invention. FIG. 2 is a schematic diagram of a first image drawn according to an embodiment of the invention. FIG. 3 is a schematic diagram of a target image drawn according to an embodiment of the invention. 4A and 4B are schematic diagrams of using a mask to remove parts of an image that are not traffic signs according to an embodiment of the present invention. FIG. 5 is a schematic diagram of generating training images according to an embodiment of the present invention. FIG. 6 is a schematic diagram of a training image generation method according to an embodiment of the invention.

S601~S609: steps

Claims (10)

A method for generating training images for an electronic device, the method comprising: obtaining a first image containing a specific object; removing parts of the first image that are not part of the specific object to obtain a corresponding portion of the specific object. A second image of the object, wherein the second image is obtained by multiplying the first image by a mask matrix; determining a first position in a target image; according to the first position, Paste the specific object in the second image to the target image to generate a training image; and use the training image to train a model for recognizing the specific object, wherein when the first image is larger than When determining the target image, a reduction magnification is determined and the first image is reduced according to the reduction magnification so that the size of the first image is smaller than the size of the target image; and the size of the mask matrix is adjusted To meet the size of the reduced first image. The training image generation method described in claim 1, wherein the step of obtaining the first image containing the specific object includes: using an annotation tool to extract all the images containing the specific object from an original image The first image. The training image generation method according to the first item of the patent application, wherein before the step of multiplying the first image by the mask matrix, the method further includes: determining the mask matrix, wherein the mask matrix The mask matrix includes (2*r ₀ +1) columns and (2*r ₀ +1) rows, and r ₀ is a non-zero positive integer. As described in item 3 of the scope of patent application, the training image generation method further includes: when (ir ₀ ) ² +(jr ₀ ) ^{2 is} less than r ₀ ² The value is set to 1; and when (ir ₀ ) ² +(jr ₀ ) ^{2 is} not less than r ₀ ² , the value of the i-th column and j-th row in the mask matrix is set to 0. According to the training image generation method described in item 1 of the scope of patent application, the specific object is a traffic sign. An electronic device comprising: a processor, wherein the processor obtains a first image including a specific object, and the processor removes a part of the first image that does not belong to the specific object to obtain a corresponding A second image of the specific object, wherein the second image is obtained by multiplying the first image by a mask matrix, the processor determines a first position in a target image, the The processor pastes the specific object in the second image to the target image according to the first position to generate a training image, and the processor uses the training image to train for recognizing the specific A model of an object, wherein when the first image is larger than the target image, The processor determines a reduction magnification and reduces the first image according to the reduction magnification so that the size of the first image is smaller than the size of the target image, and the processor reduces the size of the mask matrix Adjusted to fit the size of the reduced first image. The electronic device described in claim 6, wherein in the operation of obtaining the first image containing the specific object, the processor uses a tagging tool to extract from an original image that contains the specific object. The first image of the object. The electronic device according to claim 6, wherein before the operation of multiplying the first image by the mask matrix, the processor determines the mask matrix, wherein the mask matrix includes (2*r ₀ +1) column and (2*r ₀ +1) row, r ₀ is a non-zero positive integer. The electronic device described in item 8 of the scope of patent application, wherein when (ir ₀ ) ² + (jr ₀ ) ^{2 is} less than r ₀ ² , the processor will The value is set to 1, and when (ir ₀ ) ² + (jr ₀ ) ^{2 is} not less than r ₀ ² , the processor sets the value of the i-th column and the j-th row in the mask matrix to 0. The electronic device described in item 6 of the scope of patent application, wherein the specific object is a traffic sign.

Images