TWI824550B - Method for generating distorted image, electronic device and storage medium - Google Patents

Abstract

The present application provides a method for generating a distorted image, an electronic device and a storage medium. The method includes: acquiring an undistorted image; acquiring a plurality of first pixel coordinates and corresponding first pixel values of the undistorted image; selecting one first pixel coordinate from the first pixel coordinates as a distortion center coordinate; calculating a distance between the distortion center coordinate and each first pixel coordinate; calculating a plurality of second pixel coordinate; taking the first pixel value to be second pixel value of each second pixel coordinate; based on the plurality of second pixel coordinates and the second pixel values of the plurality of second pixel coordinates, generating a distorted image of the undistorted image. The present application can acquire distorted images with different distortion degrees.

Description

Distorted image generation method, electronic device and computer-readable storage medium

The present application relates to the field of image processing, and in particular to a distortion image generation method, electronic equipment and computer-readable storage media.

In daily life, people often use trained deep learning models to realize functions such as image recognition and segmentation. However, the images captured by the camera are distorted, and the degree of distortion varies. As a result, the trained deep learning model cannot accurately identify or segment the obtained distorted images, resulting in the failure of the deep learning model. Reduction in recognition or segmentation accuracy.

In view of the above, it is necessary to provide a distortion image generation method, electronic device and computer-readable storage medium to solve the problem of reduced accuracy of deep learning models caused by distorted images.

Embodiments of the present application provide a method for generating a distorted image. The method for generating a distorted image includes: acquiring an undistorted image; acquiring a plurality of first pixel coordinates of the undistorted image and corresponding to each first pixel coordinate. a first pixel value; and selecting a first pixel coordinate from the plurality of first pixel coordinates as the distortion center coordinate; calculating the distortion center coordinate according to the distortion center coordinate and the plurality of first pixel coordinates. The distance from each first pixel coordinate; obtaining at least one distortion coefficient, calculated based on the at least one distortion coefficient, the plurality of first pixel coordinates, and the distance between the distortion center coordinate and each first pixel coordinate. the second pixel coordinate corresponding to each first pixel coordinate; The first pixel value corresponding to each first pixel coordinate is used as the second pixel value corresponding to each second pixel coordinate; based on the plurality of second pixel coordinates and the third pixel value corresponding to each second pixel coordinate Two pixel values generate a distorted image of the undistorted image.

In an optional implementation, the method further includes: training a deep learning model using the undistorted image and the distorted image.

In an optional implementation, the method further includes: determining a distortion type based on the plurality of second pixel coordinates; and based on the distortion type, the second pixel coordinates, and a third value corresponding to the second pixel coordinates The two-pixel value generates a distortion image corresponding to the distortion type.

In an optional implementation, the distortion type includes a radial distortion type and a tangential distortion type.

In an optional implementation, calculating the distance between the distortion center coordinate and each of the first pixel coordinates based on the distortion center coordinate and the plurality of first pixel coordinates includes: according to the formula Calculate the distance between the distortion center coordinate and each first pixel coordinate; where, r _d represents the distance between the distortion center coordinate and each first pixel coordinate, and ( x _d , y _d ) represents each first pixel coordinate. Pixel coordinates, ( x ₀ , y ₀ ) represent the distortion center coordinates.

，計算得到所述多個第二像素座標；其中，(x _u ,y _u )表示第二像素座標，(x _d ,y _d )表示第一像素座標，r _d 表示所述畸變中心座標與每個第一像素座標的距離，k為正整數，λ ₁，λ ₂...λ _k 為所述畸變係數。 In an optional implementation, the obtaining of at least one distortion coefficient is based on the at least one distortion coefficient, a plurality of first pixel coordinates, and the distance between the distortion center coordinate and each first pixel coordinate. , the calculated multiple second pixel coordinates include: according to the formula

, calculate the plurality of second pixel coordinates; where ( x _u , y _u ) represents the second pixel coordinates, ( x _d , y _d ) represents the first pixel coordinates, r _d represents the distortion center coordinate and each The distance between the first pixel coordinates, k is a positive integer, λ ₁ , λ ₂ ... λ _k is the distortion coefficient.

，計算得到所述多個第二像素座標；其中，(x _u ,y _u )表示第二像素座標，(x _d ,y _d )表示第一像素座標，r _d 表示畸變中心座標與每個第一像素座標的距離，μ ₁，μ ₂為畸變係數。 In an optional implementation, the method further includes: according to the formula

, calculate the plurality of second pixel coordinates; among them, ( x _u , y _u ) represents the second pixel coordinates, ( x _d , y _d ) represents the first pixel coordinates, r _d represents the distortion center coordinate and each th The distance of one pixel coordinate, μ ₁ and μ ₂ are the distortion coefficients.

In an optional implementation, the method further includes: amplifying the undistorted image by performing data enhancement processing on the undistorted image, wherein the data enhancement processing includes performing data enhancement on the undistorted image. The image is subjected to one or more of flipping, rotating, scaling, and shifting processing.

An embodiment of the present application also provides an electronic device. The electronic device includes a processor and a memory. The processor is configured to execute a computer program stored in the memory to implement the distortion image generation method.

Embodiments of the present application also provide a computer-readable storage medium that stores at least one instruction. When the at least one instruction is executed by a processor, the method for generating a distorted image is implemented.

The distortion image generation methods, electronic devices and computer-readable storage media described in the embodiments of the present application can generate distorted images with different degrees of distortion and different types of distorted images, and utilize distorted images with different degrees of distortion and different types of distortion images. Types of distorted images train the deep learning model so that the deep learning model can recognize a variety of distorted images, thereby improving the accuracy and robustness of the deep learning model.

2: Electronic equipment

201:Memory

202: Processor

203: Computer programs

204: Communication bus

101-106: Steps

Figure 1 is a flow chart of a distortion image generation method provided by an embodiment of the present application.

FIG. 2 is a schematic structural diagram of an electronic device provided by an embodiment of the present application.

In order to more clearly understand the above objects, features and advantages of the present application, the present application will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that the specific embodiments described here are only used to explain the present application and are not used to limit the present application.

Many specific details are set forth in the following description to facilitate a full understanding of the present application. The described embodiments are only some, rather than all, of the embodiments of the present application. Based on this application All other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the scope of protection of this application.

Hereinafter, the terms “first” and “second” are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the quantity of indicated technical features. Therefore, features defined as "first" and "second" may explicitly or implicitly include one or more of these features. In the description of some embodiments of the present application, words such as "exemplary" or "such as" are used to represent examples, illustrations or explanations. Any embodiment or design described as "exemplary" or "such as" in some embodiments of the application is not intended to be construed as preferred or advantageous over other embodiments or designs. Rather, use of the words "exemplary" or "such as" is intended to present the concept in a concrete manner.

Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein in the description of the application is for the purpose of describing specific embodiments only and is not intended to limit the application.

Refer to Figure 1, which is a flow chart of a distortion image generation method provided by an embodiment of the present application. The method is applied to an electronic device (for example, the electronic device 2 shown in Figure 2). The electronic device can be any electronic product that can interact with the user, such as a personal computer, a tablet computer, a smart phone. , Personal Digital Assistant (PDA), game consoles, interactive Internet Protocol Television (IPTV), smart wearable devices, etc.

The electronic device is a device that can automatically perform numerical calculations and/or information processing according to preset or stored instructions. Its hardware includes, but is not limited to: microprocessors, Application Specific Integrated Circuits (ASICs) ), Field-Programmable Gate Array (FPGA), Digital Signal Processor (DSP), embedded devices, etc.

The electronic equipment may also include network equipment and/or user equipment. The network equipment includes, but is not limited to, a single network server, a server group composed of multiple network servers, or a cloud composed of a large number of hosts or network servers based on cloud computing.

The method specifically includes the following steps.

101. Electronic equipment acquires undistorted images.

In at least one embodiment of the present application, the undistorted image includes an undistorted image or a distortion-corrected image. The images include, but are not limited to, road scene images and industrial product images. For example, the undistorted image may include an undistorted image of the front view of the vehicle and an undistorted image of a plastic product.

In at least one embodiment of the present application, obtaining an undistorted image includes: using a distortion-corrected camera to capture an image of a road scene or an industrial product as an undistorted image. In this embodiment, the distorted image can also be dedistorted to obtain an undistorted image.

In at least one embodiment of the present application, the method further includes: obtaining more different undistorted images by performing a data enhancement operation on the undistorted image. The data enhancement operation includes, but does not Limited to one or more of the following processing methods: flipping the image, rotating the image, scaling the image, and cropping the image. Through the data enhancement operation, the undistorted image can be effectively expanded, thereby expanding the number of training samples.

102. Obtain multiple first pixel coordinates of the undistorted image and the first pixel value corresponding to each first pixel coordinate, and select one first pixel coordinate from the multiple first pixel coordinates as the distortion center. coordinates.

In at least one embodiment of the present application, obtaining a plurality of first pixel coordinates of the undistorted image includes: obtaining a plurality of first pixel coordinates of the undistorted image based on an OpenCV method. That is, all pixel coordinates in the undistorted image are obtained.

In at least one embodiment of the present application, obtaining the first pixel value corresponding to each first pixel coordinate includes: obtaining the first pixel value corresponding to each first pixel coordinate of the undistorted image based on the OpenCV method . In this embodiment, the first pixel value is represented by RGB. For example, the first pixel coordinate is (1750,160), and the corresponding first pixel value is (113,65,79), that is, R: 113, G: 65, and B: 79.

In at least one embodiment of the present application, selecting a first pixel coordinate from a plurality of first pixel coordinates as the distortion center coordinate includes: selecting any one pixel coordinate from all first pixel coordinates as the distortion center coordinate. . The distortion center is a reference center for distortion amounts at different fields of view. The farther the field of view is from the distortion center, the greater the corresponding distortion amount. That is, the farther away the first pixel coordinate is from the distortion center, the greater the distortion amount at the first pixel coordinate.

It can be understood that all first pixel coordinates may be used as distortion centers, and all first pixel coordinates will be selected once as distortion centers to generate more distorted images.

103. Calculate the distance between the distortion center coordinate and each first pixel coordinate based on the distortion center coordinate and the plurality of first pixel coordinates.

：

In at least one embodiment of the present application, calculating the distance between the distortion center coordinate and each of the first pixel coordinates based on the distortion center coordinate and a plurality of first pixel coordinates includes: according to the formula

:

Calculate the distance between the distortion center coordinate and each first pixel coordinate; where, r _d represents the distance between the distortion center coordinate and each first pixel coordinate, ( x _d , y _d ) represents the first pixel coordinate, ( x ₀ , y ₀ ) represents the distortion center coordinates.

，藉由上述公式

即可計算畸變中心到每一個第一像素座標的距離。 Specifically, each first pixel coordinate and the distortion center coordinate can be substituted into the above formula

, by the above formula

The distance from the distortion center to each first pixel coordinate can be calculated.

104. Obtain at least one distortion coefficient, and calculate the value corresponding to each first pixel coordinate based on the at least one distortion coefficient, the plurality of first pixel coordinates, and the distance between the distortion center coordinate and each first pixel coordinate. The second pixel coordinate.

In at least one embodiment of the present application, the distortion coefficient is used to change the degree of distortion of a distorted image, and the distortion coefficient can be obtained from a corresponding camera.

：

In at least one embodiment of the present application, a plurality of third pixel coordinates are calculated based on the at least one distortion coefficient, a plurality of first pixel coordinates, and a distance between the distortion center coordinate and each of the first pixel coordinates. Two-pixel coordinates include: according to the formula

:

The plurality of second pixel coordinates are calculated; where ( x _u , yu ₎ represents the second pixel coordinate, ( x _d , y _d ) represents the first pixel coordinate, r _d represents the distortion center coordinate and each first pixel coordinate. The distance of pixel coordinates, k is a positive integer, λ ₁ , λ ₂ ... λ _k is the distortion coefficient. In this embodiment, the larger the k value, the greater the degree of distortion.

，藉由上述公式

即可計算得到第二像素座標。 Specifically, each first pixel coordinate, the distance from the distortion center to the first pixel coordinate, the distortion coefficient and the k value are substituted into the above formula

, by the above formula

The second pixel coordinates can be calculated.

：

In at least one embodiment of the present application, the method for calculating multiple second pixel coordinates further includes: according to the formula

:

The plurality of second pixel coordinates are calculated; where ( x _u , yu ₎ represents the second pixel coordinate, ( x _d , y _d ) represents the first pixel coordinate, r _d represents the distortion center coordinate and each first pixel coordinate. The distance of pixel coordinates, μ ₁ and μ ₂ are distortion coefficients.

，藉由上述公式

即可計算得到第二像素座標。 Specifically, each first pixel coordinate, the distance from the distortion center to the first pixel coordinate, and the distortion coefficient are substituted into the above formula

, by the above formula

The second pixel coordinates can be calculated.

105. Use the first pixel value corresponding to each first pixel coordinate as the second pixel value corresponding to each second pixel coordinate.

或公式

計算得到的第二像素座標為(1230,132)，確定第二像素座標的第二像素值為(113,65,79)。 In at least one embodiment of the present application, the first pixel value corresponding to each first pixel coordinate is used as the second pixel value corresponding to each second pixel coordinate. For example, a first pixel coordinate is (1750,160), and the corresponding first pixel value is (113,65,79). After the above formula

or formula

The calculated second pixel coordinate is (1230,132), and the second pixel value determined as the second pixel coordinate is (113,65,79).

106. Generate a distorted image of the undistorted image based on the plurality of second pixel coordinates and the second pixel value corresponding to each second pixel coordinate.

In at least one embodiment of the present application, a distorted image of the undistorted image is generated based on a plurality of the second pixel coordinates and a second pixel value corresponding to each second pixel coordinate.

In at least one embodiment of the present application, the method further includes: determining a distortion type based on a plurality of second pixel coordinates; and generating a The distortion image corresponding to the distortion type.

Specifically, the distortion types include: radial distortion type and tangential distortion type.

計算得到第二座標，根據公式

計算得到第二座標，確定所述畸變類型為徑向畸變類型；利用上述公式

計算得到第二座標，根據公式

計算得到第二座標，確定所述畸變類型為切向畸變類型。 In this embodiment, determining the distortion type according to the plurality of second pixel coordinates includes: using the above formula

The second coordinate is calculated according to the formula

Calculate the second coordinates and determine that the distortion type is a radial distortion type; use the above formula

The second coordinate is calculated according to the formula

The second coordinates are calculated and the distortion type is determined to be a tangential distortion type.

In at least one embodiment of the present application, the method further includes: training a deep learning model using the undistorted image and the distorted image.

Specifically, the distorted images include distorted images with different degrees of distortion and distorted images with different types of distortion generated using the above method. The deep learning model can also be any one of AlexNet, VGGNet, GoogLeNet, ResNet, DenseNet, SSDNet, RCNN, YOLO series, FCN, SegNet and other models. This application does not specifically limit the types of deep learning models.

Further, for example, the deep learning model uses the ResNet model to classify objects in images. Inputting the distorted images into the ResNet model for training can improve the robustness of the ResNet model and improve the accuracy of object classification. sex. For example, the deep learning model uses the RCNN model to detect whether the image contains a target object. Inputting the distorted image into the RCNN model for training can improve the robustness of the RCNN model and improve the accuracy of distinguishing objects. . As another example, the deep learning model uses the FCN model to segment the target object in the image. Inputting the distorted image into the FCN model for training can improve the robustness of the FCN model and improve the accuracy of segmentation. .

It can be seen from the above technical solutions that this application generates images with different distortion degrees and distortion images of different distortion types, and trains depth with different types of distortion images, images with different distortion degrees and the undistorted images. learning model, so that the deep learning model can recognize different distorted images, thereby improving the accuracy and robustness of the deep learning model.

It is understandable that the electronic device acquires images through the camera and inputs the images into the trained deep learning model to realize image recognition, segmentation and other operations. However, the acquired images may be due to insufficient correction of the camera. The existence of different distortions causes the deep learning model to be unable to accurately interpret images. Perform operations such as accurate identification and segmentation, resulting in a decrease in the accuracy of the trained deep learning model. This application generates images with different degrees of distortion and distorted images of different types of distortion, and trains a deep learning model with undistorted images to obtain a trained deep learning model, so that the trained deep learning model can recognize For different distorted images, there is no need to consider whether the camera's correction of the image is sufficient. In short, regardless of whether the image captured by the camera is distorted, the trained deep learning model can perform operations such as image recognition and segmentation, thereby reducing the dependence on camera correction and improving the accuracy of the deep learning model. and robustness.

The above are only specific embodiments of the present application, but the protection scope of the present application is not limited thereto. For those of ordinary skill in the art, without departing from the creative concept of the present application, they can also make Improvements, but these all fall within the protection scope of this application.

As shown in Figure 2, Figure 2 is a schematic structural diagram of an electronic device provided by an embodiment of the present application. The electronic device 2 includes a memory 201, at least one processor 202, a computer program 203 stored in the memory 201 and executable on the at least one processor 202, and at least one communication bus 204.

Those skilled in the art can understand that the schematic diagram shown in FIG. 2 is only an example of the electronic device 2 and does not constitute a limitation of the electronic device 2. It may include more or less components than those shown in the figure, or a combination thereof. Certain components, or different components, for example, the electronic device 2 may also include input and output devices, network access devices, etc.

The at least one processor 202 may be a Central Processing Unit (CPU), or other general-purpose processor, a Digital Signal Processor (DSP), or an Application Specific Integrated Circuit (ASIC). ), Field-Programmable Gate Array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The at least one processor 202 may be a microprocessor or the at least one processor 202 may also be any conventional processor, etc., and the at least one processor 202 may be a microprocessor. A processor 202 is the control center of the electronic device 2 and uses various interfaces and lines to connect various parts of the entire electronic device 2 .

The memory 201 can be used to store the computer program 203. The at least one processor 202 runs or executes the computer program 203 stored in the memory 201 and calls the data stored in the memory 201, Various functions of the electronic device 2 are realized. The memory 201 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; and a storage data area. The area may store data created according to use of the electronic device 2 (such as audio data) and the like. In addition, the memory 201 may include non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card (Flash Card) ), at least one disk memory device, flash memory device, or other non-volatile solid-state memory device.

If the integrated modules/units of the electronic device 2 are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the present application can implement all or part of the processes in the above embodiment methods by instructing relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When executed by the processor, the computer program can implement the steps of each of the above method embodiments. Wherein, the computer program includes computer program code, and the computer program code may be in the form of source code, object code, executable file or some intermediate form. The computer-readable media may include: any entity or device capable of carrying the computer program code, a recording medium, a flash drive, a mobile hard disk, a magnetic disk, an optical disk, computer memory, and read-only memory (ROM, Read-Only Memory). Memory).

It is obvious to those skilled in the art that the present application is not limited to the details of the above-described exemplary embodiments, and can be embodied in other specific forms without departing from the spirit or essential characteristics of the present application. Form implementation of this application. Therefore, the embodiments should be regarded as illustrative and non-restrictive from any point of view, and the scope of the present application is defined by the appended claims rather than the above description, and it is therefore intended that those falling within the claims All changes within the meaning and scope of the equivalent elements are included in this application. Any associated association markup in a request item should not be considered to limit the request item in question.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application and are not limiting. Although the present application has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present application can be modified. Modifications or equivalent substitutions may be made without departing from the spirit and scope of the technical solution of the present application.

101-106: Steps

Claims (9)

A method for generating a distorted image, wherein the method for generating a distorted image includes: acquiring an undistorted image; acquiring a plurality of first pixel coordinates of the undistorted image and the first pixel corresponding to each first pixel coordinate. value; and select a first pixel coordinate from the plurality of first pixel coordinates as the distortion center coordinate; calculate the distortion center coordinate and each of the distortion center coordinates according to the distortion center coordinate and the plurality of first pixel coordinates. distance from the first pixel coordinate; obtain at least one distortion coefficient, and calculate each third pixel coordinate based on the at least one distortion coefficient, the plurality of first pixel coordinates, and the distance between the distortion center coordinate and each first pixel coordinate. a second pixel coordinate corresponding to a pixel coordinate; using the first pixel value corresponding to each first pixel coordinate as a second pixel value corresponding to each second pixel coordinate; based on the plurality of second pixel coordinates and the The second pixel value corresponding to each second pixel coordinate generates a distorted image of the undistorted image; and the deep learning model is trained using the undistorted image and the distorted image. The method for generating a distorted image according to claim 1, wherein the method further includes: determining a distortion type based on the plurality of second pixel coordinates; and based on the distortion type, the second pixel coordinates and the third The second pixel value corresponding to the two pixel coordinates generates a distortion image corresponding to the distortion type. The method for generating a distorted image according to claim 2, wherein the distortion type includes a radial distortion type and a tangential distortion type. The method for generating a distorted image according to claim 1, wherein the distortion center coordinate and each of the first pixel coordinates are calculated based on the distortion center coordinate and the plurality of first pixel coordinates. Distance includes: According to the formula

Calculate the distance between the distortion center coordinate and each first pixel coordinate; where, r _d represents the distance between the distortion center coordinate and each first pixel coordinate, and ( x _d , y _d ) represents each first pixel coordinate. Pixel coordinates, ( x ₀ , y ₀ ) represent the distortion center coordinates. The method for generating a distorted image according to claim 1, wherein the obtaining at least one distortion coefficient is based on the at least one distortion coefficient, a plurality of the first pixel coordinates and the distortion center coordinate and each of the distortion coefficients. The distance of the first pixel coordinate, the calculated multiple second pixel coordinates include: according to the formula

, calculate the plurality of second pixel coordinates; where ( x _u , y _u ) represents the second pixel coordinates, ( x _d , y _d ) represents the first pixel coordinates, r _d represents the distortion center coordinate and each The distance between the first pixel coordinates, k is a positive integer, λ ₁ , λ ₂ ... λ _k is the distortion coefficient. The method for generating a distorted image as described in claim 4, wherein the method further includes: according to the formula

, calculate the plurality of second pixel coordinates; among them, ( x _u , y _u ) represents the second pixel coordinates, ( x _d , y _d ) represents the first pixel coordinates, r _d represents the distortion center coordinate and each th The distance of one pixel coordinate, μ ₁ and μ ₂ are the distortion coefficients. The method for generating a distorted image according to claim 1, wherein the method further includes: amplifying the undistorted image by performing data enhancement processing on the undistorted image, wherein the data enhancement processing This includes performing one or more of flipping, rotating, scaling, and shifting processing on the undistorted image. An electronic device, wherein the electronic device includes a processor and a memory, and the processor is used to execute a computer program stored in the memory to implement the distortion image generation method as described in any one of claims 1 to 7 . A computer-readable storage medium, wherein the computer-readable storage medium stores at least one instruction, and when the at least one instruction is executed by a processor, the distortion image generation as described in any one of claims 1 to 7 is achieved. method.

Images