CN115063322A - An image processing method and system based on deep learning - Google Patents

Abstract

The invention discloses an image processing method and system based on deep learning. The method comprises the following steps: receiving a video file transmitted by an external terminal, and carrying out security check on the external terminal; separating an image frame from a video file, and extracting a significant image frame containing target features from the image frame; correcting the important image frame, adjusting the definition of the corrected important image frame, and outputting a high-resolution image; and constructing a target feature rule model based on deep learning, and inputting the obtained high-resolution image into the target feature rule model to determine a target feature rule. By adopting the technical scheme of the invention, the image processing efficiency can be improved, and the target characteristic rule can be rapidly determined.

Description

An image processing method and system based on deep learning

technical field

The present invention relates to the technical field of image processing, and in particular, to an image processing method and system based on deep learning.

Background technique

Image processing technology is a technology that uses a computer to process image information. It mainly includes image digitization, image enhancement and restoration, image data encoding, image segmentation and image recognition.

When an external device requests the server to track the target trajectory regularly, it needs to upload a large number of video files to the server, and at the same time, the server needs to receive a large number of video files uploaded by the external device every day. In a video, the target to be concerned is usually traceable, so that the trajectory rule after the target can be predicted. However, the existing graphics processing methods generally perform a large amount of screening work manually when determining the trajectory rules, which is inefficient. Based on this, the present invention proposes an image processing method and system based on deep learning, which can quickly perform image processing to improve the screening rate of target trajectory rules.

SUMMARY OF THE INVENTION

The present invention provides an image processing method based on deep learning, including:

Receive video files transmitted by external terminals, and conduct security checks on external terminals;

Separate image frames from video files, and extract important image frames containing target features from image frames;

Correct the important image frames, adjust the sharpness of the corrected important image frames, and output high-resolution images;

The target feature law model is constructed based on deep learning, and the obtained high-resolution image is input into the target feature law model to determine the target feature law.

The above-mentioned image processing method based on deep learning, wherein, the external terminal information is extracted from the video file, and the security degree of the external terminal is calculated. The video file is image processed, otherwise the external terminal request is rejected.

A deep learning-based image processing method as described above, wherein extracting important image frames containing target features from image frames specifically includes the following sub-steps:

From all the image frames separated from the video, find the first important image frame containing the target feature;

Perform grayscale processing on the target feature in the first important image frame to determine the grayscale value of the target feature;

Calculate the frame distance between the image frame to be extracted and the first important image frame according to the gray distribution value of the target feature, and determine all important image frames including the target feature according to the frame distance.

The above-mentioned image processing method based on deep learning, wherein, correcting important image frames specifically includes the following sub-steps:

Extract the target area from all important image frames, perform back-projection on the target area, compare whether there is a difference between the original image and the back-projected image, and select several important image frames whose differences are within a predetermined range;

Calculate the sharpness of each important image frame, select the image with the highest sharpness as the reference image frame, and perform initial sharpness adjustment on the important image frames whose sharpness is lower than the preset value;

Perform pixel, shading, and feature area adjustments on other important image frames according to the reference image frame.

An image processing method based on deep learning as described above, wherein, extracting the target area from the important image frame specifically includes: obtaining the front background area and the unknown area in the important image frame, in order to make the points in the unknown area as far as possible. The front and background areas are close, take each point of the unknown area as the center, obtain the distance between the pixel color in the radius field and the pixel color of the point, and use the pixel point whose pixel distance is greater than the set maximum threshold as the foreground area, The pixels smaller than the set minimum threshold are used as the background area to reduce the range of the unknown area.

The present invention also provides an image processing system based on deep learning, comprising:

The security check module is used to receive the video files transmitted by the external terminal and perform security check on the external terminal;

The important image frame determination module is used to separate the image frame from the video file, and extract the important image frame containing the target feature from the image frame;

The high-resolution image output module is used to correct important image frames, adjust the sharpness of the corrected important image frames, and output high-resolution images;

The deep learning module is used to construct a target feature law model based on deep learning, and input the obtained high-resolution image into the target feature law model to determine the target feature law.

The above-mentioned deep learning-based image processing system, wherein the security checking module is specifically used to extract external terminal information from the video file, and calculate the security degree of the external terminal, if the security degree of the external device is higher than the server security degree, Then it is allowed to perform image processing on the video file of the external device, otherwise, the request of the external terminal is rejected.

A deep learning-based image processing system as described above, wherein the important image frame determination module is specifically used to find the first important image frame containing the target feature from all the image frames separated from the video; The target feature in the frame is subjected to grayscale processing to determine the grayscale value of the target feature; the frame distance between the image frame to be extracted and the first important image frame is calculated according to the grayscale distribution value of the target feature, and the target feature is determined according to the frame distance. of all important image frames.

An image processing system based on deep learning as described above, wherein the high-resolution image output module is specifically used to extract the target area from all important image frames, back-project the target area, and compare the original image and the back-projected image. Whether there is a difference, select several important image frames with the difference within a predetermined range; calculate the sharpness of each important image frame, select the image with the highest sharpness as the reference image frame, and evaluate the important images whose sharpness is lower than the preset value. The initial sharpness adjustment is performed on the frame; pixel, shading, and feature area adjustments are performed on other important image frames according to the reference image frame.

The above-mentioned deep learning-based image processing system, wherein, in the high-resolution image output module, the target area is extracted from the important image frame, which is specifically used for: obtaining the front background area and the unknown area in the important image frame, In order to make the points in the unknown area as close as possible to the forward background area, take each point in the unknown area as the center, and obtain the distance between the pixel color in the radius area and the pixel color of the point, and the pixel distance is greater than the set point. The pixels with the maximum threshold are used as the foreground area, and the pixels smaller than the set minimum threshold are used as the background area to narrow the range of the unknown area.

The beneficial effects achieved by the present invention are as follows: by adopting the technical scheme of the present invention, the image processing efficiency can be improved, and the target characteristic law can be quickly determined.

Description of drawings

In order to explain the embodiments of the present invention or the technical solutions in the prior art more clearly, the following briefly introduces the accompanying drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only These are some embodiments described in the present invention, and for those of ordinary skill in the art, other drawings can also be obtained according to these drawings.

1 is a flowchart of a deep learning-based image processing method provided in Embodiment 1 of the present invention;

FIG. 2 is a schematic diagram of an image processing system based on deep learning according to Embodiment 2 of the present invention.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative efforts shall fall within the protection scope of the present invention.

Example 1

As shown in FIG. 1, an embodiment of the present invention provides an image processing method based on deep learning, including:

Step S110, the server receives the video file transmitted by the external terminal, and performs a security check on the external terminal;

In the embodiment of the present application, the external terminal information is extracted from the video file, and the security degree of the external terminal is calculated. If the security degree of the external device is higher than the security degree of the server, image processing of the video file of the external device is allowed, otherwise, the external terminal is rejected. ask.

Specifically, the security check on the external terminal specifically includes the following sub-steps:

Step 111: Extract the external terminal information from the video file of the external terminal, including the external terminal device information, the sending and receiving time of the data packet, the IP address, the port, and the like.

Step 112, determine whether the external terminal IP address and port allow direct access in the server, if so, allow image processing to the video file of the external device, otherwise perform step 113;

If the IP address and port of the external device have been stored as secure addresses in the server, subsequent image processing of the video file of the external device is directly allowed. If the IP address and port of the external device are not stored in the server, the external device may There is a security risk, so external equipment needs to be security authenticated.

Step 113: Calculate the security degree of the external device, and calculate the access openness of the server to the external device. If the security degree of the external device is greater than the openness of the server, image processing of the video file of the external device is allowed, otherwise, the request from the external terminal is rejected. ;

计算外部设备的安全程度，其中，Se表示外部设备的安全程度；λ₁为防火墙给出的外部设备风险对安全程度的影响权重；λ₂为外部设备访问地址与防火墙过滤规则的关联关系对安全评估值的影响权重；e＝2.718；μ为防火墙对外部设备IP端口的阻断风险因子；Nr为防火墙影响外部设备IP端口数据处理的漏洞个数；Ns为防火墙总的漏洞个数；

为外部设备IP地址的向量表示；

为防火墙过滤规则的向量表示，t的取值为1到T，T为防火墙功率规则的字符总数。Specifically, using the formula

Calculate the security degree of the external device, where Se represents the security degree of the external device; λ ₁ is the influence weight of the risk of the external device given by the firewall on the security degree _; The impact weight of the evaluation value; e=2.718; μ is the risk factor of the firewall blocking the IP port of the external device; Nr is the number of vulnerabilities that the firewall affects the data processing of the IP port of the external device; Ns is the total number of vulnerabilities in the firewall;

is the vector representation of the IP address of the external device;

is the vector representation of firewall filtering rules, t ranges from 1 to T, and T is the total number of characters of firewall power rules.

计算服务器对外部设备的访问开放程度，其中，Kf表示服务器对外部设备的访问开放程度；e＝2.718；N₁表示外部设备业务种类属于服务器类别的业务范畴总个数；N₂表示外部设备业务种类不属于服务器类别的业务范畴总个数；β_IP表示外部设备IP地址是否为服务器的非法IP域，若否，则β_IP为1，若是，则β_IP为0；

表示服务器和外部设备是否有共同遵循的安全协议，若是，则

否则

Then use the formula

Calculate the access openness of the server to the external device, where Kf represents the access openness of the server to the external device; e=2.718; N ₁ represents the total number of business categories that the external device business type belongs to the server category; N ₂ represents the external device business The total number of business categories whose types do not belong to the server category; β _IP indicates whether the IP address of the external device is an illegal IP domain of the server, if not, β _IP is 1; if so, β _IP is 0;

Indicates whether the server and the external device have a common security protocol, and if so, then

otherwise

If Se≥Kf, it means that the security degree of the external device is relatively high, and the video file of the external device is allowed to perform image processing; otherwise, the security degree of the external device is low, and the server rejects the request of the external terminal.

Step S120, separate the image frame from the video, and extract the important image frame containing the target feature from the image frame;

Specifically, extracting important image frames containing target features from the image frames specifically includes the following sub-steps:

Step 121, from all the image frames separated from the video, find the first important image frame containing the target feature;

Step 122, performing grayscale processing on the target feature in the first important image frame to determine the grayscale value of the target feature;

Specifically, the RGB value of each pixel is extracted from the target feature image area in the first important image frame, and then according to the formula Gray(i,j)=WR*R(i,j)+WG*G(i, j)+WB*B(i,j) Calculate the gray value of each pixel in the target feature, WR, WG and WB are the weights of the RGB values of the pixel respectively, and then the gray value of the target feature of the first important image frame ,Right now

Step 123, calculating the frame distance between the image frame to be extracted and the first important image frame according to the grayscale distribution value of the target feature, and determining all important image frames containing the target feature according to the frame distance;

其中，N为图像帧数量，

表示第i+1、i图像帧的灰度值，

表示首个重要图像帧的灰度值。其中，根据帧距确定包含目标特征的所有重要图像帧，具体为对所有帧按照帧距进行排序，选择帧距最高的若干图像、或者帧距高于预设阈值的若干图像作为重要图像帧。Specifically, after the first important image frame is determined, the frame distance between the subsequent image frame and the first important image frame is calculated, that is,

Among them, N is the number of image frames,

Represents the gray value of the i+1, i image frame,

Represents the grayscale value of the first significant image frame. Wherein, all important image frames containing the target feature are determined according to the frame distance, specifically, all frames are sorted according to the frame distance, and several images with the highest frame distance or several images with a frame distance higher than a preset threshold are selected as important image frames.

Step S130, correcting the important image frame, adjusting the sharpness of the corrected important image frame, and outputting a high-resolution image;

In the embodiment of the present application, the correction of important image frames specifically includes the following sub-steps:

Step 131, extracting the target area from all important image frames, back-projecting the target area, comparing whether there is a difference between the original image and the back-projected image, and selecting several important image frames whose differences are within a predetermined range;

将该像素距离大于设定最大阈值的像素点作为前景区域，将小于设定最小阈值的像素点作为背景区域，以缩小未知区域的范围。对反投影得到的数值与正投影得到的数值进行比较，若二者得出的结果在预定范围外，则说明该图像是非常不清晰的，则丢弃该图像。Among them, extracting the target area from all important image frames specifically includes: obtaining the front and background areas and unknown areas in the important image frames Ii=αFi+(1-α)Bi, where α represents transparency, Fi is the foreground pixel, and Bi is the background pixel , i represents the ith pixel. In order to make the points in the unknown area as close as possible to the forward background area, take each point in the unknown area as the center to obtain the distance between the pixel color in the radius area and the pixel color of the point

The pixels whose pixel distance is greater than the set maximum threshold are regarded as the foreground area, and the pixels whose distance is less than the set minimum threshold are regarded as the background area, so as to reduce the range of the unknown area. Compare the value obtained by back projection with the value obtained by forward projection. If the results obtained by the two are outside the predetermined range, it means that the image is very unclear, and the image is discarded.

Step 132: Calculate the sharpness of each important image frame, select the image with the highest sharpness as the reference image frame, and perform initial sharpness adjustment on the important image frame whose sharpness is lower than the preset value;

计算每个重要图像帧的清晰度，其中，x为重要图像帧长度像素数，y为重要图像帧宽度像素数，z为图像尺寸，将清晰度最高的图像作为参考图像帧。Specifically, using the formula

Calculate the sharpness of each important image frame, where x is the number of pixels in the length of the important image frame, y is the number of pixels in the width of the important image frame, z is the image size, and the image with the highest sharpness is used as the reference image frame.

n为以起始区域块的中心为原点，起始区域块对角边长为半径的圆周上的点，一般选取n＝4作为圆周上均匀分布的四个点，d_i为选取的起始区域块的重建坐标与圆周像素坐标的距离，R_j为圆周上选取点的像素坐标的像素值，即d₁为选取的起始区域块的重建坐标与左边相邻区域块像素坐标的距离，R₁表示右边相邻区域块像素坐标的像素值；d₂为起始区域块重建坐标与右边相邻区域块像素坐标中心点的距离，R₂表示左边相邻区域块像素坐标的像素值；d₃为起始区域块重建坐标与上边相邻区域块像素坐标中心点的距离，R₃表示下边相邻区域块像素坐标的像素值；d₄为起始区域块重建坐标与下边相邻区域块像素坐标中心点的距离，R₄表示上边相邻区域块像素坐标的像素值。If there are important image frames whose resolution is lower than the preset value, adjust the sharpness of these data, which specifically includes: searching for blurred area blocks in the important image frames whose resolution is lower than the preset value, and reconstructing the blurred area blocks , and encode the reconstructed image frame. Among them, reconstructing the fuzzy area block is specifically performing pixel reconstruction of the fuzzy area block, reconstructing the pixel value coordinates

n is a point on the circle with the center of the starting area block as the origin and the diagonal side length of the starting area block as the radius. Generally, n=4 is selected as the four points evenly distributed on the circumference, and d _i is the starting point of the selection. The distance between the reconstructed coordinates of the region block and the pixel coordinates of the circumference, R _j is the pixel value of the pixel coordinates of the selected point on the circumference, that is, d ₁ is the distance between the reconstructed coordinates of the selected starting region block and the pixel coordinates of the adjacent region block on the left, R ₁ represents the pixel value of the pixel coordinates of the adjacent region block on the right; d ₂ is the distance between the reconstruction coordinates of the starting region block and the center point of the pixel coordinates of the adjacent region block on the right, and R ₂ represents the pixel value of the pixel coordinates of the adjacent region block on the left; d ₃ is the distance between the reconstructed coordinates of the starting area block and the center point of the pixel coordinates of the upper adjacent area block, R ₃ is the pixel value of the pixel coordinates of the lower adjacent area block; d ₄ is the reconstructed coordinates of the starting area block and the adjacent area on the lower side The distance from the center point of the block pixel coordinate, R ₄ represents the pixel value of the block pixel coordinate of the adjacent area above.

Step 133: Adjust the pixels, brightness and characteristic area of other important image frames according to the reference image frame;

其中n表示图像内的投影线条数，q_k,i表示经过像素点k的第i条投影线，计算这些目标区域与参考图像帧目标区域的像素比Z＝Pv/h(k)，P_V为参考图像帧像素值，根据像素比对其他重要图像帧进行像素缩放。明暗度调整具体包括：计算参考图像帧的明暗度＝R*0.299+G*0.587+B*0.114，明暗度计算中的权重数根据实际需要进行调整，按照参考图像帧明暗度进行其他重点图像帧的明暗度调整。特征区域调整具体包括：按照参考图像帧对其他重点图像帧进行旋转操作、等比缩放等。The pixel adjustment specifically includes: determining the pixel value of the target area of the other important image frame according to the back-projection image of the other important image frame

where n represents the number of projected lines in the image, q _k,i represents the i-th projected line passing through pixel k, calculate the pixel ratio between these target areas and the target area of the reference image frame Z=Pv/h(k), P _V For the pixel value of the reference image frame, pixel scaling is performed on other important image frames according to the pixel ratio. The brightness adjustment specifically includes: calculating the brightness of the reference image frame=R*0.299+G*0.587+B*0.114, adjusting the number of weights in the brightness calculation according to actual needs, and performing other key image frames according to the brightness of the reference image frame. shading adjustment. The feature area adjustment specifically includes: performing a rotation operation, proportional scaling, etc. on other key image frames according to the reference image frame.

Step S140, constructing a target feature law model based on deep learning, and inputting the obtained high-resolution image into the target feature law model to determine the target feature law;

Specifically, a target feature law model is constructed based on deep learning, and the obtained high-resolution image is input into the target feature law model to determine the target feature law, which specifically includes the following sub-steps:

Step 141, extracting target features from the obtained high-resolution image to form a target feature vector;

Step 142: Input the target feature vector set into the target feature regularity model, train multiple damping trend prediction models to obtain different sub-trend prediction models, use each sub-trend prediction model to perform trend prediction on the target feature vector set, and estimate the trend through the trend prediction results. Get the set of weights of each sub-prediction model;

再利用子预测模型对目标行人特征向量集进行预测，得到预测结果，通过预测结果采用公式

估计子预测模型的权重的集合{λ₁，λ₂，λ₃......λ_T}，其中，x_i为目标特征向量中的目标形状特征，y_i为目标特征向量的目标空间关系特征，μ₁和μ₂为目标形状特征和目标空间关系特征的影响权重，n为目标特征向量的总数，T为子预测模型的数量。Specifically, input the target pedestrian feature vector set into the target pedestrian feature trend model, use the target pedestrian feature vector set to train multiple damping trend prediction models, and use each damping trend prediction model as a sub-prediction model

Then use the sub-prediction model to predict the target pedestrian feature vector set to obtain the prediction result, and use the formula through the prediction result.

Estimate the set of weights of the sub-prediction model {λ ₁ , λ ₂ , λ ₃ ......λ _T }, where x _i is the target shape feature in the target feature vector, and _yi is the target space of the target feature vector Relational features, _μ1 and _μ2 are the influence weights of target shape features and target spatial relationship features, n is the total number of target feature vectors, and T is the number of sub-prediction models.

Step 143: Find the optimal value corresponding to each weight in the set of weights, and determine the target characteristic rule by combining each sub-prediction model and the optimal value of its corresponding weight;

的权重的集合{λ₁，λ₂，λ₃……λ_T}中，每个权重对应的最优值；通过各个子预测模型

和其对应的权重的最优值{λ₁，λ₂，λ₃……λ_T}组合确定目标特征规律

Specifically, each sub-prediction model is calculated

In the set of weights {λ ₁ , λ ₂ , λ ₃ ......λ _T }, the optimal value corresponding to each weight; through each sub-prediction model

and the optimal value of its corresponding weight {λ ₁ , λ ₂ , λ ₃ ......λ _T } to determine the target characteristic law

Embodiment 2

As shown in FIG. 2, an embodiment of the present invention provides an image processing system 2 based on deep learning, including:

The security check module 21 is used for receiving the video file transmitted by the external terminal, and performing security check on the external terminal;

Among them, the security checking module 21 is used to extract the external terminal information from the video file, and calculate the security degree of the external terminal. If the security degree of the external device is higher than the security degree of the server, it is allowed to perform image processing on the video file of the external device, otherwise it is rejected. External terminal request. Specifically, the security check module 21 is specifically used to extract the external terminal information from the video file of the external terminal, including the information of the external terminal equipment, the sending and receiving time of the data packet, the IP address, the port, etc.; determine whether the IP address and port of the external terminal are not Allow direct access in the server, if yes, allow image processing of the video files of the external device, otherwise calculate the security degree of the external device, and calculate the access openness of the server to the external device, if the security degree of the external device is greater than that of the server If the degree of openness is set, it is allowed to perform image processing on the video file of the external device, otherwise, the request of the external terminal is rejected.

计算外部设备的安全程度，其中，Se表示外部设备的安全程度；λ₁为防火墙给出的外部设备风险对安全程度的影响权重；λ₂为外部设备访问地址与防火墙过滤规则的关联关系对安全评估值的影响权重；e＝2.718；μ为防火墙对外部设备IP端口的阻断风险因子；Nr为防火墙影响外部设备IP端口数据处理的漏洞个数；Ns为防火墙总的漏洞个数；

为外部设备IP地址的向量表示；

为防火墙过滤规则的向量表示，t的取值为1到T，T为防火墙功率规则的字符总数。Specifically, using the formula

Calculate the security degree of the external device, where Se represents the security degree of the external device; λ ₁ is the influence weight of the risk of the external device given by the firewall on the security degree _; The impact weight of the evaluation value; e=2.718; μ is the risk factor of the firewall blocking the IP port of the external device; Nr is the number of vulnerabilities that the firewall affects the data processing of the IP port of the external device; Ns is the total number of vulnerabilities in the firewall;

is the vector representation of the IP address of the external device;

is the vector representation of firewall filtering rules, t ranges from 1 to T, and T is the total number of characters of firewall power rules.

计算服务器对外部设备的访问开放程度，其中，Kf表示服务器对外部设备的访问开放程度；e＝2.718；N₁表示外部设备业务种类属于服务器类别的业务范畴总个数；N₂表示外部设备业务种类不属于服务器类别的业务范畴总个数；β_IP表示外部设备IP地址是否为服务器的非法IP域，若否，则β_IP为1，若是，则β_IP为0；

表示服务器和外部设备是否有共同遵循的安全协议，若是，则

否则

Then use the formula

Calculate the access openness of the server to the external device, where Kf represents the access openness of the server to the external device; e=2.718; N ₁ represents the total number of business categories that the external device business type belongs to the server category; N ₂ represents the external device business The total number of business categories whose types do not belong to the server category; β _IP indicates whether the IP address of the external device is an illegal IP domain of the server, if not, β _IP is 1; if so, β _IP is 0;

Indicates whether the server and the external device have a common security protocol, and if so, then

otherwise

If Se≥Kf, it means that the security degree of the external device is relatively high, and the video file of the external device is allowed to perform image processing; otherwise, the security degree of the external device is low, and the server rejects the request of the external terminal.

The important image frame determination module 22 is used to separate the image frame from the video file, and extract the important image frame containing the target feature from the image frame;

Among them, the important image frame determination module 22 is specifically used to find the first important image frame containing the target feature from all the image frames separated from the video; perform grayscale processing on the target feature in the first important image frame to determine the target feature Calculate the frame distance between the image frame to be extracted and the first important image frame according to the grayscale distribution value of the target feature, and determine all important image frames containing the target feature according to the frame distance.

在确定出首个重要图像帧之后，计算后续图像帧与首个重要图像帧的帧距，即

其中，N为图像帧数量，

表示第i+1、i图像帧的灰度值，

表示首个重要图像帧的灰度值。其中，根据帧距确定包含目标特征的所有重要图像帧，具体为对所有帧按照帧距进行排序，选择帧距最高的若干图像、或者帧距高于预设阈值的若干图像作为重要图像帧。Specifically, the RGB value of each pixel is extracted from the target feature image area in the first important image frame, and then according to the formula Gray(i,j)=WR*R(i,j)+WG*G(i, j)+WB*B(i,j) Calculate the gray value of each pixel in the target feature, WR, WG and WB are the weights of the RGB values of the pixel respectively, and then the gray value of the target feature of the first important image frame ,Right now

After the first important image frame is determined, the frame distance between the subsequent image frame and the first important image frame is calculated, that is,

Among them, N is the number of image frames,

Represents the gray value of the i+1, i image frame,

Represents the grayscale value of the first significant image frame. Wherein, all important image frames containing the target feature are determined according to the frame distance, specifically, all frames are sorted according to the frame distance, and several images with the highest frame distance or several images with a frame distance higher than a preset threshold are selected as important image frames.

The high-resolution image output module 23 is used to correct important image frames, adjust the sharpness of the corrected important image frames, and output high-resolution images;

Among them, the high-resolution image output module 23 is specifically used to extract the target area from all important image frames, perform back-projection on the target area, compare whether there is a difference between the original image and the back-projected image, and select a number of differences within a predetermined range. Important image frames; calculate the sharpness of each important image frame, select the image with the highest sharpness as the reference image frame, and perform initial sharpness adjustment on the important image frames whose sharpness is lower than the preset value; Important image frames undergo pixel, shading, and feature area adjustments.

将该像素距离大于设定最大阈值的像素点作为前景区域，将小于设定最小阈值的像素点作为背景区域，以缩小未知区域的范围。对反投影得到的数值与正投影得到的数值进行比较，若二者得出的结果在预定范围外，则说明该图像是非常不清晰的，则丢弃该图像。Specifically, extracting the target area from all important image frames includes: acquiring the front and background areas and unknown areas in the important image frames Ii=αFi+(1-α)Bi, where α represents transparency, Fi is the foreground pixel, and Bi is the background pixel, i represents the ith pixel. In order to make the points in the unknown area as close as possible to the forward background area, take each point in the unknown area as the center to obtain the distance between the pixel color in the radius area and the pixel color of the point

The pixels whose pixel distance is greater than the set maximum threshold are regarded as the foreground area, and the pixels whose distance is less than the set minimum threshold are regarded as the background area, so as to reduce the range of the unknown area. Compare the value obtained by back projection with the value obtained by forward projection. If the results obtained by the two are outside the predetermined range, it means that the image is very unclear, and the image is discarded.

计算每个重要图像帧的清晰度，其中，x为重要图像帧长度像素数，y为重要图像帧宽度像素数，z为图像尺寸，将清晰度最高的图像作为参考图像帧。using the formula

Calculate the sharpness of each important image frame, where x is the number of pixels in the length of the important image frame, y is the number of pixels in the width of the important image frame, z is the image size, and the image with the highest sharpness is used as the reference image frame.

n为以起始区域块的中心为原点，起始区域块对角边长为半径的圆周上的点，一般选取n＝4作为圆周上均匀分布的四个点，d_i为选取的起始区域块的重建坐标与圆周像素坐标的距离，R_j为圆周上选取点的像素坐标的像素值，即d₁为选取的起始区域块的重建坐标与左边相邻区域块像素坐标的距离，R₁表示右边相邻区域块像素坐标的像素值；d₂为起始区域块重建坐标与右边相邻区域块像素坐标中心点的距离，R₂表示左边相邻区域块像素坐标的像素值；d₃为起始区域块重建坐标与上边相邻区域块像素坐标中心点的距离，R₃表示下边相邻区域块像素坐标的像素值；d₄为起始区域块重建坐标与下边相邻区域块像素坐标中心点的距离，R₄表示上边相邻区域块像素坐标的像素值。If there are important image frames whose resolution is lower than the preset value, adjust the sharpness of these data, which specifically includes: searching for blurred area blocks in the important image frames whose resolution is lower than the preset value, and reconstructing the blurred area blocks , and encode the reconstructed image frame. Among them, reconstructing the fuzzy area block is specifically performing pixel reconstruction of the fuzzy area block, reconstructing the pixel value coordinates

n is a point on the circle with the center of the starting area block as the origin and the diagonal side length of the starting area block as the radius. Generally, n=4 is selected as the four points evenly distributed on the circumference, and d _i is the starting point of the selection. The distance between the reconstructed coordinates of the region block and the pixel coordinates of the circumference, R _j is the pixel value of the pixel coordinates of the selected point on the circumference, that is, d ₁ is the distance between the reconstructed coordinates of the selected starting region block and the pixel coordinates of the adjacent region block on the left, R ₁ represents the pixel value of the pixel coordinates of the adjacent region block on the right; d ₂ is the distance between the reconstruction coordinates of the starting region block and the center point of the pixel coordinates of the adjacent region block on the right, and R ₂ represents the pixel value of the pixel coordinates of the adjacent region block on the left; d ₃ is the distance between the reconstructed coordinates of the starting area block and the center point of the pixel coordinates of the upper adjacent area block, R ₃ is the pixel value of the pixel coordinates of the lower adjacent area block; d ₄ is the reconstructed coordinates of the starting area block and the adjacent area on the lower side The distance from the center point of the block pixel coordinate, R ₄ represents the pixel value of the block pixel coordinate of the adjacent area above.

其中n表示图像内的投影线条数，q_k,i表示经过像素点k的第i条投影线，计算这些目标区域与参考图像帧目标区域的像素比Z＝Pv/h(k)，PV为参考图像帧像素值，根据像素比对其他重要图像帧进行像素缩放。明暗度调整具体包括：计算参考图像帧的明暗度＝R*0.299+G*0.587+B*0.114，明暗度计算中的权重数根据实际需要进行调整，按照参考图像帧明暗度进行其他重点图像帧的明暗度调整。特征区域调整具体包括：按照参考图像帧对其他重点图像帧进行旋转操作、等比缩放等。The pixel adjustment specifically includes: determining the pixel value of the target area of the other important image frame according to the back-projected image of the other important image frame

where n represents the number of projected lines in the image, q _k,i represents the i-th projected line passing through pixel k, calculate the pixel ratio Z=Pv/h(k) between these target areas and the target area of the reference image frame, and PV is Refer to the pixel value of the image frame, and perform pixel scaling on other important image frames according to the pixel ratio. The brightness adjustment specifically includes: calculating the brightness of the reference image frame=R*0.299+G*0.587+B*0.114, adjusting the number of weights in the brightness calculation according to actual needs, and performing other key image frames according to the brightness of the reference image frame. shading adjustment. The feature area adjustment specifically includes: performing a rotation operation, proportional scaling, etc. on other key image frames according to the reference image frame.

The deep learning module 24 is configured to construct a target feature law model based on deep learning, and input the obtained high-resolution image into the target feature law model to determine the target feature law.

Among them, the deep learning module 24 is specifically used for: extracting target features from the obtained high-resolution images to form target feature vectors; inputting the target feature vector set into the target feature law model, and training multiple damping trend prediction models to obtain different subtypes The trend prediction model uses each sub-trend prediction model to predict the trend of the target feature vector set, and estimates the set of weights of each sub-prediction model through the trend prediction result; finds the optimal value corresponding to each weight in the set of weights, Determine the target characteristic law by the combination of each sub-prediction model and the optimal value of its corresponding weight;

再利用子预测模型对目标行人特征向量集进行预测，得到预测结果，通过预测结果采用公式

估计子预测模型的权重的集合{λ₁，λ₂，λ₃……λ_T}，其中，x_i为目标特征向量中的目标形状特征，y_i为目标特征向量的目标空间关系特征，μ₁和μ₂为目标形状特征和目标空间关系特征的影响权重，n为目标特征向量的总数，T为子预测模型的数量。Specifically, input the target pedestrian feature vector set into the target pedestrian feature trend model, use the target pedestrian feature vector set to train multiple damping trend prediction models, and use each damping trend prediction model as a sub-prediction model

Then use the sub-prediction model to predict the target pedestrian feature vector set to obtain the prediction result, and use the formula through the prediction result.

The set of _weights of the estimated sub-prediction model {λ ₁ , _{λ 2} , _{λ 3} ...... ₁ and μ ₂ are the influence weights of the target shape feature and the target spatial relationship feature, n is the total number of target feature vectors, and T is the number of sub-prediction models.

的权重的集合{λ₁，λ₂，λ₃......λ_T}中，每个权重对应的最优值；通过各个子预测模型

和其对应的权重的最优值{λ₁，λ₂，λ₃......λ_T}组合确定目标特征规律

Calculate each sub-prediction model

In the set of weights {λ ₁ , λ ₂ , λ ₃ ......λ _T }, the optimal value corresponding to each weight; through each sub-prediction model

and the optimal value of its corresponding weight {λ ₁ , λ ₂ , λ ₃ ......λ _T } to determine the target characteristic law

The specific embodiments described above further describe the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above descriptions are only specific embodiments of the present invention, and are not intended to limit the scope of the present invention. Any modification, equivalent replacement, improvement, etc. made on the basis of the technical solution of the present invention shall be included within the protection scope of the present invention.

Claims (10)

1. an image processing method based on deep learning, is characterized in that, comprises: Receive video files transmitted by external terminals, and conduct security checks on external terminals; Separate image frames from video files, and extract important image frames containing target features from image frames; Correct the important image frames, adjust the sharpness of the corrected important image frames, and output high-resolution images; The target feature law model is constructed based on deep learning, and the obtained high-resolution image is input into the target feature law model to determine the target feature law. 2. a kind of image processing method based on deep learning as claimed in claim 1, is characterized in that, extracts external terminal information from video file, calculates the security degree of external terminal, if the security degree of external equipment is higher than server security degree , the image processing of the video file of the external device is allowed, otherwise the request of the external terminal is rejected. 3. a kind of image processing method based on deep learning as claimed in claim 1 is characterized in that, extracting important image frames containing target features from image frames, specifically comprises the following substeps: From all the image frames separated from the video, find the first important image frame containing the target feature; Perform grayscale processing on the target feature in the first important image frame to determine the grayscale value of the target feature; Calculate the frame distance between the image frame to be extracted and the first important image frame according to the gray distribution value of the target feature, and determine all important image frames including the target feature according to the frame distance. 4. a kind of image processing method based on deep learning as claimed in claim 1, it is characterised in that the important image frame is corrected, specifically comprises the following substeps: Extract the target area from all important image frames, perform back-projection on the target area, compare whether there is a difference between the original image and the back-projected image, and select several important image frames whose differences are within a predetermined range; Calculate the sharpness of each important image frame, select the image with the highest sharpness as the reference image frame, and perform initial sharpness adjustment on the important image frames whose sharpness is lower than the preset value; Perform pixel, shading, and feature area adjustments on other important image frames according to the reference image frame. 5. a kind of image processing method based on deep learning as claimed in claim 4 is characterized in that, extracting the target area from important image frame, specifically comprises: obtaining the front background area and unknown area in important image frame, in order to make The points in the unknown area are as close as possible to the forward background area, and each point in the unknown area is taken as the center to obtain the distance between the pixel color in the radius area and the pixel color of the point, and the pixel distance is greater than the set maximum threshold. The pixels of , are taken as the foreground area, and the pixels smaller than the set minimum threshold are taken as the background area to reduce the range of the unknown area. 6. An image processing system based on deep learning, comprising: The security check module is used to receive the video files transmitted by the external terminal and perform security check on the external terminal; The important image frame determination module is used to separate the image frame from the video file, and extract the important image frame containing the target feature from the image frame; The high-resolution image output module is used to correct important image frames, adjust the sharpness of the corrected important image frames, and output high-resolution images; The deep learning module is used to construct a target feature law model based on deep learning, and input the obtained high-resolution image into the target feature law model to determine the target feature law. 7. a kind of image processing system based on deep learning as claimed in claim 6, is characterized in that, the security check module is specifically used for extracting external terminal information from video file, calculates the security degree of external terminal, if the security of external equipment If the degree is higher than the security degree of the server, it is allowed to perform image processing on the video file of the external device, otherwise, the request of the external terminal is rejected. 8. A kind of image processing system based on deep learning as claimed in claim 6, it is characterised in that the important image frame determination module is specifically used to find the first important image containing the target feature in all the image frames separated from the video frame; perform grayscale processing on the target feature in the first important image frame to determine the grayscale value of the target feature; calculate the frame distance between the image frame to be extracted and the first important image frame according to the grayscale distribution value of the target feature, Determine all important image frames that contain the target feature according to the frame distance. 9. a kind of image processing system based on deep learning as claimed in claim 6, is characterized in that, high-resolution image output module, is specifically used for extracting target area from all important image frames, carries out back-projection to target area, Compare whether there is a difference between the original image and the back-projected image, and select several important image frames whose differences are within a predetermined range; The initial definition adjustment is performed on the important image frame of the preset value; the pixel, brightness and characteristic area adjustment is performed on other important image frames according to the reference image frame. 10. The deep learning-based image processing system according to claim 9, wherein, in the high-resolution image output module, the target area is extracted from the important image frame, and is specifically used for: obtaining the image in the important image frame. For the front-background area and the unknown area, in order to make the points in the unknown area as close as possible to the front-background area, take each point in the unknown area as the center to obtain the distance between the pixel color in the radius area and the pixel color of the point, The pixels whose pixel distance is greater than the set maximum threshold are taken as the foreground area, and the pixels less than the set minimum threshold are taken as the background area to narrow the range of the unknown area.

Images