CN111814595A - Low-light pedestrian detection method and system based on multi-task learning - Google Patents

Abstract

The invention discloses a low-light pedestrian detection method and system based on multi-task learning, including acquiring normal and low-light pedestrian data sets; constructing a lighting enhancement network, and using the normal and low-light pedestrian data sets for pre-training; constructing pedestrian detection The network uses the normal illumination pedestrian dataset for pre-training; based on multi-task learning, a multi-task learning module that can fuse the features between different tasks is designed, and the features are shared between the two networks to build a multi-task feature sharing low-light pedestrian detection. network; import two pre-trained models into the low-light pedestrian detection network, and use normal and low-light pedestrian datasets for training to obtain a low-light pedestrian detection model with multi-task feature sharing; a low-light pedestrian detection model using multi-task feature sharing The detection model detects the detected image and obtains the position of the pedestrian in the image. The present invention can accurately and efficiently detect the position of a pedestrian in a low-light image.

Description

Low-light pedestrian detection method and system based on multi-task learning

technical field

The invention relates to the technical field of computer vision target detection, in particular to a low-light pedestrian detection method and system based on multi-task learning.

Background technique

The rapid economic development has brought about frequent exchanges of people in different regions and cities, and the accompanying public safety hazards have caused relevant departments to spend a lot of energy. At present, video surveillance equipment, which is an important part of the urban security system, has been widely used, and they are installed in public areas such as streets, schools, and stations. These devices are mainly used to record and store what happened in the relevant places, so that people can fulfill the needs of remote monitoring and emergency command, and ensure the public safety of the society. Pedestrians are the main body in video surveillance. Using intelligent technology to study and analyze pedestrian behavior is an important part of intelligent surveillance technology. Pedestrian detection is one of the key technologies in this field.

On the other hand, pedestrian detection also plays a very critical role in the field of unmanned driving. In recent years, with the vigorous development of artificial intelligence, the research of driverless vehicles has been greatly developed. Pedestrian detection is an important topic in unmanned vehicle research, which has an important impact on improving the ability of vehicles to perceive pedestrians around them.

Pedestrian detection is a specific application of target detection. From this perspective, pedestrian detection algorithms can be divided into two categories: anchor box-based and keypoint-based methods. Convolutional Neural Networks (CNNs) were first introduced to object detection in RCNNs, which can detect without the need for handcrafted features. Keypoint-based object detection algorithms generate object bounding boxes by detecting and grouping keypoints. This greatly simplifies the output of the network and eliminates the need to design anchor boxes. CornerNet and CornerNet-Lite are typical representatives of keypoint-based methods, which determine the object boundary by predicting the upper left and lower right corners of the object.

Although the above pedestrian detection algorithms achieve satisfactory performance under normal lighting conditions, in practical applications, normal lighting is not always guaranteed, and low-light environments are very common. The main reason for the poor pedestrian detection effect in low-light environment is that the color information in the input image is seriously lost due to low light, and color information is the key to pedestrian detection. In order to solve the problem of low-light pedestrian detection, infrared pedestrian detection has been developed at present, but infrared pedestrian detection requires pedestrian detection in infrared images, and cannot directly use the original low-light image detection.

SUMMARY OF THE INVENTION

The technical problem solved by the present invention is to provide a low-light pedestrian detection method and system based on multi-task learning, so as to solve the problem of poor pedestrian detection effect in a low-light environment.

In order to solve the above technical problems, the present invention provides a low-light pedestrian detection method based on multi-task learning, which includes the following steps:

S1. Obtain a normal light pedestrian dataset and a low light pedestrian dataset;

S2. Build a lighting enhancement network. The lighting enhancement network includes a decomposition network and an enhancement network. The lighting enhancement network is trained by using the normal lighting pedestrian dataset and the low lighting pedestrian dataset to obtain a lighting enhancement pre-training model;

S3. Build a pedestrian detection network. The pedestrian detection network uses two hourglass networks as the backbone network, and adds a spatial transformation network and a squeeze excitation network respectively. The pedestrian detection network is trained using the normal illumination pedestrian data set, and a pedestrian detection pre-training model is obtained. ;

S4. Based on multi-task learning, design a multi-task learning module that can fuse the features between different tasks, and share the features of the illumination enhancement network and the pedestrian detection network, in which the features of the first 3*3 convolutional network of the enhanced network will be enhanced. The features of the last spatial transformation network of the pedestrian detection network are added and fed back to the two networks, and the features of the last 3*3 convolutional network of the enhanced network and the features of the first residual module of the pedestrian detection network are also Add and feed back to the two networks to construct a low-light pedestrian detection network with multi-task feature sharing;

S5. Import the illumination enhancement pre-training model and the pedestrian detection pre-training model into the multi-task feature sharing low-light pedestrian detection network, and use the normal-light pedestrian dataset and the low-light pedestrian dataset for multi-task feature sharing. Low-light pedestrian detection The network is trained to obtain a low-light pedestrian detection model with multi-task feature sharing;

S6, using the low-light pedestrian detection model shared by multi-task features to detect the image to be detected, and obtain the position of the pedestrian in the image.

Further, in step S2: construct an illumination enhancement network based on the RetinexNet convolutional neural network. The loss function of the illumination enhancement network is:

L _enh =L _recon +λ _ir L _ir +λ _is L _is

where λ _ir and λ _is the weight coefficients, L _recon , L _ir and L _is represent the reconstruction, reflectance and illumination smoothness loss functions, respectively.

Further, in step S3: construct a pedestrian detection network based on CornerNet-Saccade. The loss function of the pedestrian detection network is:

L _cor =L _det +δL _pull +ηL _push +γL _off

In the formula, δ, η and γ are the weights of the three loss functions L _pull , L _push and L _off respectively,

in,

where L _det is the corner loss, N is the number of objects in the image, α and β are hyperparameters that control the contribution of each corner, C, H and W represent the number of input channels, height and width, respectively, p _aij is the score at the (i, j) position of class a in the predicted image, and y _aij is the unnormalized original image;

where L _off is the offset loss, ok is the offset, x _k and y _k are the x and y coordinates of the corner _k , and n is the downsampling factor;

为对象m的左上角的嵌入，为

为对象的右下角的嵌入，e_m为

和

的平均值。In the formula, L _pull is used to group the corners, L _push is used to separate the corners, m represents the number of objects,

is the embedding of the upper left corner of the object m, which is

is the embedding of the lower right corner of the object, and _em is

and

average of.

Further, the total training loss function of the low-light pedestrian detection network with multi-task feature sharing is:

L=L _det +L _cor =L _det +δL _pull +ηL _push +γL _off +ζL _enh

where L is the total loss, and ζ is the weight of the illumination enhancement loss L _enh .

The present invention also provides a low-light pedestrian detection system based on multi-task learning for realizing the above-mentioned multi-task learning-based low-light pedestrian detection method, including:

The dataset module is used to obtain the normal light pedestrian dataset and the low light pedestrian dataset;

The illumination enhancement module is used to construct an illumination enhancement network. The illumination enhancement network includes a decomposition network and an enhancement network. The illumination enhancement network is trained by using the normal illumination pedestrian dataset and the low illumination pedestrian dataset to obtain the illumination enhancement pre-training model;

The pedestrian detection module is used to construct a pedestrian detection network. The pedestrian detection network uses two hourglass networks as the backbone network, and adds a spatial transformation network and a squeeze excitation network respectively. The pedestrian detection network is trained using the normal illumination pedestrian dataset to obtain pedestrians. Detect pretrained models;

The multi-task learning module is used for feature sharing between the illumination enhancement network and the pedestrian detection network. The features of the first 3*3 convolutional network of the enhanced network and the features of the last spatial transformation network of the pedestrian detection network are added and merged. Feedback to the two networks, the features of the last 3*3 convolutional network of the enhanced network and the features of the first residual module of the pedestrian detection network are also added and fed back to the two networks to build a shared multi-task feature. Low-light pedestrian detection network;

The model training module is used to import the illumination enhancement pre-training model and the pedestrian detection pre-training model into the multi-task feature sharing low-light pedestrian detection network, and use the normal illumination pedestrian dataset and the low-light pedestrian dataset to share the multi-task features. The low-light pedestrian detection network is trained to obtain a low-light pedestrian detection model with multi-task feature sharing;

The image detection module is used to detect the image to be detected by using the low-light pedestrian detection model shared by multi-task features, and obtain the position of the pedestrian in the image.

Further, an illumination enhancement network is constructed based on the RetinexNet convolutional neural network.

Further, a pedestrian detection network is constructed based on CornerNet-Saccade.

The present invention also provides a computer storage medium, in which a computer program executable by a computer processor is stored, and the computer program executes the above-mentioned low-light pedestrian detection method based on multi-task learning.

The beneficial effects of the present invention are as follows: the present invention provides a low-light pedestrian detection method and system based on multi-task learning, which can accurately and efficiently detect the position of pedestrians in low-light images; Two modules of spatial transformation network and squeeze excitation network are added to the network-hourglass network, which improves the spatial and channel attention of the network, thereby improving the detection performance.

Description of drawings

FIG. 1 is a flowchart of the low-light pedestrian detection method based on multi-task learning of the present invention.

FIG. 2 is a structure diagram of the low-light pedestrian detection network based on multi-task learning of the present invention.

FIG. 3 is a schematic diagram of the low-light pedestrian detection system based on multi-task learning of the present invention.

FIG. 4 is a comparison chart of the test results of the embodiment of the present invention.

Detailed ways

The low-light pedestrian detection method and system based on multi-task learning of the present invention will be further described below in conjunction with the accompanying drawings:

The present invention is mainly divided into four parts: a pre-training model for illumination enhancement, a pre-training model for pedestrian detection, a low-light pedestrian detection model shared by multi-task features, and a low-light pedestrian detection model shared by multi-task features to infer images from low-light images Pedestrian location.

The low-light pedestrian detection method based on multi-task learning according to the embodiment of the present invention, as shown in FIG. 1 , includes the following steps:

S1. Obtain a normal light pedestrian dataset and a low light pedestrian dataset.

It can be obtained directly by taking pictures; it is also possible to reduce the illumination of the pedestrian detection training set and validation set of CityPersons (a public large-scale pedestrian detection data set). Here we use OpenCV (a cross-platform computer vision library based on BSD license (open source)). ) based on the RGB space brightness adjustment algorithm, this algorithm is adjusted based on the current RGB value, that is, the larger the R, G, and B values, the larger the adjustment, for example: the current pixel is (100, 200, 50), and the adjustment coefficient is 1.1, then adjusted to (110,220,55). In this example, we use an adjustment factor of 0.8. After reducing the brightness, we use the normal and low-light CityPersons pedestrian detection training and validation sets as our training and testing sets.

S2. Build a lighting enhancement network. The lighting enhancement network includes a decomposition network and an enhancement network. The lighting enhancement network is trained by using the normal lighting pedestrian dataset and the low lighting pedestrian dataset to obtain a lighting enhancement pre-training model.

The illumination enhancement network is separately trained for 100 cycles with the normal and low-light pedestrian detection training sets, and the illumination enhancement pre-training model is obtained. The network structure of the illumination enhancement network is shown in Figure 2. The illumination enhancement network is constructed based on the RetinexNet convolutional neural network, which introduces the Retinex theory. The classic Retinex theory establishes a model of human color perception, which assumes that the observed image can be decomposed into two parts, the reflection channel and the illumination channel. Let S represent the source image, then it can be represented by S=R*I, where R represents the reflectance component, I represents the illumination component, and * represents element-wise multiplication. For the loss function, in order to ensure that the restored image can retain the edge information of the object and also retain the smooth transition of the illumination information, the following loss function is used in the illumination enhancement network:

L _enh =L _recon +λ _ir L _ir +λ _is L _is

Among them, _λir and _λis represent coefficients that balance reflectivity and illuminance. The loss functions L _recon , L _ir and L _is represent reconstruction, reflectance and illumination smoothness functions, respectively.

S3. Build a pedestrian detection network. The pedestrian detection network uses two hourglass networks as the backbone network, and adds a spatial transformation network and a squeeze excitation network respectively. The pedestrian detection network is trained using the normal illumination pedestrian data set, and a pedestrian detection pre-training model is obtained. .

The pedestrian detection network is trained separately for 100 cycles from the pedestrian detection training set of normal illumination, and the pedestrian detection pre-training model is obtained. The network structure of the pedestrian detection network is shown in Figure 2. The pedestrian detection network is constructed based on CornerNet-Saccade. For the pedestrian detection network, the idea of object detection algorithm based on key points is referenced. The object detection algorithm based on key points generates object bounding boxes by detecting and grouping its key points, which greatly simplifies the output of the network and eliminates the need to design anchor boxes. At the same time, an attention mechanism is introduced into the network to further improve the detection performance. This patent uses the hourglass network as the backbone network of the pedestrian detection network, and adds two modules, the Spatial Transformation Network (STN) and the Squeeze Excitation Network (SENet), to the backbone network-hourglass network to improve the spatial and channel attention of the network. the detection performance. Second, we pruned the structure of the hourglass network from the original 54-layer network to a 36-layer network structure, which greatly reduces the amount of network parameters and reduces the cost of network training.

For the loss function of the pedestrian detection network, the focal loss function of α=2, β=4 can be used, let p _aij be the score at the (i, j) position of class a in the predicted image, and let y _aij be the unreported Normalized original image.

where N is the number of objects in the image, and α and β are hyperparameters that control the contribution of each point. C, H and W represent the number of channels, height and width of the input, respectively.

其中n是下采样因子。当我们将位置从热图重新映射到输入图像时，可能会损失一些精度。为了解决这个问题，我们预测了位置偏移，这会稍微调整角点位置，然后再将它们重新映射到输入分辨率。When the input image is passed through a convolutional layer, the size of the output is usually smaller than the input image. So the position (x,y) in the image maps to the position in the heatmap

where n is the downsampling factor. Some precision may be lost when we remap the locations from the heatmap to the input image. To address this, we predict position offsets, which slightly adjust the corner positions before remapping them to the input resolution.

where o _k is the offset, and x _k and y _k are the x and y coordinates of the corner k. We predict that one set of offsets is shared by the top-left corner of all classes, and the other set is shared by the bottom-right corner. For training, label the offset loss as L _off , and apply a smoothed L1 loss as the offset loss:

为对象m的左上角的嵌入，为

为对象的右下角的嵌入。使用“pull”损失来训练网络以对角进行分组，并使用“push”损失来对角进行分离：There may be multiple objects in the image, so multiple upper-left and lower-right corners can be detected, so it is necessary to determine whether a pair of upper-left and lower-right corners are from the same bounding box. make

is the embedding of the upper left corner of the object m, which is

Embed for the bottom right corner of the object. Use the "pull" loss to train the network to group the corners, and use the "push" loss to separate the corners:

和

的平均值。行人检测的训练总损失如下：where _em is

and

average of. The total training loss for pedestrian detection is as follows:

L _cor =L _det +δL _pull +ηL _push +γL _off ,

Among them, δ, η and γ are the weights of the three loss functions L _pull , L _push and L _off respectively, and L _cor is the total loss of the pedestrian detection network.

S4. Based on multi-task learning, design a multi-task learning module that can fuse the features between different tasks, share the features of the illumination enhancement network and the pedestrian detection network, and combine the features of the first 3*3 convolutional network of the enhanced network and The features of the last spatial transformation network of the pedestrian detection network are added and fed back to the two networks, and the features of the last 3*3 convolutional network of the enhanced network and the features of the first residual module of the pedestrian detection network are also processed. Add and feed back to the two networks to build a low-light pedestrian detection network with multi-task feature sharing.

The low-light pedestrian detection network with multi-task feature sharing includes: illumination enhancement network and pedestrian detection network. In order to further improve the performance of the two networks, a multi-task learning module is introduced on this basis. The detailed structure of the multi-task learning module is shown in the figure 2 shown. Among them, the features a _{1 and a 2 of the enhanced network are respectively added to the features b 1 and} b ₂ in the hourglass network + STN module, and the _two added features are marked as F ₁ and F ₂ , and Feedback to both networks. Features a ₁ and a ₂ come from the first 3*3 convolutional layer and the last 3*3 convolutional layer in the augmentation network, respectively. Feature b ₁ comes from the hourglass network + the last STN module in the STN, and feature b ₂ comes from the first residual module. At the same time, these features are of the same size. F1 and _F2 are represented as _:

F ₁ =a ₁ +b ₁ , F ₂ =a ₂ +b ₂ .

Finally, the training loss function L of the low-light pedestrian detection network with multi-task feature sharing is as follows:

L=L _det +δL _pull +ηL _push +γL _off +ζL _enh ,

where δ, η and γ are the weights of the three loss functions L _pull , L _push and L _off , respectively. ζ is the weight of the light enhancement loss L _enh . We set δ and η to 0.1, γ to 1, and ζ to 0.05.

S5. Import the illumination enhancement pre-training model and the pedestrian detection pre-training model into the multi-task feature sharing low-light pedestrian detection network, and use the normal-light pedestrian dataset and the low-light pedestrian dataset for multi-task feature sharing. Low-light pedestrian detection The network is trained to obtain a low-light pedestrian detection model with multi-task feature sharing.

The normal and low-light pedestrian detection training sets are used to train the low-light pedestrian detection network shared by multi-task features, and at the same time, the previously trained illumination enhancement pre-training model and pedestrian detection pre-training model are imported, and the multi-task features are obtained by training for 100 cycles. Shared low-light pedestrian detection model.

S6, using the low-light pedestrian detection model shared by multi-task features to detect the image to be detected, and obtain the position of the pedestrian in the image. The low-light test set images are input into the trained multi-task feature sharing low-light pedestrian detection model for inference, and the location of the pedestrian is framed in the image.

The present invention also provides a low-light pedestrian detection system based on multi-task learning for realizing the above-mentioned multi-task learning-based low-light pedestrian detection method, as shown in FIG. 3 , including:

The dataset module 101 is used to obtain a normal lighting pedestrian dataset and a low lighting pedestrian dataset;

The illumination enhancement module 102 is used to construct an illumination enhancement network, the illumination enhancement network includes a decomposition network and an enhancement network, and the illumination enhancement network is trained by using the normal illumination pedestrian dataset and the low illumination pedestrian dataset to obtain an illumination enhancement pre-training model;

The pedestrian detection module 103 is used to construct a pedestrian detection network. The pedestrian detection network uses two hourglass networks as the backbone network, and adds a spatial transformation network and a squeeze excitation network respectively, and uses the normal illumination pedestrian dataset to train the pedestrian detection network to obtain Pedestrian detection pre-training model;

The multi-task learning module 104 is used to fuse the features between different tasks, share the features of the illumination enhancement network and the pedestrian detection network, and combine the characteristics of the first 3*3 convolutional network of the enhancement network with the characteristics of the pedestrian detection network. The features of the last spatial transformation network are added and fed back to the two networks. The features of the last 3*3 convolutional network of the enhancement network and the features of the first residual module of the pedestrian detection network are also added and fed back. For two networks, build a low-light pedestrian detection network with multi-task feature sharing;

The model training module 105 is used to import the illumination enhancement pre-training model and the pedestrian detection pre-training model into the multi-task feature sharing low-light pedestrian detection network, and use the normal illumination pedestrian data set and the low-light pedestrian data set to share the multi-task features The low-light pedestrian detection network is trained to obtain a low-light pedestrian detection model with multi-task feature sharing;

The image detection module 106 is configured to detect the image to be detected by using the low-light pedestrian detection model shared by multi-task features, and obtain the position of the pedestrian in the image.

Further, the illumination enhancement network is constructed based on the RetinexNet convolutional neural network; the pedestrian detection network is constructed based on the CornerNet-Saccade.

The present invention also provides a computer storage medium, in which a computer program executable by a computer processor is stored, and the computer program executes the above-mentioned low-light pedestrian detection method based on multi-task learning.

The present invention finally provides a test example, using normal and low-light CityPersons data sets, including 2975 pictures each in the normal and low-light training sets, and 500 pictures in each of the normal and low-light test sets. The experiments were implemented in Pytorch and trained using 2 RTX 2080Ti graphics cards, while applying the Adam optimization algorithm. In the selection of experimental parameters, the learning rate is set to 0.0001. The evaluation metrics follow Caltech's evaluation criteria: log mean miss rate (MR ^-2 ) per image, with lower values of MR ^-2 indicating better algorithm performance. The superiority of the present invention is proved by comparing with other excellent pedestrian detection algorithms through the MR ^-2 evaluation index. Table 1 shows the comparison results through the MR ^-2 evaluation index, and Figure 4 is a comparison diagram of pedestrian position detection results.

The pedestrian detection methods chosen for comparison include: CSP, ALFNet, and CornerNet-Saccade. ALFNet is the most representative anchor box-based algorithm in pedestrian detection, and CSP is the best algorithm using center point method in pedestrian detection. Meanwhile, CornerNet-Saccade is the best algorithm based on keypoint method in object detection. Two different models are proposed in CSP, one is the CSPoff model that considers the offset, and the other is the CSPnooff model that does not consider the offset. The first four rows of algorithms in Table 1 all train the algorithms on the normal and low-light training sets, so all these pedestrian detection networks have the ability to process low-light images, ensuring the fairness of the experiments. In addition, in order to further illustrate the role of the multi-task learning module in the patented algorithm, the algorithm in the fifth row in Table 1 uses a cascaded method to cascade the RetinexNet illumination enhancement algorithm and the CornerNet-Saccade detection algorithm. It can be seen from the results in the table that the index of the cascade method is still not better than the algorithm of the present invention, which proves that the multi-task learning module plays an important role.

Table 1 The present invention and five excellent algorithms comparison result table

From the experimental results in the table above, it can be seen that this algorithm has obvious advantages compared with the other five methods.

The parts not described in the specification are the prior art or common knowledge. This embodiment is only used to illustrate the invention, but not to limit the scope of the invention. Those skilled in the art make modifications such as equivalent replacement of the invention, which are considered to fall within the protection scope of the claims of the invention.

Those skilled in the art can easily understand that the above are only preferred embodiments of the present invention, and are not intended to limit the present invention. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present invention should be Included in the protection scope of the present invention.

Claims (10)

1. A low-illumination pedestrian detection method based on multitask learning is characterized by comprising the following steps:

s1, acquiring a normal illumination pedestrian data set and a low illumination pedestrian data set;

s2, constructing an illumination enhancement network, wherein the illumination enhancement network comprises a decomposition network and an enhancement network, and the illumination enhancement network is trained by utilizing a normal illumination pedestrian data set and a low illumination pedestrian data set to obtain an illumination enhancement pre-training model;

s3, constructing a pedestrian detection network, wherein the pedestrian detection network takes two hourglass networks as a backbone network, respectively adds a space conversion network and an extrusion excitation network, and trains the pedestrian detection network by using a normal illumination pedestrian data set to obtain a pedestrian detection pre-training model;

s4, designing a multi-task learning module capable of fusing features among different tasks based on multi-task learning, and sharing the features of the illumination enhancement network and the pedestrian detection network, wherein the features of the first 3 x 3 convolution network of the enhancement network and the features of the last space conversion network of the pedestrian detection network are added and fed back to the two networks, and the features of the last 3 x 3 convolution network of the enhancement network and the features of the first residual module of the pedestrian detection network are also added and fed back to the two networks, so that the low-illumination pedestrian detection network shared by the multi-task features is constructed;

s5, importing the illumination enhancement pre-training model and the pedestrian detection pre-training model into a low-illumination pedestrian detection network shared by the multitask features, and training the low-illumination pedestrian detection network shared by the multitask features by utilizing a normal illumination pedestrian data set and a low illumination pedestrian data set to obtain a low-illumination pedestrian detection model shared by the multitask features;

and S6, detecting the image to be detected by using the low-illumination pedestrian detection model shared by the multitask characteristics to obtain the position of the pedestrian in the image.

2. The low-light pedestrian detection method based on multitask learning according to claim 1, wherein in step S2: and constructing an illumination enhancement network based on the RetinexNet convolutional neural network.

3. The low-illumination pedestrian detection method based on multitask learning according to claim 2, characterized in that the loss function of the illumination enhancement network is:

L_enh＝L_recon+λ_irL_ir+λ_isL_is

in the formula, λ_irAnd λ_isIs a weight coefficient, L_recon，L_irAnd L_isRepresenting reconstruction, reflectance and illumination smoothness loss functions, respectively.

4. The low-light pedestrian detection method based on multitask learning according to claim 3, wherein in step S3: and constructing a pedestrian detection network based on CornerNet-Saccade.

5. The low-light pedestrian detection method based on multitask learning according to claim 4, characterized in that the loss function of said pedestrian detection network is:

L_cor＝L_det+L_pull+ηL_push+γL_off

wherein eta and gamma are L_pull，L_pushAnd L_offThe weights of the three loss functions are such that,

wherein,

in the formula, L_detFor corner loss, N is the number of objects in the image, α and β are the hyperparameters controlling the contribution of each corner, C, H and W represent the number of channels, height and width, respectively, input, p_aijIs the score at the (i, j) position of class a in the predicted image, y_aijIs an original image without normalization;

in the formula, L_offFor offset loss, o_kIs an offset, x_kAnd y_kIs the x and y coordinates of the corner point k, n is the down-sampling factor;

in the formula, L_pullFor grouping corners, L_pushThe corners are separated, m represents the number of objects,

for the embedding of the upper left corner of object m, are

For embedding in the lower right corner of the object, e_mIs composed of

And

average value of (a).

6. The low-light pedestrian detection method based on multitask learning according to claim 5, wherein the overall training loss function of the low-light pedestrian detection network shared by multitask features is as follows:

L＝L_det+L_cor＝L_det+L_pull+ηL_push+γL_off+ζL_enh

wherein L is the total loss, and ζ is the light enhancement loss L_enhThe weight of (c).

7. A low-light pedestrian detection system based on multitask learning for implementing the low-light pedestrian detection method based on multitask learning according to claim 1, characterized by comprising:

the data set module is used for acquiring a normal illumination pedestrian data set and a low illumination pedestrian data set;

the illumination enhancement module is used for constructing an illumination enhancement network, the illumination enhancement network comprises a decomposition network and an enhancement network, and the illumination enhancement network is trained by utilizing a normal illumination pedestrian data set and a low illumination pedestrian data set to obtain an illumination enhancement pre-training model;

the pedestrian detection module is used for constructing a pedestrian detection network, the pedestrian detection network takes two hourglass networks as a backbone network, and respectively adds a space conversion network and an extrusion excitation network, and the pedestrian detection network is trained by utilizing a normal illumination pedestrian data set to obtain a pedestrian detection pre-training model;

the multitask learning module is used for sharing the features of the illumination enhancement network and the pedestrian detection network, adding the features of the first 3 x 3 convolution network of the enhancement network and the features of the last space conversion network of the pedestrian detection network and feeding back the added features to the two networks, adding the features of the last 3 x 3 convolution network of the enhancement network and the features of the first residual module of the pedestrian detection network and feeding back the added features to the two networks, and constructing the low-illumination pedestrian detection network shared by the multitask features;

the model training module is used for importing an illumination enhancement pre-training model and a pedestrian detection pre-training model into a multi-task feature-shared low-illumination pedestrian detection network, and training the multi-task feature-shared low-illumination pedestrian detection network by utilizing a normal illumination pedestrian data set and a low illumination pedestrian data set to obtain the multi-task feature-shared low-illumination pedestrian detection model;

and the image detection module is used for detecting the image to be detected by utilizing the low-illumination pedestrian detection model shared by the multitask characteristics to obtain the position of the pedestrian in the image.

8. The low-illumination pedestrian detection system based on multitask learning according to claim 7, characterized in that the illumination enhancement network is constructed based on a RetinexNet convolutional neural network.

9. The low-light pedestrian detection system based on multitask learning according to claim 7, wherein the pedestrian detection network is constructed based on CornerNet-Saccade.

10. A computer storage medium, characterized in that: stored therein is a computer program executable by a computer processor, the computer program performing the low-light pedestrian detection method based on multi-task learning according to any one of claims 1 to 6.

Images