[go: up one dir, main page]

CN104778453B - A kind of night pedestrian detection method based on infrared pedestrian's brightness statistics feature - Google Patents

A kind of night pedestrian detection method based on infrared pedestrian's brightness statistics feature Download PDF

Info

Publication number
CN104778453B
CN104778453B CN201510154382.8A CN201510154382A CN104778453B CN 104778453 B CN104778453 B CN 104778453B CN 201510154382 A CN201510154382 A CN 201510154382A CN 104778453 B CN104778453 B CN 104778453B
Authority
CN
China
Prior art keywords
pedestrian
mrow
image
infrared
frame
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related
Application number
CN201510154382.8A
Other languages
Chinese (zh)
Other versions
CN104778453A (en
Inventor
徐向华
王路杰
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hangzhou Dianzi University
Original Assignee
Hangzhou Dianzi University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hangzhou Dianzi University filed Critical Hangzhou Dianzi University
Priority to CN201510154382.8A priority Critical patent/CN104778453B/en
Publication of CN104778453A publication Critical patent/CN104778453A/en
Application granted granted Critical
Publication of CN104778453B publication Critical patent/CN104778453B/en
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Landscapes

  • Closed-Circuit Television Systems (AREA)
  • Image Processing (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a kind of night pedestrian detection method based on infrared pedestrian's brightness statistics feature.The present invention counts gray average information to each part of pedestrian in Sample Storehouse and negative sample first, and mapping range border is determined using the information, constructs a brightness histogram feature for distinguishing ballot interval division;Then gradient orientation histogram feature is calculated, and the two features are subjected to joint and form final feature descriptor;Secondly model training is carried out using the method for Adaboost combination decision trees, and pedestrian's judgement and positioning are carried out by sliding window scanning method, finally when grader is obtained compared with low confidence to the classification judgement of some detection block, judged again using brightness section template, so as to realize the pedestrian detection at night.The present invention have effectively achieved the pedestrian detection under night environment, have the characteristics of high verification and measurement ratio, strong adaptability.

Description

一种基于红外行人亮度统计特征的夜间行人检测方法A Pedestrian Detection Method at Night Based on Infrared Pedestrian Brightness Statistical Features

技术领域technical field

本发明涉及车载视频图像的行人检测方法,特别一种基于红外行人亮度统计特征的夜间行人检测方法。The invention relates to a pedestrian detection method of a vehicle-mounted video image, in particular to a pedestrian detection method at night based on the statistical characteristics of infrared pedestrian brightness.

背景技术Background technique

行人检测技术是计算机视觉的一个重要应用,在日常的生活和生产中有很高的实用价值。基于视觉的行人检测就是根据一定图像处理技术从输入的图片或者视频帧序列判断行人出现的具体位置。智能车辆辅助驾驶系统可以提高车辆驾驶的安全性,从而减少交通事故的发生,行人检测技术则是智能辅助驾驶系统中的核心技术之一。Pedestrian detection technology is an important application of computer vision, which has high practical value in daily life and production. Pedestrian detection based on vision is to judge the specific location of pedestrians from the input picture or video frame sequence according to certain image processing technology. The intelligent vehicle assisted driving system can improve the safety of vehicle driving, thereby reducing the occurrence of traffic accidents, and pedestrian detection technology is one of the core technologies in the intelligent assisted driving system.

基于视觉的夜间行人检测技术主要采用的是可见光图像、红外图像等技术。在夜晚情况下,由于光照等条件不理想,可见光摄像机的成像效果较差,影响行人检测的效果。红外摄像机通过被动红外技术捕获红外线信息感知物体,不同温度的物体在图像中呈现出不同亮度。在道路场景红外图像中,行人一般比背景辐射更多的热量,红外图像中的行人一般比背景更为明亮,且不受夜晚光线阴暗、雾天视线不清等影响,具有良好的夜视能力,对不同的光照环境都有较强的适应能力。因此红外图像的行人检测技术是实现夜间行人检测的有效解决方法。Vision-based pedestrian detection technology at night mainly uses visible light images, infrared images and other technologies. At night, due to unsatisfactory lighting and other conditions, the imaging effect of the visible light camera is poor, which affects the effect of pedestrian detection. Infrared cameras capture infrared information to perceive objects through passive infrared technology, and objects with different temperatures show different brightness in the image. In the infrared image of the road scene, pedestrians generally radiate more heat than the background, and the pedestrians in the infrared image are generally brighter than the background, and are not affected by dark light at night, unclear vision in foggy days, etc., and have good night vision ability , have strong adaptability to different lighting environments. Therefore, the pedestrian detection technology of infrared images is an effective solution to realize pedestrian detection at night.

目前多数红外行人检测技术采用基于机器学习的方法,如发明专利《基于特征组合的行人检测方法及装置》(CN103632170A)利用HOG特征与LBP特征构成一个联合特征,利用支持向量机(SVM)作为学习算法进行分类器训练实现行人检测。上述方法涉及的LBP特征对图像纹理有较强描述能力,而红外图像的纹理特征并不明显,因此LBP特征应用于红外行人检测中的效果一般。发明专利《一种基于红外图像的行人检测方法》(CN103902976A)在HOG特征基础上融合与亮度直方图特征描述行人,利用SVM进行分类器训练实现夜间行人检测。该方法除了利用行人的轮廓信息外,也对红外行人的亮度信息进行了提取,因此,有更好的红外行人检测效果。At present, most infrared pedestrian detection technologies use machine learning-based methods. For example, the invention patent "Pedestrian Detection Method and Device Based on Feature Combination" (CN103632170A) uses HOG features and LBP features to form a joint feature, and uses support vector machines (SVM) as learning Algorithm for classifier training to achieve pedestrian detection. The LBP feature involved in the above method has a strong ability to describe the image texture, but the texture feature of the infrared image is not obvious, so the effect of the LBP feature applied to infrared pedestrian detection is general. Invention patent "A Pedestrian Detection Method Based on Infrared Image" (CN103902976A) based on HOG feature fusion and brightness histogram features to describe pedestrians, using SVM for classifier training to realize nighttime pedestrian detection. In addition to using the outline information of pedestrians, this method also extracts the brightness information of infrared pedestrians, so it has a better detection effect of infrared pedestrians.

上述方法并没有考虑红外行人的亮度分布统计特征。本发明在HOG特征基础上,融合红外行人的亮度分布统计特征,构建一种描述能力更强的红外行人特征,利用Adaboost作为学习算法,设计行人检测分类器以及亮度区间模板再判定的行人行人框架。本发明的方法在红外行人检测中有更好的行人特征描述能力和检测效果。The above methods do not consider the statistical characteristics of the brightness distribution of infrared pedestrians. Based on the HOG feature, the present invention combines the statistical characteristics of the brightness distribution of infrared pedestrians to construct an infrared pedestrian feature with stronger description ability, uses Adaboost as a learning algorithm, and designs a pedestrian detection classifier and a pedestrian framework for re-determining the brightness interval template . The method of the invention has better pedestrian feature description ability and detection effect in infrared pedestrian detection.

发明内容Contents of the invention

本发明的目的是针对现有技术的不足,提供一种基于红外行人亮度统计特征的夜间行人检测方法。本发明利用红外行人的轮廓信息和亮度信息,首先通过对样本库中的行人部件(包括头部、上半身、下半身)和负样本图像统计灰度均值信息,根据灰度均值信息的分布特征确定行人描述特征的各投票映射区间,构造一个区分投票区间划分的亮度直方图特征(DBHOI);然后计算梯度方向直方图(HOG)特征,并将这两个特征进行联合构成最终的行人特征描述符;其次,利用Adaboost结合决策树的方法进行模型训练,通过滑窗扫描法进行行人判定及定位;最后,当分类器对某个行人框分类判断得到较低置信度时,采用亮度区间模板进行再次判定,从而实现夜间红外行人检测。该方法具有检测率高、适应性强的特点。The purpose of the present invention is to provide a pedestrian detection method at night based on statistical characteristics of infrared pedestrian brightness. The present invention utilizes the profile information and brightness information of infrared pedestrians, firstly calculates the average gray value information of the pedestrian parts (including head, upper body, lower body) and negative sample images in the sample library, and determines the pedestrian according to the distribution characteristics of the gray average value information Describe each voting mapping interval of the feature, construct a brightness histogram feature (DBHOI) that distinguishes the voting interval division; then calculate the gradient orientation histogram (HOG) feature, and combine these two features to form the final pedestrian feature descriptor; Secondly, use Adaboost combined with the decision tree method for model training, and use the sliding window scanning method to determine and locate pedestrians; finally, when the classifier obtains a low confidence level for the classification and judgment of a certain pedestrian frame, it uses the brightness interval template for re-judgment , so as to realize infrared pedestrian detection at night. This method has the characteristics of high detection rate and strong adaptability.

本发明解决其技术问题采用的技术方案步骤如下:The technical solution steps adopted by the present invention to solve its technical problems are as follows:

步骤1、构建红外图像的正负样本数据集;Step 1. Construct positive and negative sample data sets of infrared images;

步骤2、基于亮度统计特征的红外图像DBHOI特征构造;Step 2, infrared image DBHOI feature construction based on brightness statistical features;

步骤3、基于轮廓信息的红外图像HOG特征构造;Step 3, infrared image HOG feature construction based on contour information;

步骤4、利用Adaboost进行分类器训练;Step 4, utilize Adaboost to carry out classifier training;

步骤5、检测窗口判定及定位;Step 5, detection window judgment and positioning;

步骤6、行人框重判定。Step 6: Pedestrian frame weight determination.

步骤1所述的构建红外图像的正负样本数据集具体如下:The details of constructing the positive and negative sample data sets of infrared images described in step 1 are as follows:

正样本数据集构建方法如下:采用最小矩形窗口,提取红外图像中的行人样本;假设行人的高为h,宽为w,使得w/h=0.41;共提取行人正样本N1张;The method of constructing the positive sample data set is as follows: use the smallest rectangular window to extract pedestrian samples in the infrared image; assume that the height of the pedestrian is h and the width is w, so that w/h=0.41; a total of N 1 positive samples of pedestrians are extracted;

负样本数据集构建方法如下:随机在N0张不包含行人的红外图像中共抽取N0×10张负样本,即在每张红外图像中随机抽取10张负样本;The negative sample data set construction method is as follows: Randomly extract N 0 × 10 negative samples from N 0 infrared images that do not contain pedestrians, that is, randomly select 10 negative samples from each infrared image;

将所有正负样本缩放至宽为64像素,高为128像素的样本图像。Scales all positive and negative samples to a sample image with a width of 64 pixels and a height of 128 pixels.

步骤2所述的基于亮度统计特征的红外图像DBHOI特征构造具体如下:The infrared image DBHOI feature construction based on the brightness statistical feature described in step 2 is specifically as follows:

2-1.设样本图像尺寸为w′×h′,则将样本图像划分成多份大小相同的局部图像;每一个局部图像的大小为8×8,即将样本图像划分成相等的(w′/8)×(h′/8)个局部图像,并将这些局部图像记为cell;2-1. Set the size of the sample image as w′×h′, then divide the sample image into multiple partial images of the same size; the size of each partial image is 8×8, that is, divide the sample image into equal (w′ /8)×(h′/8) partial images, and record these partial images as cells;

2-2.根据行人部件确定映射规则;2-2. Determine the mapping rules according to the pedestrian components;

2-2-1.首先根据N1个正样本截取行人各部件,行人各部件包括头部、上半身、下半身;2-2-1. Firstly, according to N 1 positive samples, each part of the pedestrian is intercepted, and each part of the pedestrian includes the head, upper body, and lower body;

2-2-2.在负样本中随机截取N1个背景图像;2-2-2. Randomly intercept N 1 background images in the negative sample;

2-2-3.分别计算头部图像、上半身图像、下半身图像和背景图像的灰度均值;2-2-3. Calculate the gray mean value of the head image, upper body image, lower body image and background image respectively;

2-2-4.设四种图像的灰度均值为G1、G2、G3、G4,且大小依次增大;,根据四个灰度均值确定三个映射边界,分别为t1=(G1+G2)/2、t2=(G2+G3)/2、t3=(G3+G4)/2,根据映射边界将灰度值范围划分为四个灰度区间,分别为[0,t1),[t1,t2),[t2,t3),[t3,255];2-2-4. Set the average gray values of the four images as G 1 , G 2 , G 3 , and G 4 , and the sizes increase sequentially; and determine three mapping boundaries according to the four average gray values, respectively t 1 =(G 1 +G 2 )/2, t 2 =(G 2 +G 3 )/2, t 3 =(G 3 +G 4 )/2, divide the gray value range into four gray values according to the mapping boundary degree interval, respectively [0,t 1 ),[t 1 ,t 2 ),[t 2 ,t 3 ),[t 3 ,255];

2-2-5.根据该映射规则并以红外图像中像素的灰度值为权值,在每个cell内构建亮度直方图,得到一个四维的特征向量;2-2-5. According to the mapping rule and using the gray value of the pixel in the infrared image as the weight, construct a brightness histogram in each cell to obtain a four-dimensional feature vector;

2-3.相邻的2×2个cell进行块内归一化;2-3. Adjacent 2×2 cells are normalized within the block;

以一个cell为步长,将(w′/8)×(h′/8)个cell在样本图像中以从上到下、从左到右的顺序将相邻的四个cell通过L1-Sqrt方法进行块内归一化;Taking one cell as the step size, put (w'/8)×(h'/8) cells in the sample image in order from top to bottom and from left to right to pass the adjacent four cells through L1-Sqrt method for intra-block normalization;

其中L1-Sqrt方法如下:v为亮度直方图向量,ε为一个很小值,取值为0.001;The L1-Sqrt method is as follows: v is the brightness histogram vector, ε is a very small value, the value is 0.001;

2-4.将所有块的特征进行串联得到DBHOI特征;2-4. Concatenate the features of all blocks to obtain the DBHOI features;

所述的DBHOI为不同区间大小的亮度直方图。The DBHOI is a brightness histogram with different interval sizes.

步骤3所述的于轮廓信息的红外图像HOG特征构造具体如下:The HOG feature structure of the infrared image based on the contour information described in step 3 is specifically as follows:

利用经典sobel算子计算每个像素水平和垂直方向的梯度分量Gx(i,j)、Gy(i,j),然后计算对应的梯度大小G(i,j)和方向D(i,j)如下:Use the classic sobel operator to calculate the gradient components G x (i, j) and G y (i, j) of each pixel in the horizontal and vertical directions, and then calculate the corresponding gradient size G (i, j) and direction D (i, j) as follows:

步骤4所述的利用Adaboost进行分类器训练具体如下:The classifier training using Adaboost described in step 4 is as follows:

4-1.将所有正负样本缩放至相同的尺度,然后针对每一个样本图像,提取DBHOI特征向量和HOG特征向量,并且标记正样本的标签为1,负样本的标签为-1;4-1. Scale all positive and negative samples to the same scale, then extract the DBHOI feature vector and HOG feature vector for each sample image, and mark the label of the positive sample as 1, and the label of the negative sample as -1;

4-2.将N1个正样本和N0×10个负样本对应的DBHOI-HOG特征向量以及样本标签输入到Adaboost学习算法进行训练,得到一个具有一系列弱分类器构成的分类器。4-2. Input the DBHOI-HOG feature vectors and sample labels corresponding to N 1 positive samples and N 0 ×10 negative samples to the Adaboost learning algorithm for training, and obtain a classifier composed of a series of weak classifiers.

步骤5所述的检测窗口判定及定位如下:The determination and positioning of the detection window described in step 5 are as follows:

5-1.根据待检测红外图像的尺度和检测窗口的大小确定缩放因子,确定待检测红外图像的缩放层数;然后用步骤4得到的分类器按步长为4个像素大小进行逐层扫描;设待检测红外图像的原始大小为Wi×Hi,其中Wi表示待检测红外图像的宽度,Hi表示待检测红外图像高度,检测窗口大小为Wd×Hd,其中Wd表示检测窗口的宽度,Hd表示检测窗口高度,用ss表示缩放因子;则初始缩放因子为ss=1,终止缩放因子为ss=min{Wi/Wd,Hi/Hd};对每一个尺度下的待检测红外图像进行滑窗扫描,利用训练得到的行人分类器进行检测窗口判定;如果该检测窗口是行人框,则记录该检测窗口的位置以及置信度,将该记录表示为{posX,posY,width,height,score},其中posX,posY为行人框的左上角点,width,height为行人框的宽度和高度,score为置信度;5-1. Determine the scaling factor according to the scale of the infrared image to be detected and the size of the detection window, and determine the number of scaling layers of the infrared image to be detected; then use the classifier obtained in step 4 to scan layer by layer with a step size of 4 pixels ; Suppose the original size of the infrared image to be detected is W i ×H i , where W i represents the width of the infrared image to be detected, H i represents the height of the infrared image to be detected, and the size of the detection window is W d ×H d , where W d represents The width of the detection window, H d represents the height of the detection window, and s s represents the scaling factor; then the initial scaling factor is s s =1, and the termination scaling factor is s s =min{W i /W d ,H i /H d } ; Sliding window scanning is performed on the infrared image to be detected at each scale, and the detection window is judged by using the trained pedestrian classifier; if the detection window is a pedestrian frame, record the position and confidence of the detection window, and record the Expressed as {posX, posY, width, height, score}, where posX and posY are the upper left corner of the pedestrian frame, width and height are the width and height of the pedestrian frame, and score is the confidence level;

所述的置信度为所有二层决策树叶子结点记录的错误率;The confidence degree is the error rate recorded by all the leaf nodes of the two-layer decision tree;

5-2.通过对待检测红外图像进行多尺度的滑窗扫描检测后,同一个行人在不同尺度的待检测红外图像中被检测出来,采用非极大值抑制法对多个不同尺度的行人框结果进行融合;5-2. After multi-scale sliding window scanning detection of the infrared image to be detected, the same pedestrian is detected in different scales of the infrared image to be detected, and the non-maximum value suppression method is used to detect multiple pedestrian frames of different scales. The results are merged;

所述的极大值抑制的标准是每个行人框的置信度score;The criterion for maximum value suppression is the confidence score of each pedestrian frame;

5-3.将所有行人框按置信度从低到高进行排列成数组A[n],然后从该数组A[n]中取出当前置信度最大的检测窗口信息A[n];5-3. Arrange all the pedestrian frames into an array A[n] according to the confidence level from low to high, and then take out the detection window information A[n] with the highest current confidence level from the array A[n];

5-4.判断该行人框A[n]与后续行人框A[n-1]的关系,如果两个行人框的重合度α大于0.5,则认为是同一个行人框,否则将行人框A[n-1]作为当前置信度最大的行人框;5-4. Determine the relationship between the pedestrian frame A[n] and the subsequent pedestrian frame A[n-1]. If the coincidence degree α of the two pedestrian frames is greater than 0.5, it is considered to be the same pedestrian frame, otherwise the pedestrian frame A [n-1] as the pedestrian frame with the highest current confidence;

所述的重合度其中area(Bn)为第n个行人框的面积,area(Bn∩Bn-1)为第n个行人框面积与第n-1个行人框面积的交集;The degree of coincidence Where area(B n ) is the area of the nth pedestrian frame, area(B n ∩B n-1 ) is the intersection of the area of the nth pedestrian frame and the area of the n-1th pedestrian frame;

5-5.重复步骤5-4,直至判定完所有的行人框。5-5. Repeat steps 5-4 until all pedestrian frames are determined.

步骤6所述的行人框重判定如下:The pedestrian frame weight determination described in step 6 is as follows:

6-1.通过步骤5之后,每一个被判定为行人的检测窗口都会有一个置信度值score;采用了亮度区间模板对行人框进行再判定;当scorei≤τ时,则进行行人框再判定,其中阈值τ的取值由统计方式确定;如果scorei>τ,则仅用分类器进行行人检测,假设得到N个行人框的置信度,那么τ满足式 6-1. After passing step 5, each detection window that is judged as a pedestrian will have a confidence value score; the pedestrian frame is re-judged using the brightness interval template; when the score i ≤τ, the pedestrian frame is re-determined Judgment, where the value of the threshold τ is determined by a statistical method; if score i > τ, only the classifier is used for pedestrian detection, assuming that the confidence of N pedestrian frames is obtained, then τ satisfies the formula

所述的scorei为步骤5-3定义的数组A[n]中第i个的置信度值;The score i is the i-th confidence value in the array A[n] defined in step 5-3;

6-2.根据步骤2以及红外行人成像特点,[0,t1),[t1,t2),[t2,t3),[t3,255]这4个区间分别对应背景、上半身、下半身、头部;设行人宽高为w×h,如果行人框正确判定行人位置,那么将已经被判定为行人的行人框的高度h从上到下划分成4等分,则在第一个1/4等分处必定存在属于[t3,255]范围的亮度信息,并记灰度值在此区间的像素总个数为Σp1,第二个1/4等分处必定存在[t1,t2)范围的亮度信息,并记灰度值在此区间的像素总个数为Σp2,以及后两个1/4等分处必定存在[t2,t3)范围内的亮度信息,并记灰度值在此区间的像素总个数为Σp3;设置如下验证规则:Σp1/(w×h)≥1/16,Σp2/(w×h)≥1/8,Σp3/(w×h)≥1/16;如果Σp1、Σp2、Σp3同时满足这三个条件,那么该行人框再次被判定为行人框,否则该行人框判定为非行人框。本发明的有益效果:6-2. According to step 2 and the characteristics of infrared pedestrian imaging, the four intervals [0,t 1 ),[t 1 ,t 2 ),[t 2 ,t 3 ),[t 3 ,255] correspond to the background, Upper body, lower body, head; if the width and height of the pedestrian is w×h, if the pedestrian frame correctly determines the position of the pedestrian, then the height h of the pedestrian frame that has been judged as a pedestrian is divided into 4 equal parts from top to bottom, then at Brightness information belonging to the range [t 3 ,255] must exist in a 1/4 equal division, and the total number of pixels whose gray value is in this interval is Σp 1 , and the second 1/4 equal division must exist [t 1 , t 2 ) range of brightness information, and record the total number of grayscale pixels in this range as Σp 2 , and the last two 1/4 equal parts must exist in the range of [t 2 , t 3 ) luminance information, and record the total number of pixels with gray values in this interval as Σp 3 ; set the following verification rules: Σp 1 /(w×h)≥1/16, Σp 2 /(w×h)≥1/ 8. Σp 3 /(w×h)≥1/16; if Σp 1 , Σp 2 , and Σp 3 meet these three conditions at the same time, then the pedestrian frame is judged as a pedestrian frame again, otherwise the pedestrian frame is judged as a non-pedestrian frame. Beneficial effects of the present invention:

本发明针对红外图像中行人的特点,构建了一个更具表达能力的亮度特征描述符——DBHOI,并将它与HOG特征相结合。DBHOI描述符通过统计训练样本中行人各部件及背景的亮度分布,并在构造特征时对该分布信息进行编码,使得该特征描述符更能够刻画出行人与背景在亮度信息上的区别。该特征描述符的构造方式提高了特征的描述能力,从而使得最终的分类器具有更强的分类能力。Aiming at the characteristics of pedestrians in infrared images, the present invention constructs a more expressive brightness feature descriptor——DBHOI, and combines it with HOG features. The DBHOI descriptor counts the brightness distribution of pedestrian parts and background in training samples, and encodes the distribution information when constructing features, so that the feature descriptor can better describe the difference in brightness information between pedestrians and background. The construction method of the feature descriptor improves the description ability of the feature, so that the final classifier has a stronger classification ability.

当使用分类器对行人框进行判定后得到较低置信度时,则根据亮度区间模板进行行人框再判定。该方法大大地降低了因分类器误判而引起的错误率,提高了检测精度。When the classifier is used to determine the pedestrian frame and get a lower confidence, the pedestrian frame is re-determined according to the brightness interval template. This method greatly reduces the error rate caused by the misjudgment of the classifier and improves the detection accuracy.

附图说明Description of drawings

图1为本发明的整体流程图。Fig. 1 is the overall flow chart of the present invention.

图2为DBHOI特征描述符提取流程图。Figure 2 is a flow chart of DBHOI feature descriptor extraction.

图3为本发明行人检测模型的检测效果展示。Fig. 3 is a display of the detection effect of the pedestrian detection model of the present invention.

具体实施方式detailed description

下面结合附图,对本发明的具体实施方案作进一步详细描述。The specific embodiments of the present invention will be further described in detail below in conjunction with the accompanying drawings.

如图1、图2和图3所示,一种基于红外行人亮度统计特征的夜间行人检测方法,其具体包括如下步骤:As shown in Figure 1, Figure 2 and Figure 3, a night pedestrian detection method based on the statistical characteristics of infrared pedestrian brightness, which specifically includes the following steps:

步骤1、构建红外图像的正负样本数据集。Step 1. Construct positive and negative sample datasets of infrared images.

本发明主要针对车载情况下的行人检测,所以采集图像的场景主要是道路场景下的红外图像。The present invention is mainly aimed at the detection of pedestrians in the case of vehicles, so the scenes for collecting images are mainly infrared images in road scenes.

正样本数据集构建方法如下:采用最小矩形窗口,也就是说用一个能够刚好包围行人的矩形框,提取红外图像中的行人样本;假设行人的高为h,宽为w,使得w/h=0.41;一共提取行人正样本N1张;The method of constructing the positive sample data set is as follows: Use the smallest rectangular window, that is to say, use a rectangular frame that can just surround pedestrians to extract pedestrian samples in the infrared image; assume that the height of the pedestrian is h and the width is w, so that w/h = 0.41; A total of N 1 positive pedestrian samples were extracted;

负样本数据集构建方法如下:随机在N0张不包含行人的红外图像中共抽取N0×10张负样本,即在每张红外图像中随机抽取10张负样本;The negative sample data set construction method is as follows: Randomly extract N 0 × 10 negative samples from N 0 infrared images that do not contain pedestrians, that is, randomly select 10 negative samples from each infrared image;

将所有正负样本缩放至宽为64像素,高为128像素的样本图像。Scales all positive and negative samples to a sample image with a width of 64 pixels and a height of 128 pixels.

所述的最小矩形窗口的定义如作者P.Dollár在文献《Pedestrian Detection:AnEvaluation of the State of the Art》所述。The definition of the minimum rectangular window is as described by the author P. Dollár in the document "Pedestrian Detection: AnEvaluation of the State of the Art".

步骤2:基于亮度统计特征的红外图像DBHOI特征构造。Step 2: Construction of infrared image DBHOI features based on brightness statistical features.

所述的DBHOI为不同区间大小的亮度直方图,全称为Different Bins Histogramof Intensity。The DBHOI is a brightness histogram of different interval sizes, and its full name is Different Bins Histogram of Intensity.

如图2所示,本发明通过对红外图像中的亮度信息进行编码,使得亮度信息成为一个具有较强分辨能力的描述符,其具体构造方式如下。As shown in FIG. 2 , the present invention encodes the brightness information in the infrared image, so that the brightness information becomes a descriptor with strong resolution capability, and its specific structure is as follows.

2-1.假设样本图像尺寸为w′×h′,则将样本图像划分成多份大小相同的局部图像;每一个局部图像的大小为8×8,即将样本图像划分成相等的(w′/8)×(h′/8)个局部图像,并将这些局部图像记为cell。2-1. Assuming that the size of the sample image is w′×h′, the sample image is divided into multiple partial images of the same size; the size of each partial image is 8×8, that is, the sample image is divided into equal (w′ /8)×(h′/8) partial images, and record these partial images as cells.

2-2.根据行人部件确定映射规则。首先根据N1个正样本截取行人各部件,行人各部件包括头部、上半身、下半身;再在负样本中随机截取N1个背景图像,然后分别计算头部图像、上半身图像、下半身图像和背景图像的灰度均值。设四种图像的灰度均值为G1、G2、G3、G4,且大小依次增大;,根据四个灰度均值确定三个映射边界,分别为t1=(G1+G2)/2、t2=(G2+G3)/2、t3=(G3+G4)/2,根据映射边界将灰度值范围划分为四个灰度区间,分别为[0,t1),[t1,t2),[t2,t3),[t3,255]。根据该映射规则,并以灰度值为权值,在每个cell内构建亮度直方图,得到一个四维的特征向量。2-2. Determine the mapping rule according to the pedestrian component. Firstly, according to N 1 positive samples, each part of the pedestrian is intercepted, and each part of the pedestrian includes the head, upper body, and lower body; then N 1 background images are randomly intercepted in the negative samples, and then the head image, upper body image, lower body image and background are calculated respectively The grayscale mean of the image. Assume that the average gray levels of the four images are G 1 , G 2 , G 3 , and G 4 , and the sizes increase sequentially; and determine three mapping boundaries according to the four gray average values, which are respectively t 1 =(G 1 +G 2 )/2, t 2 =(G 2 +G 3 )/2, t 3 =(G 3 +G 4 )/2, according to the mapping boundary, the gray value range is divided into four gray ranges, which are [ 0,t 1 ),[t 1 ,t 2 ),[t 2 ,t 3 ),[t 3 ,255]. According to the mapping rule and with the gray value as the weight, a brightness histogram is constructed in each cell to obtain a four-dimensional feature vector.

2-3.相邻的2×2个cell进行块内归一化。2-3. Adjacent 2×2 cells are normalized within the block.

个cell,以一个cell为步长,将(w′/8)×(h′/8)个cell在样本图像中以从上到下、从左到右的顺序将相邻的四个cell进行块内归一化。归一化方法为L1-Sqrt:其中v为亮度直方图向量,ε为一个很小值,本发明中取值为0.001。cells, with a cell as the step size, (w′/8)×(h′/8) cells are placed in the sample image in order from top to bottom and from left to right. Intra-block normalization. The normalization method is L1-Sqrt: Where v is the brightness histogram vector, ε is a very small value, which is 0.001 in the present invention.

2-4.将所有块的特征进行串联得到DBHOI特征。通过这样的方式构建的DBHOI描述符具有较强的描述能力。2-4. Concatenate the features of all blocks to obtain the DBHOI features. The DBHOI descriptor constructed in this way has strong descriptive ability.

步骤3、基于轮廓信息的红外图像HOG特征构造。Step 3. The infrared image HOG feature construction based on the contour information.

利用经典sobel算子计算每个像素水平和垂直方向的梯度分量Gx(i,j)、Gy(i,j),然后计算对应的梯度大小G(i,j)和方向D(i,j)如下:Use the classic sobel operator to calculate the gradient components G x (i, j) and G y (i, j) of each pixel in the horizontal and vertical directions, and then calculate the corresponding gradient size G (i, j) and direction D (i, j) as follows:

HOG特征的构造采用金典的构造方法,比如将64×128大小的样本图像划分成若干个cell,每个cell的尺寸大小为8×8。针对每一个cell区域将梯度方向划分成均等的9等分,并将梯度大小作为权值构建梯度方向直方图。将相邻的4个cell组合成块,并L1-sqrt方法进行归一化,最后串联不同块之间的梯度直方图,构成最终的HOG描述符。The construction of HOG features adopts the construction method of Jindian, such as dividing the sample image of 64×128 size into several cells, and the size of each cell is 8×8. For each cell area, the gradient direction is divided into 9 equal parts, and the gradient size is used as the weight to construct a gradient direction histogram. The adjacent 4 cells are combined into blocks, and the L1-sqrt method is used for normalization, and finally the gradient histograms between different blocks are concatenated to form the final HOG descriptor.

步骤4:利用Adaboost进行分类器训练。Step 4: Use Adaboost for classifier training.

将所有正负样本缩放至相同的尺度,如64×128。然后针对每一个样本图像,提取DBHOI特征向量和HOG特征向量,并且标记正样本的标签为1,负样本的标签为-1。Scale all positive and negative samples to the same scale, such as 64×128. Then for each sample image, extract the DBHOI feature vector and HOG feature vector, and mark the label of the positive sample as 1, and the label of the negative sample as -1.

将N1个正样本和N0×10个负样本对应的DBHOI-HOG特征向量以及样本标签输入到Adaboost学习算法进行训练,得到一个具有一系列弱分类器构成的分类器。其中,弱分类器为二层决策树模型。The DBHOI-HOG feature vectors and sample labels corresponding to N 1 positive samples and N 0 × 10 negative samples are input into the Adaboost learning algorithm for training, and a classifier consisting of a series of weak classifiers is obtained. Among them, the weak classifier is a two-layer decision tree model.

步骤5:行人框判定及定位。Step 5: Pedestrian frame determination and positioning.

如图3所示,由于行人分类器的大小是确定的,但是行人在红外图像中的尺寸却不同。比如当行人距离摄像机很近时,它的尺寸将比分类器的尺寸大。因此为了检测不同尺度下的行人,需要对图像进行缩放,然后针对每一个尺度下的图像进行划窗扫描。同一个行人在不同的图像尺度下都有可能被分类器判定为行人,因此需要进行行人框的融合。As shown in Figure 3, since the size of the pedestrian classifier is fixed, the size of pedestrians in the infrared image is different. For example, when a pedestrian is very close to the camera, its size will be larger than that of the classifier. Therefore, in order to detect pedestrians at different scales, it is necessary to zoom the image, and then perform a window scan for the image at each scale. The same pedestrian may be judged as a pedestrian by the classifier at different image scales, so the fusion of pedestrian frames is required.

5-1.根据待检测红外图像的尺度和检测窗口的大小确定缩放因子,确定待检测红外图像的缩放层数;然后用步骤4得到的分类器按步长为4个像素大小进行逐层扫描。设待检测红外图像的原始大小为Wi×Hi,其中Wi表示待检测红外图像的宽度,Hi表示待检测红外图像高度,检测窗口大小为Wd×Hd,其中Wd表示检测窗口的宽度,Hd表示检测窗口高度,用ss表示缩放因子。则初始缩放因子为ss=1,终止缩放因子为ss=min{Wi/Wd,Hi/Hd}。对每一个尺度下的待检测红外图像进行滑窗扫描,利用训练得到的行人分类器进行窗口判定。如果该检测窗口是行人框,则记录该检测窗口的位置以及置信度,将该记录表示为{posX,posY,width,height,score},其中posX,posY为行人框的左上角点,width,height为行人框的宽度和高度,score为置信度。5-1. Determine the scaling factor according to the scale of the infrared image to be detected and the size of the detection window, and determine the number of scaling layers of the infrared image to be detected; then use the classifier obtained in step 4 to scan layer by layer with a step size of 4 pixels . Suppose the original size of the infrared image to be detected is W i ×H i , where W i represents the width of the infrared image to be detected, H i represents the height of the infrared image to be detected, and the size of the detection window is W d ×H d , where W d represents the detection The width of the window, H d represents the height of the detection window, and s s represents the scaling factor. Then the initial scaling factor is s s =1, and the termination scaling factor is s s =min{W i /W d , H i /H d }. Sliding window scanning is performed on the infrared images to be detected at each scale, and the window judgment is performed by using the trained pedestrian classifier. If the detection window is a pedestrian frame, record the position and confidence of the detection window, and express the record as {posX, posY, width, height, score}, where posX, posY are the upper left corner of the pedestrian frame, width, height is the width and height of the pedestrian frame, and score is the confidence level.

所述的置信度为所有二层决策树叶子结点记录的错误率。The confidence level is the error rate recorded by all the leaf nodes of the two-level decision tree.

5-2.通过对待检测红外图像进行多尺度的滑窗扫描检测后,同一个行人在不同尺度的待检测红外图像中被检测出来,为了使系统输出一个最可能对应实际行人位置的行人框,采用非极大值抑制法对多个不同尺度的行人框结果进行融合。所述的极大值抑制的标准是每个行人框的置信度score。5-2. After multi-scale sliding window scanning detection of the infrared image to be detected, the same pedestrian is detected in different scales of the infrared image to be detected. In order for the system to output a pedestrian frame that most likely corresponds to the actual pedestrian position, The non-maximum suppression method is used to fuse the pedestrian frame results of multiple different scales. The criterion for maximum value suppression is the confidence score of each pedestrian frame.

5-3.将所有行人框按置信度从低到高进行排列成数组A[n],然后从该数组A[n]中取出当前置信度最大的检测窗口信息A[n];5-3. Arrange all the pedestrian frames into an array A[n] according to the confidence level from low to high, and then take out the detection window information A[n] with the highest current confidence level from the array A[n];

5-4.判断该行人框A[n]与后续行人框A[n-1]的关系,如果两个行人框的重合度α大于0.5,则认为是同一个行人框,否则将行人框A[n-1]作为当前置信度最大的行人框。5-4. Determine the relationship between the pedestrian frame A[n] and the subsequent pedestrian frame A[n-1]. If the coincidence degree α of the two pedestrian frames is greater than 0.5, it is considered to be the same pedestrian frame, otherwise the pedestrian frame A [n-1] as the pedestrian frame with the highest current confidence.

所述的重合度其中area(Bn)为第n个行人框的面积,area(Bn∩Bn-1)为第n个行人框面积与第n-1个行人框面积的交集。The degree of coincidence Among them, area(B n ) is the area of the nth pedestrian frame, and area(B n ∩B n-1 ) is the intersection of the area of the nth pedestrian frame and the area of the n-1th pedestrian frame.

5-5.重复步骤5-4,直至判定完所有的行人框。5-5. Repeat steps 5-4 until all pedestrian frames are determined.

步骤6、行人框重判定。Step 6: Pedestrian frame weight determination.

6-1.通过步骤5之后,每一个被判定为行人的检测窗口都会有一个置信度值score。如果分类器对一个检测窗口的判定越明确,则相应的score值也越大;相反,如果分类器对检测窗口的判定越不明确,则会得到一个很低的score值。因此当一个区域被错误的判定为行人框时,该行人框所得到的置信度score都比较低。为了降低这种因分类器在置信度不够时,做出的错误判定,本发明采用了亮度区间模板再判定技术。当scorei≤τ时,则进行行人框再判定,其中阈值τ的取值由统计方式确定。如果scorei>τ,则仅用分类器进行行人检测,假设得到N个行人框的置信度(默认值N为1000),那么τ满足式 6-1. After passing step 5, each detection window that is determined to be a pedestrian will have a confidence value score. If the classifier has a clearer judgment on a detection window, the corresponding score value will be larger; on the contrary, if the classifier is less clear on the judgment of the detection window, a very low score value will be obtained. Therefore, when an area is wrongly judged as a pedestrian frame, the confidence score obtained by the pedestrian frame is relatively low. In order to reduce the erroneous judgment made by the classifier when the confidence level is not enough, the present invention adopts the brightness interval template re-judgment technology. When score i ≤τ, the pedestrian frame re-judgment is performed, and the value of the threshold τ is determined by a statistical method. If score i >τ, then only the classifier is used for pedestrian detection, assuming that the confidence of N pedestrian frames is obtained (the default value N is 1000), then τ satisfies the formula

所述的scorei为步骤5-3定义的数组A[n]中第i个的置信度值。The score i is the confidence value of the i-th in the array A[n] defined in step 5-3.

6-2.根据步骤2以及红外行人成像特点,[0,t1),[t1,t2),[t2,t3),[t3,255]这4个区间分别对应背景、上半身、下半身、头部。设行人宽高为w×h,如果行人框正确判定行人位置,那么将已经被判定为行人的行人框的高度h从上到下划分成4等分,则在第一个1/4等分处必定存在属于[t3,255]范围的亮度信息,并记灰度值在此区间的像素总个数为Σp1,第二个1/4等分处必定存在[t1,t2)范围的亮度信息,并记灰度值在此区间的像素总个数为Σp2,以及后两个1/4等分处必定存在[t2,t3)范围内的亮度信息,并记灰度值在此区间的像素总个数为Σp3。设置如下验证规则:Σp1/(w×h)≥1/16,Σp2/(w×h)≥1/8,Σp3/(w×h)≥1/16。如果Σp1、Σp2、Σp3同时满足这三个条件,那么该行人框被判定为行人框,否则该行人框判定为非行人框。6-2. According to step 2 and the characteristics of infrared pedestrian imaging, the four intervals [0,t 1 ),[t 1 ,t 2 ),[t 2 ,t 3 ),[t 3 ,255] correspond to the background, Upper body, lower body, head. Let the width and height of the pedestrian be w×h, if the pedestrian frame correctly determines the position of the pedestrian, then divide the height h of the pedestrian frame that has been judged as a pedestrian into 4 equal parts from top to bottom, then in the first 1/4 equal part There must be brightness information belonging to the range of [t 3 ,255], and the total number of pixels with gray values in this range is Σp 1 , and the second 1/4 equal division must exist at [t 1 ,t 2 ) The brightness information of the range, and record the total number of pixels of the gray value in this interval as Σp 2 , and there must be brightness information in the range [t 2 ,t 3 ) at the last two 1/4 equal parts, and record the gray The total number of pixels with degrees in this range is Σp 3 . Set the verification rules as follows: Σp 1 /(w×h)≥1/16, Σp 2 /(w×h)≥1/8, Σp 3 /(w×h)≥1/16. If Σp 1 , Σp 2 , and Σp 3 satisfy these three conditions at the same time, then the pedestrian frame is judged as a pedestrian frame, otherwise the pedestrian frame is judged as a non-pedestrian frame.

Claims (2)

1.一种基于红外行人亮度统计特征的夜间行人检测方法,其特征在于包括如下步骤:1. a nighttime pedestrian detection method based on infrared pedestrian brightness statistical features, is characterized in that comprising the steps: 步骤1、构建红外图像的正负样本数据集;Step 1. Construct positive and negative sample data sets of infrared images; 步骤2、基于亮度统计特征的红外图像DBHOI特征构造;Step 2, infrared image DBHOI feature construction based on brightness statistical features; 步骤3、基于轮廓信息的红外图像HOG特征构造;Step 3, infrared image HOG feature construction based on contour information; 步骤4、利用Adaboost进行分类器训练;Step 4, utilize Adaboost to carry out classifier training; 步骤5、检测窗口判定及定位;Step 5, detection window judgment and positioning; 步骤6、行人框重判定;Step 6, pedestrian frame weight determination; 步骤1所述的构建红外图像的正负样本数据集具体如下:The details of constructing the positive and negative sample data sets of infrared images described in step 1 are as follows: 正样本数据集构建方法如下:采用最小矩形窗口,提取红外图像中的行人样本;假设行人的高为h,宽为w,使得w/h=0.41;共提取行人正样本N1张;The method of constructing the positive sample data set is as follows: use the smallest rectangular window to extract pedestrian samples in the infrared image; assume that the height of the pedestrian is h and the width is w, so that w/h=0.41; a total of N 1 positive samples of pedestrians are extracted; 负样本数据集构建方法如下:随机在N0张不包含行人的红外图像中共抽取N0×10张负样本,即在每张红外图像中随机抽取10张负样本;The negative sample data set construction method is as follows: Randomly extract N 0 × 10 negative samples from N 0 infrared images that do not contain pedestrians, that is, randomly select 10 negative samples from each infrared image; 将所有正负样本缩放至宽为64像素,高为128像素的样本图像;Scale all positive and negative samples to a sample image with a width of 64 pixels and a height of 128 pixels; 步骤2所述的基于亮度统计特征的红外图像DBHOI特征构造具体如下:The infrared image DBHOI feature construction based on the brightness statistical feature described in step 2 is specifically as follows: 2-1.设样本图像尺寸为w′×h′,则将样本图像划分成多份大小相同的局部图像;每一个局部图像的大小为8×8,即将样本图像划分成相等的(w′/8)×(h′/8)个局部图像,并将这些局部图像记为cell;2-1. Set the size of the sample image as w′×h′, then divide the sample image into multiple partial images of the same size; the size of each partial image is 8×8, that is, divide the sample image into equal (w′ /8)×(h′/8) partial images, and record these partial images as cells; 2-2.根据行人部件确定映射规则;2-2. Determine the mapping rules according to the pedestrian components; 2-2-1.首先根据N1个正样本截取行人各部件,行人各部件包括头部、上半身、下半身;2-2-1. Firstly, according to N 1 positive samples, each part of the pedestrian is intercepted, and each part of the pedestrian includes the head, upper body, and lower body; 2-2-2.在负样本中随机截取N1个背景图像;2-2-2. Randomly intercept N 1 background images in the negative sample; 2-2-3.分别计算头部图像、上半身图像、下半身图像和背景图像的灰度均值;2-2-3. Calculate the gray mean value of the head image, upper body image, lower body image and background image respectively; 2-2-4.设四种图像的灰度均值为G1、G2、G3、G4,且大小依次增大;,根据四个灰度均值确定三个映射边界,分别为t1=(G1+G2)/2、t2=(G2+G3)/2、t3=(G3+G4)/2,根据映射边界将灰度值范围划分为四个灰度区间,分别为[0,t1),[t1,t2),[t2,t3),[t3,255];2-2-4. Set the average gray values of the four images as G 1 , G 2 , G 3 , and G 4 , and the sizes increase sequentially; and determine three mapping boundaries according to the four average gray values, respectively t 1 =(G 1 +G 2 )/2, t 2 =(G 2 +G 3 )/2, t 3 =(G 3 +G 4 )/2, divide the gray value range into four gray values according to the mapping boundary degree interval, respectively [0,t 1 ),[t 1 ,t 2 ),[t 2 ,t 3 ),[t 3 ,255]; 2-2-5.根据该映射规则并以红外图像中像素的灰度值为权值,在每个cell内构建亮度直方图,得到一个四维的特征向量;2-2-5. According to the mapping rule and using the gray value of the pixel in the infrared image as the weight, construct a brightness histogram in each cell to obtain a four-dimensional feature vector; 2-3.相邻的2×2个cell进行块内归一化;2-3. Adjacent 2×2 cells are normalized within the block; 以一个cell为步长,将(w′/8)×(h′/8)个cell在样本图像中以从上到下、从左到右的顺序将相邻的四个cell通过L1-Sqrt方法进行块内归一化;Taking one cell as the step size, put (w'/8)×(h'/8) cells in the sample image in order from top to bottom and from left to right to pass the adjacent four cells through L1-Sqrt method for intra-block normalization; 其中L1-Sqrt方法如下:v为亮度直方图向量,ε为一个很小值,取值为0.001;The L1-Sqrt method is as follows: v is the brightness histogram vector, ε is a very small value, the value is 0.001; 2-4.将所有块的特征进行串联得到DBHOI特征;2-4. Concatenate the features of all blocks to obtain the DBHOI features; 所述的DBHOI为不同区间大小的亮度直方图;The DBHOI is a brightness histogram of different interval sizes; 步骤3所述的基于轮廓信息的红外图像HOG特征构造具体如下:The HOG feature construction of the infrared image based on the contour information described in step 3 is as follows: 利用经典sobel算子计算每个像素水平和垂直方向的梯度分量Gx(i,j)、Gy(i,j),然后计算对应的梯度大小G(i,j)和方向D(i,j)如下:Use the classic sobel operator to calculate the gradient components G x (i, j) and G y (i, j) of each pixel in the horizontal and vertical directions, and then calculate the corresponding gradient size G (i, j) and direction D (i, j) as follows: <mrow> <mi>G</mi> <mrow> <mo>(</mo> <mi>i</mi> <mo>,</mo> <mi>j</mi> <mo>)</mo> </mrow> <mo>=</mo> <msqrt> <mrow> <msub> <mi>G</mi> <mi>x</mi> </msub> <msup> <mrow> <mo>(</mo> <mi>i</mi> <mo>,</mo> <mi>j</mi> <mo>)</mo> </mrow> <mn>2</mn> </msup> <mo>+</mo> <msub> <mi>G</mi> <mi>y</mi> </msub> <msup> <mrow> <mo>(</mo> <mi>i</mi> <mo>,</mo> <mi>j</mi> <mo>)</mo> </mrow> <mn>2</mn> </msup> </mrow> </msqrt> </mrow> <mrow><mi>G</mi><mrow><mo>(</mo><mi>i</mi><mo>,</mo><mi>j</mi><mo>)</mo></mrow><mo>=</mo><msqrt><mrow><msub><mi>G</mi><mi>x</mi></msub><msup><mrow><mo>(</mo><mi>i</mi><mo>,</mo><mi>j</mi><mo>)</mo></mrow><mn>2</mn></msup><mo>+</mo><msub><mi>G</mi><mi>y</mi></msub><msup><mrow><mo>(</mo><mi>i</mi><mo>,</mo><mi>j</mi><mo>)</mo></mrow><mn>2</mn></msup></mrow></msqrt></mrow> <mrow> <mi>D</mi> <mrow> <mo>(</mo> <mi>i</mi> <mo>,</mo> <mi>j</mi> <mo>)</mo> </mrow> <mo>=</mo> <mi>a</mi> <mi>r</mi> <mi>c</mi> <mi>t</mi> <mi>a</mi> <mi>n</mi> <mrow> <mo>(</mo> <mfrac> <mrow> <msub> <mi>G</mi> <mi>y</mi> </msub> <mrow> <mo>(</mo> <mi>i</mi> <mo>,</mo> <mi>j</mi> <mo>)</mo> </mrow> </mrow> <mrow> <msub> <mi>G</mi> <mi>x</mi> </msub> <mrow> <mo>(</mo> <mi>i</mi> <mo>,</mo> <mi>j</mi> <mo>)</mo> </mrow> </mrow> </mfrac> <mo>)</mo> </mrow> <mo>;</mo> </mrow> <mrow><mi>D</mi><mrow><mo>(</mo><mi>i</mi><mo>,</mo><mi>j</mi><mo>)</mo></mrow><mo>=</mo><mi>a</mi><mi>r</mi><mi>c</mi><mi>t</mi><mi>a</mi><mi>n</mi><mrow><mo>(</mo><mfrac><mrow><msub><mi>G</mi><mi>y</mi></msub><mrow><mo>(</mo><mi>i</mi><mo>,</mo><mi>j</mi><mo>)</mo></mrow></mrow><mrow><msub><mi>G</mi><mi>x</mi></msub><mrow><mo>(</mo><mi>i</mi><mo>,</mo><mi>j</mi><mo>)</mo></mrow></mrow></mfrac><mo>)</mo></mrow><mo>;</mo></mrow> 步骤4所述的利用Adaboost进行分类器训练具体如下:The classifier training using Adaboost described in step 4 is as follows: 4-1.将所有正负样本缩放至相同的尺度,然后针对每一个样本图像,提取DBHOI特征向量和HOG特征向量,并且标记正样本的标签为1,负样本的标签为-1;4-1. Scale all positive and negative samples to the same scale, then extract the DBHOI feature vector and HOG feature vector for each sample image, and mark the label of the positive sample as 1, and the label of the negative sample as -1; 4-2.将N1个正样本和N0×10个负样本对应的DBHOI-HOG特征向量以及样本标签输入到Adaboost学习算法进行训练,得到一个具有一系列弱分类器构成的分类器;4-2. Input the DBHOI-HOG feature vectors and sample labels corresponding to N 1 positive samples and N 0 × 10 negative samples to the Adaboost learning algorithm for training, and obtain a classifier composed of a series of weak classifiers; 步骤5所述的检测窗口判定及定位如下:The determination and positioning of the detection window described in step 5 are as follows: 5-1.根据待检测红外图像的尺度和检测窗口的大小确定缩放因子,确定待检测红外图像的缩放层数;然后用步骤4得到的分类器按步长为4个像素大小进行逐层扫描;设待检测红外图像的原始大小为Wi×Hi,其中Wi表示待检测红外图像的宽度,Hi表示待检测红外图像高度,检测窗口大小为Wd×Hd,其中Wd表示检测窗口的宽度,Hd表示检测窗口高度,用ss表示缩放因子;则初始缩放因子为ss=1,终止缩放因子为ss=min{Wi/Wd,Hi/Hd};对每一个尺度下的待检测红外图像进行滑窗扫描,利用训练得到的行人分类器进行检测窗口判定;如果该检测窗口是行人框,则记录该检测窗口的位置以及置信度,将该记录表示为{posX,posY,width,height,score},其中posX,posY为行人框的左上角点,width,height为行人框的宽度和高度,score为置信度;5-1. Determine the scaling factor according to the scale of the infrared image to be detected and the size of the detection window, and determine the number of scaling layers of the infrared image to be detected; then use the classifier obtained in step 4 to scan layer by layer with a step size of 4 pixels ; Suppose the original size of the infrared image to be detected is W i ×H i , where W i represents the width of the infrared image to be detected, H i represents the height of the infrared image to be detected, and the size of the detection window is W d ×H d , where W d represents The width of the detection window, H d represents the height of the detection window, and s s represents the scaling factor; then the initial scaling factor is s s =1, and the termination scaling factor is s s =min{W i /W d ,H i /H d } ; Sliding window scanning is performed on the infrared image to be detected at each scale, and the detection window is judged by using the trained pedestrian classifier; if the detection window is a pedestrian frame, record the position and confidence of the detection window, and record the Expressed as {posX, posY, width, height, score}, where posX and posY are the upper left corner of the pedestrian frame, width and height are the width and height of the pedestrian frame, and score is the confidence level; 所述的置信度为所有二层决策树叶子结点记录的错误率;The confidence degree is the error rate recorded by all the leaf nodes of the two-layer decision tree; 5-2.通过对待检测红外图像进行多尺度的滑窗扫描检测后,同一个行人在不同尺度的待检测红外图像中被检测出来,采用非极大值抑制法对多个不同尺度的行人框结果进行融合;5-2. After multi-scale sliding window scanning detection of the infrared image to be detected, the same pedestrian is detected in different scales of the infrared image to be detected, and the non-maximum value suppression method is used to detect multiple pedestrian frames of different scales. The results are merged; 所述的极大值抑制的标准是每个行人框的置信度score;The criterion for maximum value suppression is the confidence score of each pedestrian frame; 5-3.将所有行人框按置信度从低到高进行排列成数组A[n],然后从该数组A[n]中取出当前置信度最大的检测窗口信息A[n];5-3. Arrange all the pedestrian frames into an array A[n] according to the confidence level from low to high, and then take out the detection window information A[n] with the highest current confidence level from the array A[n]; 5-4.判断该行人框A[n]与后续行人框A[n-1]的关系,如果两个行人框的重合度α大于0.5,则认为是同一个行人框,否则将行人框A[n-1]作为当前置信度最大的行人框;5-4. Determine the relationship between the pedestrian frame A[n] and the subsequent pedestrian frame A[n-1]. If the coincidence degree α of the two pedestrian frames is greater than 0.5, it is considered to be the same pedestrian frame, otherwise the pedestrian frame A [n-1] as the pedestrian frame with the highest current confidence; 所述的重合度其中area(Bn)为第n个行人框的面积,area(Bn∩Bn-1)为第n个行人框面积与第n-1个行人框面积的交集;The degree of coincidence Where area(B n ) is the area of the nth pedestrian frame, area(B n ∩B n-1 ) is the intersection of the area of the nth pedestrian frame and the area of the n-1th pedestrian frame; 5-5.重复步骤5-4,直至判定完所有的行人框。5-5. Repeat steps 5-4 until all pedestrian frames are determined. 2.根据权利要求1所述的一种基于红外行人亮度统计特征的夜间行人检测方法,其特征在于步骤6所述的行人框重判定如下:2. a kind of night pedestrian detection method based on infrared pedestrian luminance statistical feature according to claim 1, it is characterized in that the pedestrian frame heavy determination described in step 6 is as follows: 6-1.通过步骤5之后,每一个被判定为行人的检测窗口都会有一个置信度值score;采用了亮度区间模板对行人框进行再判定;当scorei≤τ时,则进行行人框再判定,其中阈值τ的取值由统计方式确定;如果scorei>τ,则仅用分类器进行行人检测,假设得到N个行人框的置信度,那么τ满足式 6-1. After passing step 5, each detection window that is judged as a pedestrian will have a confidence value score; the pedestrian frame is re-judged using the brightness interval template; when the score i ≤τ, the pedestrian frame is re-determined Judgment, where the value of the threshold τ is determined by a statistical method; if score i > τ, only the classifier is used for pedestrian detection, assuming that the confidence of N pedestrian frames is obtained, then τ satisfies the formula 所述的scorei为步骤5-3定义的数组A[n]中第i个的置信度值;The score i is the i-th confidence value in the array A[n] defined in step 5-3; 6-2.根据步骤2以及红外行人成像特点,[0,t1),[t1,t2),[t2,t3),[t3,255]这4个区间分别对应背景、上半身、下半身、头部;设行人宽高为w×h,如果行人框正确判定行人位置,那么将已经被判定为行人的行人框的高度h从上到下划分成4等分,则在第一个1/4等分处必定存在属于[t3,255]范围的亮度信息,并记灰度值在此区间的像素总个数为∑p1,第二个1/4等分处必定存在[t1,t2)范围的亮度信息,并记灰度值在此区间的像素总个数为∑p2,以及后两个1/4等分处必定存在[t2,t3)范围内的亮度信息,并记灰度值在此区间的像素总个数为∑p3;设置如下验证规则:∑p1/(w×h)≥1/16,∑p2/(w×h)≥1/8,∑p3/(w×h)≥1/16;如果∑p1、∑p2、∑p3同时满足这三个条件,那么该行人框再次被判定为行人框,否则该行人框判定为非行人框。6-2. According to step 2 and the characteristics of infrared pedestrian imaging, the four intervals [0,t 1 ),[t 1 ,t 2 ),[t 2 ,t 3 ),[t 3 ,255] correspond to the background, Upper body, lower body, head; if the width and height of the pedestrian is w×h, if the pedestrian frame correctly determines the position of the pedestrian, then the height h of the pedestrian frame that has been judged as a pedestrian is divided into 4 equal parts from top to bottom, then at There must be brightness information belonging to the range of [t 3 ,255] at a 1/4 equal division, and the total number of pixels with gray values in this interval is ∑p 1 , and the second 1/4 equal division must be There is brightness information in the range of [t 1 ,t 2 ), and the total number of pixels with gray value in this interval is ∑p 2 , and there must be [t 2 ,t 3 ) at the last two 1/4 equal parts Brightness information within the range, and record the total number of pixels with gray values in this range as ∑p 3 ; set the following verification rules: ∑p 1 /(w×h)≥1/16, ∑p 2 /(w×h h)≥1/8, ∑p 3 /(w×h)≥1/16; if ∑p 1 , ∑p 2 , ∑p 3 meet these three conditions at the same time, then the pedestrian frame is judged as a pedestrian frame again , otherwise the pedestrian frame is judged as a non-pedestrian frame.
CN201510154382.8A 2015-04-02 2015-04-02 A kind of night pedestrian detection method based on infrared pedestrian's brightness statistics feature Expired - Fee Related CN104778453B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201510154382.8A CN104778453B (en) 2015-04-02 2015-04-02 A kind of night pedestrian detection method based on infrared pedestrian's brightness statistics feature

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201510154382.8A CN104778453B (en) 2015-04-02 2015-04-02 A kind of night pedestrian detection method based on infrared pedestrian's brightness statistics feature

Publications (2)

Publication Number Publication Date
CN104778453A CN104778453A (en) 2015-07-15
CN104778453B true CN104778453B (en) 2017-12-22

Family

ID=53619906

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201510154382.8A Expired - Fee Related CN104778453B (en) 2015-04-02 2015-04-02 A kind of night pedestrian detection method based on infrared pedestrian's brightness statistics feature

Country Status (1)

Country Link
CN (1) CN104778453B (en)

Families Citing this family (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105139869B (en) * 2015-07-27 2018-11-30 安徽清新互联信息科技有限公司 A kind of baby crying detection method based on section Differential Characteristics
CN105139427B (en) * 2015-09-10 2018-06-22 华南理工大学 A kind of component dividing method identified again suitable for pedestrian's video
CN105279754B (en) * 2015-09-10 2018-06-22 华南理工大学 A kind of component dividing method suitable for bicycle video detection
CN105389546A (en) * 2015-10-22 2016-03-09 四川膨旭科技有限公司 System for identifying person at night during vehicle driving process
CN105426852B (en) * 2015-11-23 2019-01-08 天津津航技术物理研究所 Vehicle-mounted monocular LONG WAVE INFRARED camera pedestrian recognition method
CN105631410B (en) * 2015-12-18 2019-04-09 华南理工大学 A classroom detection method based on intelligent video processing technology
CN105913003B (en) * 2016-04-07 2019-06-07 国家电网公司 A kind of pedestrian detection method of multiple features multi-model
CN106372624B (en) * 2016-10-15 2020-04-14 杭州艾米机器人有限公司 Face recognition method and system
CN106600635A (en) * 2016-11-03 2017-04-26 上海机电工程研究所 Infrared target radiation characteristic simulation model checking verifying method based on small subsamples
CN106919930A (en) * 2017-03-10 2017-07-04 成都智锗科技有限公司 A kind of low resolution infrared image parahypnosis situation determination methods
CN109034174B (en) * 2017-06-08 2021-07-09 北京君正集成电路股份有限公司 Cascade classifier training method and device
CN108734178A (en) * 2018-05-18 2018-11-02 电子科技大学 A kind of HOG feature extracting methods of rule-basedization template
CN111126444A (en) * 2019-11-28 2020-05-08 天津津航技术物理研究所 Classifier ensemble method
CN113095120B (en) * 2020-01-09 2024-05-07 北京君正集成电路股份有限公司 System for realizing reduction of false alarm of detection of human upper body
CN111382718B (en) * 2020-03-17 2023-08-11 电子科技大学中山学院 A pedestrian detection system at night and a pedestrian detection method based on the system
CN111508003B (en) * 2020-04-20 2020-12-11 北京理工大学 Infrared small target detection tracking and identification method
CN113313078B (en) * 2021-07-02 2022-07-08 昆明理工大学 Lightweight night infrared image pedestrian detection method and system based on model optimization

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102609682A (en) * 2012-01-13 2012-07-25 北京邮电大学 Feedback pedestrian detection method for region of interest
CN103632170A (en) * 2012-08-20 2014-03-12 深圳市汉华安道科技有限责任公司 Pedestrian detection method and device based on characteristic combination
CN103886308A (en) * 2014-04-15 2014-06-25 中南大学 Pedestrian detection method through soft cascade classifiers according to polymerization channel characteristics
CN103902976A (en) * 2014-03-31 2014-07-02 浙江大学 Pedestrian detection method based on infrared image

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
TWI419082B (en) * 2010-12-20 2013-12-11 Ind Tech Res Inst Moving object detection method and image processing system for moving object detection

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102609682A (en) * 2012-01-13 2012-07-25 北京邮电大学 Feedback pedestrian detection method for region of interest
CN103632170A (en) * 2012-08-20 2014-03-12 深圳市汉华安道科技有限责任公司 Pedestrian detection method and device based on characteristic combination
CN103902976A (en) * 2014-03-31 2014-07-02 浙江大学 Pedestrian detection method based on infrared image
CN103886308A (en) * 2014-04-15 2014-06-25 中南大学 Pedestrian detection method through soft cascade classifiers according to polymerization channel characteristics

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
A modular tracking system for far infrared pedestrian recognition;E. Binelli 等;《2005 proceedings IEEE Intelligent Vehicles Symposium》;20050608;第759-763页 *
Pedestrian Detection for Driving Assistance Systems: Single-frame Classification and System Level Performance;Amnon Shashua 等;《2004 IEEE Intelligent Vehicles Symposium》;20040617;第1-6页 *
夜晚环境下的行人检测技术研究;朱聪聪;《中国优秀硕士学位论文全文数据库·信息科技辑》;20140615(第06期);摘要,第1.2.1-1.2节,第2.3-2.7节,图2-1、图2-17、图2-18 *
红外图像中人体目标检测技术研究;赵君钦 等;《现代电子技术》;20120930;第35卷(第18期);第111-113,118页 *

Also Published As

Publication number Publication date
CN104778453A (en) 2015-07-15

Similar Documents

Publication Publication Date Title
CN104778453B (en) A kind of night pedestrian detection method based on infrared pedestrian&#39;s brightness statistics feature
CN107134144B (en) A vehicle detection method for traffic monitoring
US10198657B2 (en) All-weather thermal-image pedestrian detection method
TWI431559B (en) A vehicle registration-plate detecting method and system thereof
CN105447503B (en) Pedestrian detection method based on rarefaction representation LBP and HOG fusion
WO2017190574A1 (en) Fast pedestrian detection method based on aggregation channel features
CN107301378B (en) Pedestrian detection method and system based on multi-classifier integration in image
WO2019169816A1 (en) Deep neural network for fine recognition of vehicle attributes, and training method thereof
Yang et al. Real-time pedestrian and vehicle detection for autonomous driving
CN108460403A (en) The object detection method and system of multi-scale feature fusion in a kind of image
CN103020985B (en) A kind of video image conspicuousness detection method based on field-quantity analysis
WO2023082784A1 (en) Person re-identification method and apparatus based on local feature attention
CN109948566B (en) Double-flow face anti-fraud detection method based on weight fusion and feature selection
CN109886086B (en) Pedestrian detection method based on HOG feature and linear SVM cascade classifier
CN105335716A (en) Improved UDN joint-feature extraction-based pedestrian detection method
CN105913040A (en) Real time double cameras provided pedestrian detection system for use under scotopic vision conditions
CN107315998B (en) Method and system for classifying vehicle types based on lane lines
CN112766056B (en) Method and device for detecting lane lines in low-light environment based on deep neural network
CN102855500A (en) Haar and HoG characteristic based preceding car detection method
CN109063619A (en) A kind of traffic lights detection method and system based on adaptive background suppression filter and combinations of directions histogram of gradients
CN110659550A (en) Traffic sign recognition method, traffic sign recognition device, computer equipment and storage medium
CN104978567A (en) Vehicle detection method based on scenario classification
CN106446792A (en) A feature extraction method for pedestrian detection in road traffic assisted driving environment
CN106886778A (en) A kind of car plate segmentation of the characters and their identification method under monitoring scene
CN102609720A (en) Pedestrian detection method based on position correction model

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
EXSB Decision made by sipo to initiate substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
EE01 Entry into force of recordation of patent licensing contract

Application publication date: 20150715

Assignee: HANGZHOU ZHISHU TECHNOLOGY CO.,LTD.

Assignor: HANGZHOU DIANZI University

Contract record no.: X2022330000062

Denomination of invention: A night pedestrian detection method based on infrared pedestrian brightness statistical characteristics

Granted publication date: 20171222

License type: Common License

Record date: 20220331

EE01 Entry into force of recordation of patent licensing contract
CF01 Termination of patent right due to non-payment of annual fee

Granted publication date: 20171222

CF01 Termination of patent right due to non-payment of annual fee