CN1332357C

CN1332357C - Sensitive video frequency detection based on kinematic skin division

Info

Publication number: CN1332357C
Application number: CNB2004100335406A
Authority: CN
Inventors: 谭铁牛; 胡卫明; 王谦; 杨金峰
Original assignee: Institute of Automation of Chinese Academy of Science
Current assignee: Institute of Automation of Chinese Academy of Science
Priority date: 2004-04-06
Filing date: 2004-04-06
Publication date: 2007-08-15
Anticipated expiration: 2024-04-06
Also published as: CN1680977A

Abstract

A sensitive video detection method based on moving skin color segmentation, comprising the steps of: segmenting and boundary extraction of moving objects in the video; performing skin color detection on the segmented object to obtain the exposure degree of the skin relative to the moving object; A comprehensive evaluation of the sensitivity of the entire video is made on the basis of frame calculation. The invention applies computer vision technology to the Internet, identifies and filters uncivilized information on the Internet, and protects users from being poisoned by uncivilized information. Through the test of the international standard library, the invention achieves a higher recognition rate.

Description

Sensitive video detection method based on motion skin color segmentation

技术领域technical field

本发明涉及计算机网络技术与计算机视觉技术相结合的领域，特别涉及基于运动肤色分割的敏感视频检测方法。The invention relates to the field of combining computer network technology and computer vision technology, in particular to a sensitive video detection method based on moving skin color segmentation.

背景技术Background technique

Internet的迅速普及和广泛应用对计算机技术的发展产生了深刻影响：计算机软件应用的网络化对软件技术提出了新的要求，网络信息安全是新的要求中的一个极为重要的问题，对网络敏感信息的过滤则是信息安全中的一个具体课题。对网络敏感信息的过滤已经进行了一些研究，并出现了一些网页过滤与检测的产品，例如SmartFilter[ http://www.smartfilter.de/]、NoPorn[ http://www.noporn.com.tw/]等防黄软件可以防止一般的计算机使用者利用浏览器访问色情网站。其中SmartFilter对互联网络访问的管理及监督就是通过SmartFilter控制列表资料库来达到的，SmartFilter公司的专业人员每天由世界各地的服务器收集目前增加或消失的网站资讯，并且每星期更新一次，SmartFilter控制列表资料库提供的完整URL资料库，所有采用SmartFilter产品的客户都可以每星期下载最新的控制列表资料库。为方便管理者设定且依据各单位不同的需求、兴趣及政策，控制列表资料库被区分为27种类别。如：聊天，网络约会，赌博，过激言论，谣言，色情等。但是这种产品人为参与的因素太多，不能实现信息的自动处理。VisionNEXT公司生产的eefind[ http://www.eefind.com/]多媒体搜索系列，过滤系列软件能实现简单的图像检测，搜索与过滤，但是在敏感图片的检测，过滤与搜索方面准确率太低。The rapid popularization and wide application of the Internet have had a profound impact on the development of computer technology: the networking of computer software applications has put forward new requirements for software technology, and network information security is an extremely important issue in the new requirements. It is sensitive to the network. Information filtering is a specific topic in information security. Some researches have been done on the filtering of sensitive information on the web, and some web filtering and detection products have emerged, such as SmartFilter[ http://www.smartfilter.de/ ], NoPorn[ http://www.noporn.com. tw/ ] and other anti-pornography software can prevent general computer users from using browsers to access pornographic websites. Among them, SmartFilter's management and supervision of Internet access is achieved through the SmartFilter control list database. SmartFilter's professionals collect the current website information that is currently added or disappeared from servers around the world every day, and update it once a week. SmartFilter control list The database provides a complete URL database, and all customers using SmartFilter products can download the latest control list database every week. For the convenience of administrators and according to the different needs, interests and policies of each unit, the control list database is divided into 27 categories. Such as: chatting, online dating, gambling, radical speech, rumors, pornography, etc. However, there are too many factors of human participation in this product, and the automatic processing of information cannot be realized. The eefind[ http://www.eefind.com/ ] multimedia search series and filter series software produced by VisionNEXT can realize simple image detection, search and filter, but the accuracy rate is too low in sensitive image detection, filter and search .

在敏感信息检测方面，国外一些大学(Berkeley，Iowa)开展了部分对网络上敏感图片进行分析的探索。Fleck与Forsyth[Margaret Fleck，DavidForsyth，and Chris Bregler，“Finding Naked People”European Conference onComputer Vision，Volume II，1996，pp.592-602]通过检测人体的皮肤，并把各部分皮肤区域连成一组，来识别一幅图片是否包含裸体内容。该系统使用组合的颜色和纹理属性标记出类似皮肤的象素，然后将这些皮肤区域送到一特定的成组器。成组器利用人体结构上的几何约束把这些区域组成一个人的轮廓。如果成组器发现了一个足够复杂的结构，它就认为这幅图片中包含有人。这种方法对于存在大范围阴影和皮肤颜色的场合是很有效的。Aberdeen的Ian Craw在皮肤检测中用SOM网对肤色的概率模型进行学习，检验样本输入网络后得到一个可能为肤色的概率值，然后设置一阀值来判定是否为肤色[David Brown，Ian Craw，and JulianLewthwaite，A SOM based approach to skin detection with application in realtime systems.PDF preprint，Department of Mathematical Sciences，Universityof Aberdeen，2001.]。此外，还有一些通用的基于内容的图像检索系统，如IBM的QBIC，Attrasoft的ImageFinder、MWLabs的Imatch等，这些系统均支持对颜色、形状、纹理等特征的匹配[Colin C.Venters and Dr.Matthew Cooper，“A Review of Content-Based Image Retrieval Systems”，University of Manchester，2000]。但是这种通用的图像检索系统并非特定为敏感图片而设计，在进行敏感图片搜索时效率不高。In terms of sensitive information detection, some foreign universities (Berkeley, Iowa) have carried out some explorations on the analysis of sensitive pictures on the Internet. Fleck and Forsyth [Margaret Fleck, David Forsyth, and Chris Bregler, "Finding Naked People" European Conference on Computer Vision, Volume II, 1996, pp.592-602] By detecting the skin of the human body and connecting various skin regions into a group, to identify whether an image contains nudity. The system marks skin-like pixels using combined color and texture attributes, and then sends these skin regions to a specific grouper. The grouper uses the geometric constraints on the human anatomy to compose these regions into a human silhouette. If the grouper finds a sufficiently complex structure, it considers the picture to contain a human. This method works well for situations where there is a wide range of shades and skin colors. Ian Craw of Aberdeen uses the SOM network to learn the probability model of skin color in skin detection. After the test sample is input into the network, a probability value that may be skin color is obtained, and then a threshold is set to determine whether it is skin color [David Brown, Ian Craw, and Julian Lewthwaite, A SOM based approach to skin detection with application in realtime systems. PDF preprint, Department of Mathematical Sciences, University of Aberdeen, 2001.]. In addition, there are some common content-based image retrieval systems, such as IBM's QBIC, Attrasoft's ImageFinder, MWLabs' Imatch, etc., all of which support the matching of color, shape, texture and other features [Colin C. Venters and Dr. Matthew Cooper, "A Review of Content-Based Image Retrieval Systems", University of Manchester, 2000]. However, this general image retrieval system is not specifically designed for sensitive images, and its efficiency is not high when searching for sensitive images.

国内网络安全方面的产品有PC卫士，PC卫士1.0版主要通过两种手段来过滤网络色情信息，一种是基于网站地址的数据包过滤，另一种是智能信息过滤。其中，智能过滤的基础是截获网络包上一级的数据和不良站点网络数据包特征的提取，该产品不具备敏感图像的自动识别与理解功能。Domestic network security products include PC Guard, and PC Guard 1.0 mainly uses two methods to filter Internet pornographic information, one is data packet filtering based on website address, and the other is intelligent information filtering. Among them, the basis of intelligent filtering is to intercept the upper-level data of network packets and extract the characteristics of network data packets of bad sites. This product does not have the function of automatic identification and understanding of sensitive images.

视频过滤是建立在图象过滤的基础之上的。目前动态黄色信息的过滤依然是个空白，国内外本来做网络图像过滤的就不多，做网络视频过滤的几乎没有。这主要是因为视频过滤的技术还很不成熟，对视频的过滤比对静态图像的过滤要更难，实时性要求更高。但社会对此有迫切需求，因为动态黄色信息危害性更大。而且我们从研究动态黄色视频过滤中得出的一整套方法，对计算机视觉中人的行为分析和语义理解具有重要的借鉴意义。Video filtering is based on image filtering. At present, the filtering of dynamic pornographic information is still a blank. There are not many network image filtering at home and abroad, and almost no network video filtering. This is mainly because the technology of video filtering is still very immature, the filtering of video is more difficult than that of static images, and the real-time requirements are higher. But society has an urgent need for this, because dynamic pornographic information is more harmful. Moreover, the set of methods we obtained from the study of dynamic yellow video filtering has important reference significance for human behavior analysis and semantic understanding in computer vision.

发明内容Contents of the invention

本发明的目的是提供一种基于运动肤色分割的敏感视频检测方法。The purpose of the present invention is to provide a sensitive video detection method based on moving skin color segmentation.

为实现上述目的，一种基于运动肤色分割的敏感视频检测方法，包括步骤：In order to achieve the above object, a sensitive video detection method based on motion skin color segmentation, comprising steps:

采用lever set对偏微分方程演化的方法对视频中的运动对象进行分割和边界提取；Segmentation and boundary extraction of moving objects in the video using the lever set method of evolution of partial differential equations;

采用基于关系数据库的立方体肤色模型对被分割对象上进行肤色检测，求出皮肤相对于运动对象的暴露程度；Use the cube skin color model based on the relational database to detect the skin color on the segmented object, and find out the exposure degree of the skin relative to the moving object;

在每一帧计算单帧敏感度f(t)的基础上对整个视频的敏感性做综合评价。The sensitivity of the entire video is comprehensively evaluated on the basis of calculating the single-frame sensitivity f(t) for each frame.

本发明将计算机视觉技术应用于互联网，识别并过滤互联网上的不文明信息，是用户免受不文明信息的毒害。经过国际标准库的测试，本发明达到了较高的识别率。The invention applies computer vision technology to the Internet, identifies and filters uncivilized information on the Internet, and protects users from poisoning of uncivilized information. Through the test of the international standard library, the invention achieves a higher recognition rate.

附图说明Description of drawings

图1是运动区域分割与边界提取示例；Figure 1 is an example of motion region segmentation and boundary extraction;

图2是立方体肤色模型；Fig. 2 is a cube skin color model;

图3是敏感视频测试总体框图；Fig. 3 is an overall block diagram of sensitive video testing;

图4是不同的δ值对敏感视频监测的影响；Figure 4 is the impact of different δ values on sensitive video monitoring;

图5是视频中的敏感帧分布示意图。Fig. 5 is a schematic diagram of distribution of sensitive frames in a video.

具体实施方式Detailed ways

视频中的运动对象边界提取：Boundary extraction of moving objects in video:

在一个视频中分割出运动对象是视频处理，视频压缩和计算机视觉中最难和最重要的问题之一。传统的方法是先做运动参数估计，然后再做分割，这样如果运动估计不够精确的话，分割的质量就很差。在这里，我们采用level set对偏微分方程演化的方法来进行运动边界的确定和分割。Level set是由Osher和Sethian提出的一种偏微方程的数值解法，近年来在计算机视觉和图形学界引起广泛关注。与传统的在图像上做分割建立偏微分方程不同的是我们在视频序列中建立的方程利用了运动信息。Segmenting moving objects in a video is one of the hardest and most important problems in video processing, video compression and computer vision. The traditional method is to do motion parameter estimation first, and then do segmentation, so if the motion estimation is not accurate enough, the quality of the segmentation will be poor. Here, we use the method of level set to evolve partial differential equations to determine and segment motion boundaries. Level set is a numerical solution of partial differential equations proposed by Osher and Sethian, which has attracted widespread attention in the fields of computer vision and graphics in recent years. Different from the traditional partial differential equations established by segmentation on images, our equations established in video sequences utilize motion information.

设r(x，y，t)表示初始曲线r0所产生的曲线簇，假设在方向上速度为F，则曲线速率表示为：Let r(x, y, t) denote the cluster of curves generated by the initial curve r0, assuming that in the direction The upper speed is F, then the curve speed is expressed as:

${r r}_{t t} ((x x,, y the y,, t t)) = = F f \overset{&OverBar; &OverBar;}{N N}$

设闭曲线r(t)表示为隐函数形式：Let the closed curve r(t) be expressed as an implicit function form:

Φ(r(x，y，t)，t)＝0，初始条件为Φ(x，y，t＝0)＝r₀ Φ(r(x,y,t),t)=0, the initial condition is Φ(x,y,t=0)=r ₀

两边对t求导：Differentiate both sides with respect to t:

通过对固定网格微分解上述PDE：By differentiating the above PDE on a fixed grid:

${Φ Φ}_{i i,, j j}^{n no + + 11} = = {Φ Φ}_{i i,, j j}^{n no} - - Δt Δt \cdot \cdot h h \cdot \cdot ((max max (({F f}_{i i,, j j},, 00)) {&dtri; &dtri;}^{+ +} + + min min (({F f}_{i i,, j j},, 00)) {&dtri; &dtri;}^{- -}))$

其中h为网格步长，n为迭代次数，Δt为时间步长，Φ_i，j ⁿ是像素(i，j)在时间为n时的level值，F_i，j表示相应速度。并且：Where h is the grid step size, n is the number of iterations, Δt is the time step size, Φ _{i, j} ⁿ is the level value of the pixel (i, j) at time n, F _{i, j} represents the corresponding velocity. and:

${&dtri; &dtri;}^{+ +} = = {((max max {(({Φ Φ}_{i i,, j j} - - {Φ Φ}_{i i - - i i,, j j},, 00))}^{22} + + min min {(({Φ Φ}_{i i + + i i,, j j} - - {Φ Φ}_{i i,, j j},, 00))}^{22} + + max max {(({Φ Φ}_{i i,, j j} - - {Φ Φ}_{i i,, j j - - 11},, 00))}^{22} + + min min {(({Φ Φ}_{i i,, j j + + 11} - - {Φ Φ}_{i i,, j j},, 00))}^{22}))}^{\frac{11}{22}}$

${&dtri; &dtri;}^{+ +} = = {((max max {(({Φ Φ}_{i i + + i i,, j j} - - {Φ Φ}_{i i,, j j},, 00))}^{22} + + min min {(({Φ Φ}_{i i,, j j} - - {Φ Φ}_{i i - - 11,, j j},, 00))}^{22} + + max max {(({Φ Φ}_{i i,, j j + + i i} - - {Φ Φ}_{i i,, j j},, 00))}^{22} + + min min {(({Φ Φ}_{i i,, j j} - - {Φ Φ}_{i i,, j j - - 11},, 00))}^{22}))}^{\frac{11}{22}}$

通常在静态图像的时候，速度F是由图像梯度决定的，而在视频序列中，我们可以利用运动信息。速度F如下所示：Usually in the case of static images, the velocity F is determined by the image gradient, while in video sequences, we can use motion information. The speed F is as follows:

其中K为曲率，r是常量，g(I_D，σ_D)是对帧间差的高斯估计，g(|I|，σ_T)是对图像梯度I的高斯估计。where K is the curvature, r is a constant, g(I _D , σ _D ) is a Gaussian estimate of the inter-frame difference, and g(|I|, σ _T ) is a Gaussian estimate of the image gradient I.

视频中的运动区域分割与边界提取如图1所示。Motion region segmentation and boundary extraction in video are shown in Figure 1.

视频运动对象中的皮肤检测：Skin detection in video moving objects:

判断图像的点(x₀，y₀)是否在闭曲线内部：Determine whether the point (x ₀ , y ₀ ) of the image is in a closed curve internal:

假设曲线

上坐标为x₀的纵坐标集合为U_y，曲线

上坐标为y₀的横坐标集合为U_x。如果满足条件：U_y的元素个数大于1，y₀在U_y的最小与最大元素之间，U_x的元素个数大于1，x₀在U_x的最小与最大元素之间，则判定点(x₀，y₀)在闭曲线内部。但是此方法只对凸性的闭曲线有效。闭曲线所包围的面积就是所有在闭曲线

内的像素的总和。在闭曲线内部的像素中，检验其是否为肤色，我们采用了基于数据库统计的立方体肤色模型。hypothetical curve

The set of ordinates whose upper coordinate is x ₀ is U _y , and the curve

The set of abscissas whose upper coordinate is y ₀ is U _x . If the conditions are met: the number of elements of U _y is greater than 1, y ₀ is between the minimum and maximum elements of U _y , the number of elements of U _x is greater than 1, and x ₀ is between the minimum and maximum elements of U _x , the decision is made The point (x ₀ , y ₀ ) is on the closed curve internal. But this method is only valid for convex closed curves. closed curve The enclosed area is all the closed curves

The sum of the pixels within. In the pixels inside the closed curve, to check whether it is skin color, we use a cube skin color model based on database statistics.

传统肤色模型[Jones 1998]：Traditional skin tone model [Jones 1998]:

在RGB空间中，r，g，b三分量不仅代表颜色，还代表光照的明暗。为消除光照影响，对颜色采用正则化处理：r＝R/(R+G+B)，b＝B/(R+G+B).颜色模型可以用高斯模型N(m，c)表示。In the RGB space, the r, g, and b three components not only represent the color, but also represent the brightness and darkness of the light. In order to eliminate the influence of light, the color is regularized: r=R/(R+G+B), b=B/(R+G+B). The color model can be expressed by the Gaussian model N(m, c).

均值：m＝E{x}其中x＝(rb)^T Mean: m=E{x} where x=(rb) ^T

方差：c＝E{(x-m)(x-m)^T}Variance: c=E{(xm)(xm) ^T }

P(r，b)＝exp[-0.5(x-m)^Tc^-1(x-m)]其中x＝(r b)^T P(r,b)=exp[-0.5(xm) ^T c ^-1 (xm)] where x=(r b) ^T

通过取一定阈值，就能把皮肤分割出来。By taking a certain threshold, the skin can be segmented.

其缺陷：事实并非如此，可能比多高斯分布还要复杂，另外反馈麻烦Its defect: This is not the case, it may be more complicated than the multi-Gaussian distribution, and the feedback is troublesome

我们所采用的方法：The approach we take:

我们采用了一种基于立方体统计的方法。对于一个RGB立方体，其大小为256×256×256。我们将立方体进行细分，每个小立方体大小为8×8×8，总共得到32×32×32个立方体。立方体肤色模型如图2所示。We adopted an approach based on cube statistics. For an RGB cube, its size is 256×256×256. We subdivide the cube, and each small cube has a size of 8×8×8, resulting in a total of 32×32×32 cubes. The cube skin color model is shown in Figure 2.

同时，为了统计的精确性，我们增加了每个小立方体内的约束，以此为依据，我们进行数据库设计，并动态建立皮肤数据库。同时数据库具有如下特点：可以在识别过程中动态建立数据库，动态反馈。可以对数据库进行快速检索(库的记录一般在3万条左右)At the same time, for statistical accuracy, we increased the constraints in each small cube, based on this, we designed the database and dynamically built the skin database. At the same time, the database has the following characteristics: the database can be dynamically established during the identification process, and the dynamic feedback can be provided. The database can be quickly searched (the records in the database are generally about 30,000)

视频敏感程度估计：Video Sensitivity Estimation:

每帧的敏感程度f(t)可以按如下方式做一个评估：The sensitivity f(t) of each frame can be evaluated as follows:

闭曲线所包围的面积就是所有在闭曲线

内的像素的总和。整个视频的敏感程度E可以按如下方式做一个评估：closed curve The enclosed area is all the closed curves

The sum of the pixels within. The sensitivity E of the entire video can be evaluated as follows:

$E E. = = \frac{\underset{t t 22 - - t t 11 = = δ δ}{Max Max} {&Integral; &Integral;}_{t t 11}^{t t 22} f f ((t t)) dt dt}{δ δ}$

上述方程实际上计算的是从t1到t1+δ之间的平均敏感程度，并取一个上限。不同的帧间距离差δ值对敏感视频监测的影响如图4所示，一般取δ为4。The above equation actually calculates the average sensitivity from t1 to t1+δ, and takes an upper limit. The impact of different inter-frame distance difference δ values on sensitive video monitoring is shown in Figure 4, and δ is generally taken as 4.

实施例Example

整个敏感视频检测系统做成COM组件的形式。首先是输入一段视频，视频的输入可以是本地输入，也可以接收远程的视频URL输入。当接收远程URL时，该组件可以自动完成视频下载功能，并且以流媒体的方式进行下载和播放。在视频下载的同时进行视频解压缩处理，然后计算每一帧的的运动分割区域和边界。对闭区域内的像素进行皮肤检测，在进行皮肤分割时先读取皮肤数据库信息，在预先建立的肤色模型的基础上进行皮肤分割。敏感视频的检测与静态敏感图象的检测有所不同：静态图象是单帧的，这一幅图象要么是敏感，要么非敏感。而视频则不同，里面含有较多的冗余信息，如果某一帧是敏感的，那最好不要马上决策这段视频是敏感的，因为这样会使检测的错误率提高。因为根据常识，如果一段视频是敏感的，那么这段视频绝不可能只有一个关键帧是敏感的。因此，我们就需要计算敏感帧的分布情况。如果在某个时间段敏感关键帧的分布密度过高，则我们有理由认为这一段视频就含有敏感信息。实际上，在敏感帧分布密度的基础上来判别视频的敏感性，其准确性往往比静态的敏感图象检测要高。敏感视频的检测框图如附图3所示。The entire sensitive video detection system is made in the form of COM components. The first is to input a piece of video. The video input can be local input or remote video URL input. When receiving a remote URL, this component can automatically complete the video download function, and download and play in the way of streaming media. The video decompression process is performed while the video is downloaded, and then the motion segmentation area and boundary of each frame are calculated. Perform skin detection on the pixels in the closed area, and read the skin database information when performing skin segmentation, and perform skin segmentation on the basis of the pre-established skin color model. The detection of sensitive video is different from the detection of static sensitive images: a static image is a single frame, and this image is either sensitive or non-sensitive. The video is different, which contains more redundant information. If a certain frame is sensitive, it is best not to immediately decide that this video is sensitive, because this will increase the detection error rate. Because according to common sense, if a video is sensitive, then it is absolutely impossible that only one key frame of this video is sensitive. Therefore, we need to calculate the distribution of sensitive frames. If the distribution density of sensitive key frames is too high in a certain period of time, we have reason to think that this segment of video contains sensitive information. In fact, the accuracy of judging video sensitivity based on the distribution density of sensitive frames is often higher than static sensitive image detection. The block diagram of sensitive video detection is shown in Figure 3.

敏感性评测：Sensitivity Evaluation:

我们通过对每一关键帧进行敏感性估计，得到视频的敏感帧分布情况，如附图5所示，其中红色代表可能含有敏感信息的帧。We estimate the sensitivity of each key frame to obtain the distribution of sensitive frames of the video, as shown in Figure 5, where red represents frames that may contain sensitive information.

通过对敏感帧分布密度进行估计，可以决策出视频是否敏感。我们在100幅视频的样本上做检测，对敏感视频检测的准确率达到86.5％，误检率为4％。By estimating the distribution density of sensitive frames, it can be determined whether the video is sensitive or not. We do detection on a sample of 100 videos, and the accuracy rate of sensitive video detection reaches 86.5%, and the false detection rate is 4%.

Claims

1. A sensitive video detection method based on motion skin color segmentation, comprising steps:

Segmentation and boundary extraction of moving objects in the video using the lever set method of evolution of partial differential equations;

Use the cube skin color model based on the relational database to detect the skin color on the segmented object, and find out the exposure degree of the skin relative to the moving object;

The sensitivity of the entire video is comprehensively evaluated on the basis of calculating the single-frame sensitivity f(t) for each frame.

2. The method according to claim 1, wherein said skin detection comprises the steps of:

Determine whether the image point is inside the closed curve;

Subdivides the RGB cube.

3. The method according to claim 2, characterized in that each subdivided small cube is constrained.

4. The method according to claim 2 or 3, characterized in that the subdivided cubes and constraints in the cubes are stored in a database, and the constraints are dynamically updated.

5. by the described method of claim 1, it is characterized in that the sensitivity of each frame is evaluated as follows:

Wherein, the area enclosed by the closed curve is the sum of all pixels within the closed curve.

6. by the described method of claim 1 or 5, it is characterized in that the degree of sensitivity E of whole video is evaluated as follows:

E E. = = \frac{\underset{t t 22 - - t t 11 = = δ δ}{Max Max} {&Integral; &Integral;}_{t t 11}^{t t 22} f f ((t t)) dt dt}{δ δ}

Among them, δ is the distance difference between frames.

7. The method according to claim 6, characterized in that the interframe distance difference δ is 4.