[go: up one dir, main page]

CN111476273A - Image processing method and device - Google Patents

Image processing method and device Download PDF

Info

Publication number
CN111476273A
CN111476273A CN202010168017.3A CN202010168017A CN111476273A CN 111476273 A CN111476273 A CN 111476273A CN 202010168017 A CN202010168017 A CN 202010168017A CN 111476273 A CN111476273 A CN 111476273A
Authority
CN
China
Prior art keywords
image frame
processed
image
sensitive
type
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202010168017.3A
Other languages
Chinese (zh)
Other versions
CN111476273B (en
Inventor
李超
范志刚
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Xian Wanxiang Electronics Technology Co Ltd
Original Assignee
Xian Wanxiang Electronics Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Xian Wanxiang Electronics Technology Co Ltd filed Critical Xian Wanxiang Electronics Technology Co Ltd
Priority to CN202410557422.2A priority Critical patent/CN118537603A/en
Priority to CN202010168017.3A priority patent/CN111476273B/en
Publication of CN111476273A publication Critical patent/CN111476273A/en
Application granted granted Critical
Publication of CN111476273B publication Critical patent/CN111476273B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/70Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74Image or video pattern matching; Proximity measures in feature spaces
    • G06V10/75Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
    • G06V10/751Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44Arrangements for executing specific programs
    • G06F9/455Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533Hypervisors; Virtual machine monitors
    • G06F9/45558Hypervisor-specific management and integration aspects
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/40Extraction of image or video features
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/70Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/70Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10Character recognition
    • G06V30/18Extraction of features or characteristics of the image
    • G06V30/1801Detecting partial patterns, e.g. edges or contours, or configurations, e.g. loops, corners, strokes or intersections
    • G06V30/18019Detecting partial patterns, e.g. edges or contours, or configurations, e.g. loops, corners, strokes or intersections by matching or filtering
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10Character recognition
    • G06V30/19Recognition using electronic means
    • G06V30/19007Matching; Proximity measures
    • G06V30/19013Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44Arrangements for executing specific programs
    • G06F9/455Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
    • G06F9/45533Hypervisors; Virtual machine monitors
    • G06F9/45558Hypervisor-specific management and integration aspects
    • G06F2009/45595Network integration; Enabling network access in virtual machine instances

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • General Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Databases & Information Systems (AREA)
  • Medical Informatics (AREA)
  • General Engineering & Computer Science (AREA)
  • Biomedical Technology (AREA)
  • Mathematical Physics (AREA)
  • Molecular Biology (AREA)
  • Data Mining & Analysis (AREA)
  • Computational Linguistics (AREA)
  • Biophysics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Image Processing (AREA)

Abstract

The present disclosure provides an image processing method and apparatus, relating to the technical field of computer images, wherein the method comprises: acquiring a rendering instruction, wherein the rendering instruction is generated according to an operation message of terminal equipment; rendering according to the rendering instruction to generate an image frame to be processed; identifying the type of the image frame to be processed, wherein the type of the image frame to be processed comprises a variable image frame and a constant image frame; filtering the image frame to be processed based on the type of the image frame to be processed to obtain a target image frame; and sending the target image frame to the terminal equipment. The method and the device can realize that the user cannot watch the sensitive words or the sensitive images on the terminal equipment.

Description

图像处理方法及装置Image processing method and device

技术领域technical field

本公开涉及计算机图像技术领域,尤其涉及图像处理方法及装置。The present disclosure relates to the field of computer image technology, and in particular, to an image processing method and apparatus.

背景技术Background technique

随着云端虚拟化技术的迅速发展,企业对于虚拟桌面云系统的需求进一步加大。虚拟桌面云系统包括云端服务器和多个用户终端(R端),云端服务器和R端之间基于图片进行传输。在云端服务器安装系统和应用程序,由云端服务器完成几乎所有的处理任务,而R端为具有编码功能的显示器。云端服务器生成多个虚拟机(Virtual machine,VM),一个VM对应一个R端,用户通过R端,对VM进行操作。在由于虚拟桌面云系统中的用户终端不会获取数据信息本身,比如原代码,也不具有存储功能,因此,即使用户终端被攻击或盗取,也不会丢失重要数据,安全性较高。With the rapid development of cloud virtualization technology, enterprises have further increased demand for virtual desktop cloud systems. The virtual desktop cloud system includes a cloud server and multiple user terminals (R terminals), and pictures are transmitted between the cloud server and the R terminal. Install systems and applications on the cloud server, and the cloud server completes almost all processing tasks, while the R terminal is a display with coding function. The cloud server generates multiple virtual machines (Virtual machines, VMs), one VM corresponds to one R terminal, and the user operates the VM through the R terminal. Because the user terminal in the virtual desktop cloud system does not obtain the data information itself, such as the original code, and does not have the storage function, therefore, even if the user terminal is attacked or stolen, important data will not be lost, and the security is high.

在虚拟桌面云系统应用于企业办公的场景下,如何对员工的办公行为进行管理限制,是尚待解决的问题。In the scenario where the virtual desktop cloud system is applied to corporate office, how to manage and restrict employees' office behavior is a problem to be solved.

发明内容SUMMARY OF THE INVENTION

本公开实施例提供一种图像处理方法及装置,本公开能够实现用户在终端设备上无法观看到敏感字或敏感图像。所述技术方案如下:Embodiments of the present disclosure provide an image processing method and apparatus, and the present disclosure can realize that a user cannot view sensitive words or sensitive images on a terminal device. The technical solution is as follows:

根据本公开实施例的第一方面,提供一种图像处理方法,应用于图像处理装置,该方法包括:According to a first aspect of the embodiments of the present disclosure, there is provided an image processing method, which is applied to an image processing apparatus, and the method includes:

获取渲染指令,所述渲染指令是根据终端设备的操作消息生成的;Obtaining a rendering instruction, the rendering instruction is generated according to the operation message of the terminal device;

根据所述渲染指令进行渲染,生成待处理图像帧;Rendering according to the rendering instruction to generate a to-be-processed image frame;

基于所述待处理图像帧的类型对所述待处理图像帧进行过滤处理,得到目标图像帧;Perform filtering processing on the to-be-processed image frame based on the type of the to-be-processed image frame to obtain a target image frame;

将所述目标图像帧发送给所述终端设备。Send the target image frame to the terminal device.

在一个实施例中,基于所述待处理图像帧的类型对所述待处理图像帧进行过滤处理,得到目标图像帧包括:In one embodiment, filtering the to-be-processed image frame based on the type of the to-be-processed image frame to obtain the target image frame includes:

识别所述待处理图像帧的类型,待处理图像帧的类型包括变化图像帧和不变图像帧;Identifying the type of the image frame to be processed, the type of the image frame to be processed includes a changing image frame and an invariant image frame;

如果是变化图像帧,则将变化图像帧作为样本图像帧,根据预设图像识别算法判断所述样本图像帧是否为敏感类图像;If it is a changing image frame, the changing image frame is used as a sample image frame, and whether the sample image frame is a sensitive image is judged according to a preset image recognition algorithm;

如果是敏感类图像帧,将所述变化图像帧替换为预设图像帧。If it is a sensitive image frame, replace the changed image frame with a preset image frame.

在一个实施例中,该方法还包括:In one embodiment, the method further includes:

如果是不变图像帧,则记录所述不变图像帧,并确定下一图像帧的类型,直到检测到变化图像帧;If it is a constant image frame, record the constant image frame, and determine the type of the next image frame, until a changed image frame is detected;

在每M帧不变图像帧中,选取样本图像帧,根据预设图像识别算法判断所述样本图像帧是否为敏感类图像;In every M frames of invariant image frames, select a sample image frame, and determine whether the sample image frame is a sensitive image according to a preset image recognition algorithm;

如果是敏感类图像,将所述M帧不变图像帧替换为预设图像帧。If it is a sensitive image, replace the M frames of invariant image frames with preset image frames.

在一个实施例中,待处理图像帧包括至少一个宏块,识别所述待处理图像帧的类型包括:In one embodiment, the image frame to be processed includes at least one macroblock, and identifying the type of the image frame to be processed includes:

判断所述待处理图像帧中的宏块与前一图像帧中对应位置的宏块的像素点是否完全相同;Judging whether the macroblock in the to-be-processed image frame is exactly the same as the pixel point of the macroblock at the corresponding position in the previous image frame;

如果完全相同,则识别所述待处理图像帧为不变宏块,若不完全相同,则识别所述待处理图像帧为变化宏块;If they are identical, then identify the to-be-processed image frame as a constant macroblock, if not, identify the to-be-processed image frame as a changed macroblock;

统计所述不变宏块的数量,如果所述不变宏块的数量不大于预设阈值,则确定所述待处理图像帧为不变图像帧;如果所述不变宏块的数量小于预设阈值,则确定所述待处理图像帧为变化图像帧。Count the number of unchanged macroblocks, and if the number of unchanged macroblocks is not greater than a preset threshold, then determine that the image frame to be processed is an unchanged image frame; if the number of unchanged macroblocks is less than the predetermined threshold If the threshold is set, it is determined that the image frame to be processed is a changed image frame.

在一个实施例中,识别所述待处理图像帧的类型之前,该方法还包括:In one embodiment, before identifying the type of the image frame to be processed, the method further includes:

确定所述待处理图像帧中包含视频宏块。It is determined that the to-be-processed image frame contains video macroblocks.

在一个实施例中,根据预设图像识别算法判断所述待处理图像帧是否为敏感类图像包括:In one embodiment, judging whether the image frame to be processed is a sensitive image according to a preset image recognition algorithm includes:

提取所述样本图像帧中的关键字/特征值,将提取的关键字/特征值与预设的敏感关键字/敏感特征值进行比对;Extracting keywords/feature values in the sample image frame, and comparing the extracted keywords/feature values with preset sensitive keywords/sensitive feature values;

如果样本图像帧的关键字/特征值中包含预设的敏感关键字/敏感特征值,则确定所述样本图像帧为敏感类图像。If the keyword/feature value of the sample image frame contains a preset sensitive keyword/sensitive feature value, the sample image frame is determined to be a sensitive image.

在一个实施例中,渲染指令是服务器接收终端设备发送的至少一个操作消息,根据所述账号信息分配与所述终端设备对应的虚拟机,通过所述虚拟机依次解析所述至少一个操作消息,根据解析结果生成的,所述至少一个操作消息包括账号信息。In one embodiment, the rendering instruction is that the server receives at least one operation message sent by a terminal device, allocates a virtual machine corresponding to the terminal device according to the account information, and sequentially parses the at least one operation message through the virtual machine, Generated according to the parsing result, the at least one operation message includes account information.

在一个实施例中,该方法还包括:In one embodiment, the method further includes:

根据所述账号信息,统计每个账号信息对应的敏感类图像。According to the account information, the sensitive images corresponding to each account information are counted.

根据本公开实施例的第二方面,提供一种图像处理装置,该装置包括:According to a second aspect of the embodiments of the present disclosure, there is provided an image processing apparatus, the apparatus comprising:

获取模块,用于获取渲染指令,所述渲染指令是根据终端设备的操作消息生成的;an acquisition module, used for acquiring rendering instructions, the rendering instructions are generated according to the operation message of the terminal device;

渲染模块,用于根据所述渲染指令进行渲染,生成待处理图像帧;a rendering module, configured to perform rendering according to the rendering instruction, and generate a to-be-processed image frame;

识别模块,用于识别所述待处理图像帧的类型,待处理图像帧的类型包括变化图像帧和不变图像帧;an identification module for identifying the type of the image frame to be processed, and the type of the image frame to be processed includes a changing image frame and an invariant image frame;

处理模块,用于基于所述待处理图像帧的类型对所述待处理图像帧进行过滤处理,得到目标图像帧;a processing module, configured to filter the to-be-processed image frame based on the type of the to-be-processed image frame to obtain a target image frame;

发送模块,用于将所述目标图像帧发送给所述终端设备。A sending module, configured to send the target image frame to the terminal device.

在一个实施例中,处理模块包括:In one embodiment, the processing module includes:

判断子模块,用于如果是变化图像帧,则将变化图像帧作为样本图像帧,根据预设图像识别算法判断所述样本图像帧是否为敏感类图像;a judging sub-module, configured to use the changed image frame as a sample image frame if it is a changing image frame, and judge whether the sample image frame is a sensitive image according to a preset image recognition algorithm;

第一替换子模块,用于将所述变化图像帧替换为预设图像帧。The first replacement submodule is configured to replace the changed image frame with a preset image frame.

在一个实施例中,处理模块包括:In one embodiment, the processing module includes:

记录子模块,用于如果是不变图像帧,则记录所述不变图像帧,并确定下一图像帧的类型,直到检测到变化图像帧;a recording submodule, for recording the invariant image frame if it is an invariant image frame, and determining the type of the next image frame, until the changed image frame is detected;

选取子模块,用于在每M帧不变图像帧中,选取样本图像帧,根据预设图像识别算法判断所述样本图像帧是否为敏感类图像;A selection sub-module for selecting sample image frames in every M frames of invariant image frames, and judging whether the sample image frames are sensitive images according to a preset image recognition algorithm;

第二替换子模块,用于如果是敏感类图像,将所述M帧不变图像帧替换为预设图像帧。The second replacement sub-module is configured to replace the M frames of unchanged image frames with preset image frames if it is a sensitive image.

在一个实施例中,待处理图像帧包括宏块,识别模块包括:In one embodiment, the image frame to be processed includes macroblocks, and the identification module includes:

判断子模块,用于判断所述待处理图像帧中的宏块与前一图像帧中对应位置的宏块的像素点是否完全相同;Judging submodule, for judging whether the macroblock in the image frame to be processed is exactly the same as the pixel point of the macroblock at the corresponding position in the previous image frame;

标识子模块,用于如果完全相同,则识别所述待处理图像帧为不变宏块,若不完全相同,则识别所述待处理图像帧为变化宏块;an identification submodule, used for identifying the to-be-processed image frame as an invariant macroblock if it is identical, and identifying the to-be-processed image frame as a changing macroblock if not identical;

统计子模块,用于统计所述不变宏块的数量,如果所述不变宏块的数量不大于预设阈值,则确定所述待处理图像帧为不变图像帧;如果所述不变宏块的数量小于预设阈值,则确定所述待处理图像帧为变化图像帧。Statistics sub-module, used to count the number of unchanged macroblocks, if the number of unchanged macroblocks is not greater than a preset threshold, then determine that the image frame to be processed is an unchanged image frame; If the number of macroblocks is less than the preset threshold, it is determined that the to-be-processed image frame is a changed image frame.

在一个实施例中,该装置还包括:In one embodiment, the apparatus further includes:

确定模块,用于识别所述待处理图像帧的类型之前,确定所述待处理图像帧中包含视频宏块。The determining module is configured to determine that the to-be-processed image frame contains video macroblocks before identifying the type of the to-be-processed image frame.

在一个实施例中,判断子模块包括:In one embodiment, the judging submodule includes:

提取子单元,用于提取所述样本图像帧中的关键字/特征值,将提取的关键字/特征值与预设的敏感关键字/敏感特征值进行比对;an extraction subunit, used for extracting keywords/feature values in the sample image frame, and comparing the extracted keywords/feature values with preset sensitive keywords/sensitive feature values;

确定子单元,用于如果样本图像帧的关键字/特征值中包含预设的敏感关键字/敏感特征值,则确定所述样本图像帧为敏感类图像。A determination subunit, configured to determine that the sample image frame is a sensitive image if the keyword/feature value of the sample image frame contains a preset sensitive keyword/sensitive feature value.

在一个实施例中,渲染指令是服务器接收终端设备发送的至少一个操作消息,根据所述账号信息分配与所述终端设备对应的虚拟机,通过所述虚拟机依次解析所述至少一个操作消息,根据解析结果生成的,所述至少一个操作消息包括账号信息。In one embodiment, the rendering instruction is that the server receives at least one operation message sent by a terminal device, allocates a virtual machine corresponding to the terminal device according to the account information, and sequentially parses the at least one operation message through the virtual machine, Generated according to the parsing result, the at least one operation message includes account information.

在一个实施例中,该装置还包括:In one embodiment, the apparatus further includes:

统计模块,用于根据所述账号信息,统计每个账号信息对应的敏感类图像。The statistics module is configured to count the sensitive images corresponding to each account information according to the account information.

在本公开中,可以对渲染出的图像帧进行识别和分类,对于不符合企业要求的敏感类图像,进行替换,再将替换后的图像帧发送给终端设备,使得员工在终端设备无法观看到敏感类图像,保证办公场所的良好氛围,提高员工工作效率。In the present disclosure, the rendered image frames can be identified and classified, and the sensitive images that do not meet the requirements of the enterprise can be replaced, and then the replaced image frames can be sent to the terminal device, so that the employees cannot view it on the terminal device. Sensitive images can ensure a good atmosphere in the office and improve the work efficiency of employees.

应当理解的是,以上的一般描述和后文的细节描述仅是示例性和解释性的,并不能限制本公开。It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure.

附图说明Description of drawings

此处的附图被并入说明书中并构成本说明书的一部分,示出了符合本公开的实施例,并与说明书一起用于解释本公开的原理。The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the disclosure and together with the description serve to explain the principles of the disclosure.

图1是本公开实施例提供的一种图像处理方法流程图;FIG. 1 is a flowchart of an image processing method provided by an embodiment of the present disclosure;

图2是本公开实施例提供的变化图像帧过滤处理流程图;FIG. 2 is a flowchart of filtering processing of a changed image frame provided by an embodiment of the present disclosure;

图3是本公开实施例提供的不变图像帧过滤处理流程图;3 is a flowchart of an invariant image frame filtering process provided by an embodiment of the present disclosure;

图4是本公开实施例提供的一种图像处理装置结构图;FIG. 4 is a structural diagram of an image processing apparatus provided by an embodiment of the present disclosure;

图5是本公开实施例提供的一种图像处理装置处理模块结构图;5 is a structural diagram of a processing module of an image processing apparatus provided by an embodiment of the present disclosure;

图6是本公开实施例提供的一种图像处理装置处理模块结构图;6 is a structural diagram of a processing module of an image processing apparatus provided by an embodiment of the present disclosure;

图7是本公开实施例提供的一种图像处理装置识别模块结构图;7 is a structural diagram of an image processing device identification module provided by an embodiment of the present disclosure;

图8是本公开实施例提供的一种图像处理装置结构图;8 is a structural diagram of an image processing apparatus provided by an embodiment of the present disclosure;

图9是本公开实施例提供的一种图像处理装置判断子模块结构图;9 is a structural diagram of a judgment sub-module of an image processing apparatus provided by an embodiment of the present disclosure;

图10是本公开实施例提供的一种图像处理装置结构图;10 is a structural diagram of an image processing apparatus provided by an embodiment of the present disclosure;

图11是本本公开实施例提供的一种图像处理系统应用结构图;11 is an application structure diagram of an image processing system provided by an embodiment of the present disclosure;

图12是本公开实施例提供的基于图11系统的图像处理方法流程图。FIG. 12 is a flowchart of an image processing method based on the system of FIG. 11 provided by an embodiment of the present disclosure.

具体实施方式Detailed ways

这里将详细地对示例性实施例进行说明,其示例表示在附图中。下面的描述涉及附图时,除非另有表示,不同附图中的相同数字表示相同或相似的要素。以下示例性实施例中所描述的实施方式并不代表与本公开相一致的所有实施方式。相反,它们仅是与如所附权利要求书中所详述的、本公开的一些方面相一致的装置和方法的例子。Exemplary embodiments will be described in detail herein, examples of which are illustrated in the accompanying drawings. Where the following description refers to the drawings, the same numerals in different drawings refer to the same or similar elements unless otherwise indicated. The implementations described in the illustrative examples below are not intended to represent all implementations consistent with this disclosure. Rather, they are merely examples of apparatus and methods consistent with some aspects of the present disclosure as recited in the appended claims.

以下描述的一部分明确地或者暗含地涉及算法和对计算机存储器内数据的操作的功能或者符号表示。这些算法的描述和功能或者符号表示是图像处理领域内技术人员用于更有效地向本领域内其它技术人员表达他们工作实质的方法。此处通常将算法设想为生成期望结果的一系列有条理的步骤。这些步骤是需要对诸如能够存储、传送、组合、对比以及通过其它方式操控的电、磁或者光信号的物理量进行物理操控的步骤。Portions of the following description relate explicitly or implicitly to algorithms and functional or symbolic representations of operations on data within a computer memory. The descriptions and functional or symbolic representations of these algorithms are the means used by those skilled in the image processing arts to more effectively convey the substance of their work to others skilled in the art. Algorithms are often conceived here as a series of organized steps leading to a desired result. These steps are those requiring physical manipulations of physical quantities such as electrical, magnetic, or optical signals capable of being stored, transferred, combined, compared, and otherwise manipulated.

除非特别说明,否则如以下可显而易见地,应该理解本说明书通篇使用的诸如“选取”、“渲染”、“显示”、“发送”、“获取”、“生成”等术语的讨论,涉及计算机系统或者类似电子设备的动作和处理,上述的电子设备将表示为计算机系统内物理量的数据操控和转换成同样表示为计算机系统或者其它信息存储、传输或者显示设备内物理量的其它数据。Unless otherwise stated, it should be understood that discussions of terms such as "selecting," "rendering," "displaying," "sending," "acquiring," "generating," etc. used throughout this specification are understood to refer to computer The actions and processing of systems or similar electronic devices that manipulate and convert data represented as physical quantities within a computer system into other data also represented as physical quantities within a computer system or other information storage, transmission, or display device.

说明书还公开了用于执行方法操作的设备。这种设备为所需的目的而特别构成,或者可以包括通用计算机或者其它存储在计算机中的计算机程序选择性启动或者重新配置的其它设备。本文介绍的算法和显示不是固有地与任何具体计算机或者其它设备相关。各种通用机器可以根据本文教导的程序一起使用。可替换地,用于执行所要求的方法步骤的更特殊的设备构造是可以适用的。常规的通用计算机的结构将在以下描述中介绍。The specification also discloses apparatus for performing the operations of the method. Such apparatus is specially constructed for the required purposes, or it may comprise a general purpose computer or other apparatus selectively activated or reconfigured by a computer program stored in the computer. The algorithms and displays described herein are not inherently related to any particular computer or other device. Various general-purpose machines can be used with the programs taught herein. Alternatively, more specific apparatus configurations for performing the required method steps may be applicable. The structure of a conventional general-purpose computer will be described in the following description.

此外,由于可以由计算机代码实施本文描述方法的各步骤对本领域技术人员是显而易见的,因此本说明书还暗含地公开计算机程序。该计算机程序不试图限制于任何具体的编程语言及其执行。应该理解,可以使用多种编程语言及其代码以执行本文包含的公开的教导。此外,该计算机程序不试图限制于任何具体的控制流。在不脱离本公开精神或者范围的情况下,存在许多其它种类的、可以使用不同控制流的计算机程序。Furthermore, this specification also implicitly discloses computer programs, since it will be apparent to those skilled in the art that the steps of the methods described herein can be implemented by computer code. The computer program is not intended to be limited to any particular programming language and its implementation. It should be understood that a variety of programming languages and codes thereof may be used to implement the teachings of the disclosure contained herein. Furthermore, the computer program is not intended to be limited to any particular flow of control. There are many other kinds of computer programs that can use different control flows without departing from the spirit or scope of the present disclosure.

而且,可以并行地而不是顺序地执行计算机程序的一个或者多个步骤。这种计算机程序可以存储在任何计算机可读介质上。计算机可读介质可以包括的存储设备诸如为磁盘或者光盘、存储器芯片或者适于与通用计算机接口的其它存储设备等。计算机可读介质还可以包括诸如在因特网系统中的硬接线介质,或者无线介质。当在这种通用计算机上加载和执行计算机程序时,计算机程序有效地产生实施优选方法的步骤的设备。Furthermore, one or more steps of a computer program may be executed in parallel rather than sequentially. Such a computer program can be stored on any computer-readable medium. The computer-readable medium may include storage devices such as magnetic or optical disks, memory chips, or other storage devices suitable for interfacing with a general purpose computer, and the like. Computer-readable media may also include hard-wired media, such as in Internet systems, or wireless media. When loaded and executed on such a general purpose computer, the computer program effectively produces an apparatus for carrying out the steps of the preferred method.

本公开还可被实施为硬件模块。更具体地,在硬件意义下,模块是被设计为与其它部件或模块一起使用的功能性硬件单元。例如,模块可使用分立电子部件实施,或者其可以形成整个电子电路诸如特定用途集成电路(ASIC)的一部分。还存在许多其它可能。本领域技术人员应理解,该系统还可被实施为硬件和软件模块的组合。The present disclosure may also be implemented as hardware modules. More specifically, in the hardware sense, a module is a functional hardware unit designed for use with other components or modules. For example, a module may be implemented using discrete electronic components, or it may form part of an overall electronic circuit such as an application specific integrated circuit (ASIC). Many other possibilities exist. Those skilled in the art will understand that the system can also be implemented as a combination of hardware and software modules.

图1是本公开实施例提供的一种图像处理方法流程图,应用于图像处理装置,如图1所示的图像处理方法包括以下步骤:FIG. 1 is a flowchart of an image processing method provided by an embodiment of the present disclosure, which is applied to an image processing apparatus. The image processing method shown in FIG. 1 includes the following steps:

步骤101、获取渲染指令,所述渲染指令是根据终端设备的操作消息生成的;Step 101, obtaining a rendering instruction, the rendering instruction is generated according to the operation message of the terminal device;

步骤102、根据所述渲染指令进行渲染,生成待处理图像帧;Step 102, performing rendering according to the rendering instruction to generate a to-be-processed image frame;

步骤103、识别所述待处理图像帧的类型,待处理图像帧的类型包括变化图像帧和不变图像帧;Step 103, identifying the type of the image frame to be processed, the type of the image frame to be processed includes a changing image frame and an invariant image frame;

具体的,待处理图像帧包括至少一个宏块,例如待处理图像帧被划分为多个宏块,每个宏块包括M*N个像素点,识别所述待处理图像帧的类型包括:Specifically, the image frame to be processed includes at least one macroblock. For example, the image frame to be processed is divided into multiple macroblocks, and each macroblock includes M*N pixels. Identifying the type of the image frame to be processed includes:

判断所述待处理图像帧中的宏块与前一图像帧中对应位置的宏块的像素点是否完全相同;Judging whether the macroblock in the to-be-processed image frame is exactly the same as the pixel point of the macroblock at the corresponding position in the previous image frame;

如果完全相同,则识别所述待处理图像帧为不变宏块,若不完全相同,则识别所述待处理图像帧为变化宏块;If they are identical, then identify the to-be-processed image frame as a constant macroblock, if not, identify the to-be-processed image frame as a changed macroblock;

示例性的,判断待处理图像帧中的宏块X与前一图像帧中的宏块Y的每个像素点是否完全相同,也就是说,需要比较宏块X与宏块Y中每个像素点的YUV三个值,YUV三个值全部相等,才认为两个像素点完全相同),其中,宏块Y在前一图像帧中的位置与宏块X在待处理图像帧中的位置相同;若待处理图像帧中宏块X中的像素点与前一图像帧中宏块Y的像素点完全相同,则可以确定宏块X为不变unchange宏块;若待处理图像帧中宏块X中的像素点与前一图像帧中宏块Y的像素点不完全相同,则可以确定宏块X为不变unchange宏块。Exemplarily, it is determined whether each pixel of the macroblock X in the image frame to be processed is exactly the same as that of the macroblock Y in the previous image frame, that is, it is necessary to compare the macroblock X and each pixel in the macroblock Y. The three values of YUV of the point, and all three values of YUV are equal, the two pixels are considered to be identical), wherein the position of the macroblock Y in the previous image frame is the same as the position of the macroblock X in the image frame to be processed ; If the pixel point in the macroblock X in the image frame to be processed is exactly the same as the pixel point in the macroblock Y in the previous image frame, then it can be determined that the macroblock X is a constant unchange macroblock; If the macroblock in the image frame to be processed is identical The pixels in X are not exactly the same as the pixels in the macroblock Y in the previous image frame, so it can be determined that the macroblock X is an unchanged macroblock.

统计所述不变宏块的数量,如果所述不变宏块的数量不大于预设阈值,则确定所述待处理图像帧为不变图像帧;如果所述不变宏块的数量小于预设阈值,则确定所述待处理图像帧为变化图像帧。Count the number of unchanged macroblocks, and if the number of unchanged macroblocks is not greater than a preset threshold, then determine that the image frame to be processed is an unchanged image frame; if the number of unchanged macroblocks is less than the predetermined threshold If the threshold is set, it is determined that the image frame to be processed is a changed image frame.

具体的,统计待处理图像帧中unchange宏块的数量,若unchange宏块的数量超过阈值,则说明当前图像帧与前一图像帧之间的区别较小,可以认为当前图像帧在前一图像帧基础上的变化较小;若unchange宏块的数量没有超过阈值,则可以认为待处理图像帧在前一图像帧基础上的变化较大,可使用标识符X对当前图像帧进行标记。Specifically, count the number of unchange macroblocks in the image frame to be processed. If the number of unchange macroblocks exceeds the threshold, it means that the difference between the current image frame and the previous image frame is small, and it can be considered that the current image frame is in the previous image frame. The change on the basis of the frame is small; if the number of unchange macroblocks does not exceed the threshold, it can be considered that the change of the image frame to be processed on the basis of the previous image frame is large, and the identifier X can be used to mark the current image frame.

其中,若待处理图像帧的总宏块数为100块,则阈值可以为80块。阈值也可以是当前图像帧中unchange宏块的数量,占当前图像帧总宏块数量的比例,比如,80%。Wherein, if the total number of macroblocks of the image frame to be processed is 100 blocks, the threshold value may be 80 blocks. The threshold value may also be the number of unchange macroblocks in the current image frame, which accounts for the proportion of the total number of macroblocks in the current image frame, for example, 80%.

步骤104、基于所述待处理图像帧的类型对所述待处理图像帧进行过滤处理,得到目标图像帧;Step 104, filtering the to-be-processed image frame based on the type of the to-be-processed image frame to obtain a target image frame;

如果待处理图像帧是变化图像帧,如图2所示,则将变化图像帧作为样本图像帧,执行以下步骤:If the image frame to be processed is a changing image frame, as shown in Figure 2, the changing image frame is used as a sample image frame, and the following steps are performed:

步骤1041、根据预设图像识别算法判断所述样本图像帧是否为敏感类图像;Step 1041: Determine whether the sample image frame is a sensitive image according to a preset image recognition algorithm;

步骤1042、如果所述样本理图像帧为敏感类图像,将所述变化图像帧替换为预设图像帧。Step 1042: If the sample logical image frame is a sensitive image, replace the changed image frame with a preset image frame.

如果待处理图像帧是不变图像帧,如图3所示,则执行以下步骤:If the image frame to be processed is an invariant image frame, as shown in Figure 3, perform the following steps:

步骤104a、记录所述不变图像帧,并确定下一图像帧的类型,直到检测到变化图像帧;Step 104a, record the invariable image frame, and determine the type of the next image frame, until the changed image frame is detected;

步骤104b、在每M帧不变图像帧中,选取样本图像帧,根据预设图像识别算法判断所述样本图像帧是否为敏感类图像;Step 104b, in every M frames of invariant image frames, select a sample image frame, and judge whether the sample image frame is a sensitive image according to a preset image recognition algorithm;

M为正整数,选取样本图像帧比如在30帧中任意选一帧作为样本图像帧。M is a positive integer, and a sample image frame is selected, for example, any one frame among 30 frames is selected as the sample image frame.

步骤104c、所述样本图像帧是否为敏感类图像,将所述M帧不变图像帧替换为预设图像帧。Step 104c: Whether the sample image frame is a sensitive image, replace the M frames of invariant image frames with preset image frames.

对归属于敏感类图像的图像帧进行替换,并将替换后的图像帧用标识符Z标记;具体的,可以替换为纯色图片,或替换为具有“无法显示”字样的图片。Replace the image frames belonging to the sensitive image, and mark the replaced image frame with the identifier Z; specifically, it can be replaced with a solid color picture, or with a picture with the word "unable to display".

步骤105、将所述目标图像帧发送给所述终端设备。Step 105: Send the target image frame to the terminal device.

在一个实施例中,识别所述待处理图像帧的类型之前,该方法还包括:In one embodiment, before identifying the type of the image frame to be processed, the method further includes:

确定所述待处理图像帧中包含视频宏块。It is determined that the to-be-processed image frame contains video macroblocks.

具体的,判断待处理图像帧的宏块是否包括视频video宏块,若为包含,则确定待处理图像帧为video图像帧,可以使用标识符Y对当前图像帧进行标记。Specifically, it is determined whether the macroblock of the image frame to be processed includes a video macroblock, and if so, it is determined that the image frame to be processed is a video image frame, and the identifier Y can be used to mark the current image frame.

该步骤使本公开实施例对视频图像处理时能提高处理效率。This step enables the embodiments of the present disclosure to improve processing efficiency when processing video images.

上述实施例中,根据预设图像识别算法判断样本图像帧是否为敏感类图像包括:In the above-described embodiment, whether the sample image frame is judged to be a sensitive image according to a preset image recognition algorithm includes:

提取所述待处理图像帧中的关键字/特征值,将提取的关键字/特征值与预设的敏感关键字/敏感特征值进行比对;Extracting keywords/feature values in the to-be-processed image frame, and comparing the extracted keywords/feature values with preset sensitive keywords/sensitive feature values;

如果待处理图像帧的关键字/特征值中包含预设的敏感关键字/敏感特征值,则确定所述待处理图像帧为敏感类图像。If the keyword/feature value of the image frame to be processed includes a preset sensitive keyword/sensitive feature value, the image frame to be processed is determined to be a sensitive image.

另外,可以用目前比较成熟的一些智能算法,比如神经网络,自动对图像帧进行识别是否包括敏感类图像。In addition, some mature intelligent algorithms, such as neural networks, can be used to automatically identify whether image frames include sensitive images.

需要说明的是,敏感特征值可以是敏感类图像中的敏感标志物图的特征值,敏感标志物可以是敏感文字(比如书籍、文章、标语等)、敏感场景(建筑物)、敏感人物像等。若使用智能算法时,在训练神经网络时,就需要使用包含敏感标志物的图像进行训练,使得神经网络能够识别出敏感类图像。It should be noted that the sensitive feature value can be the feature value of the sensitive marker map in the sensitive image, and the sensitive marker can be sensitive text (such as books, articles, slogans, etc.), sensitive scenes (buildings), sensitive person images. Wait. When using an intelligent algorithm, when training a neural network, it is necessary to use images containing sensitive markers for training, so that the neural network can identify sensitive images.

当然,敏感标志物也可以是其他不符合公司要求,或与公司工作不相关的内容的显示画面。比如,对于国防军工单位,敏感标志物可以包括反动文章,反动图片,反动领袖人物像等,对于教育机构,敏感标志物可以是暴力色情图像视频等。Of course, the sensitive marker can also be a display screen of other content that does not meet the company's requirements or is not related to the company's work. For example, for national defense and military units, sensitive markers may include reactionary articles, reactionary pictures, and images of reactionary leaders. For educational institutions, sensitive markers may be violent pornographic images and videos.

在一个实施例中,渲染指令是服务器接收终端设备发送的操作消息,根据所述账号信息分配与所述终端设备对应的虚拟机VM,通过所述虚拟机依次解析所述操作消息,根据解析结果生成的,所述操作消息包括账号信息。In one embodiment, the rendering instruction is that the server receives an operation message sent by a terminal device, allocates a virtual machine VM corresponding to the terminal device according to the account information, parses the operation message sequentially through the virtual machine, and parses the operation message according to the parsing result. generated, the operation message includes account information.

比如,终端设备将操作信息打包压缩通过网络发送到VM,VM对接收到的操作信息,按照注入顺序逐一解析出来,得到多个操作指令;VM执行多个操作指令,比如,打开一个word文档,编辑,保存,播放视频等操作;根据执行结果,生成渲染指令,渲染指令用于生成执行操作信息对应的操作指令后显示出的画面,比如,操作指令是打开一个文档,那么,渲染指令就是用于生成该文档打开后显示的页面。For example, the terminal device packages and compresses the operation information and sends it to the VM through the network. The VM parses the received operation information one by one according to the injection sequence, and obtains multiple operation instructions; the VM executes multiple operation instructions, such as opening a word document, Edit, save, play video and other operations; generate rendering instructions according to the execution results. The rendering instructions are used to generate the screen displayed after executing the operation instructions corresponding to the operation information. For example, the operation instruction is to open a document, then the rendering instruction is to use Generates the page displayed after the document is opened.

在一个实施例中,该方法还包括:In one embodiment, the method further includes:

根据所述账号信息,统计每个账号信息对应的敏感类图像。According to the account information, the sensitive images corresponding to each account information are counted.

图4是本公开实施例提供的一种图像处理装置结构图,如图4所示的图像处理装置40包括获取模块401、渲染模块402、识别模块403、处理模块404和发送模块405,FIG. 4 is a structural diagram of an image processing apparatus provided by an embodiment of the present disclosure. The image processing apparatus 40 shown in FIG.

获取模块401,用于获取渲染指令,所述渲染指令是根据终端设备的操作消息生成的;an obtaining module 401, configured to obtain a rendering instruction, the rendering instruction is generated according to an operation message of the terminal device;

渲染模块402,用于根据所述渲染指令进行渲染,生成待处理图像帧;A rendering module 402, configured to perform rendering according to the rendering instruction, and generate an image frame to be processed;

识别模块403,用于识别所述待处理图像帧的类型,待处理图像帧的类型包括变化图像帧和不变图像帧;The identification module 403 is used to identify the type of the image frame to be processed, and the type of the image frame to be processed includes a changing image frame and an invariant image frame;

处理模块404,用于基于所述待处理图像帧的类型对所述待处理图像帧进行过滤处理,得到目标图像帧;A processing module 404, configured to perform filtering processing on the to-be-processed image frame based on the type of the to-be-processed image frame to obtain a target image frame;

发送模块405,用于将所述目标图像帧发送给所述终端设备。The sending module 405 is configured to send the target image frame to the terminal device.

图5是本公开实施例提供的一种图像处理装置处理模块结构图,如图5所示的处理模块404包括:FIG. 5 is a structural diagram of a processing module of an image processing apparatus provided by an embodiment of the present disclosure. The processing module 404 shown in FIG. 5 includes:

判断子模块4041,用于如果是变化图像帧,则将变化图像帧作为样本图像帧,根据预设图像识别算法判断所述样本图像帧是否为敏感类图像;Judging sub-module 4041, for if it is a changing image frame, then taking the changing image frame as a sample image frame, and judging whether the sample image frame is a sensitive image according to a preset image recognition algorithm;

第一替换子模块4042,用于将所述变化图像帧替换为预设图像帧。The first replacement sub-module 4042 is configured to replace the changed image frame with a preset image frame.

图6是本公开实施例提供的一种图像处理装置处理模块结构图,如图6所示的处理模块404包括:FIG. 6 is a structural diagram of a processing module of an image processing apparatus provided by an embodiment of the present disclosure. The processing module 404 shown in FIG. 6 includes:

记录子模块4043,用于如果是不变图像帧,则记录所述不变图像帧,并确定下一图像帧的类型,直到检测到变化图像帧;The recording submodule 4043 is used to record the unchanged image frame if it is an unchanged image frame, and determine the type of the next image frame until the changed image frame is detected;

选取子模块4044,用于在每M帧不变图像帧中,选取样本图像帧,根据预设图像识别算法判断所述样本图像帧是否为敏感类图像;The selection submodule 4044 is used to select a sample image frame in every M frames of invariant image frames, and judge whether the sample image frame is a sensitive image according to a preset image recognition algorithm;

第二替换子模块4045,用于如果是敏感类图像,将所述M帧不变图像帧替换为预设图像帧。The second replacement sub-module 4045 is configured to replace the M frames of unchanged image frames with preset image frames if it is a sensitive image.

图7是本公开实施例提供的一种图像处理装置识别模块结构图,如图7所示的识别模块403包括::FIG. 7 is a structural diagram of a recognition module of an image processing apparatus provided by an embodiment of the present disclosure. The recognition module 403 shown in FIG. 7 includes:

判断子模块4031,用于判断所述待处理图像帧中的宏块与前一图像帧中对应位置的宏块的像素点是否完全相同;Judging submodule 4031, for judging whether the macroblock in the image frame to be processed is exactly the same as the pixel point of the macroblock at the corresponding position in the previous image frame;

标识子模块4032,用于如果完全相同,则识别所述待处理图像帧为不变宏块,若不完全相同,则识别所述待处理图像帧为变化宏块;The identification submodule 4032 is used to identify the to-be-processed image frame as an invariant macroblock if it is identical, and to identify the to-be-processed image frame as a changed macroblock if it is not identical;

统计子模块4033,用于统计所述不变宏块的数量,如果所述不变宏块的数量不大于预设阈值,则确定所述待处理图像帧为不变图像帧;如果所述不变宏块的数量小于预设阈值,则确定所述待处理图像帧为变化图像帧。The statistics sub-module 4033 is used to count the number of the unchanged macroblocks, and if the number of the unchanged macroblocks is not greater than a preset threshold, determine that the image frame to be processed is an unchanged image frame; If the number of variable macroblocks is less than the preset threshold, the image frame to be processed is determined to be a changed image frame.

图8是本公开实施例提供的一种图像处理装置结构图,如图8所示的图像处理装置40还包括:FIG. 8 is a structural diagram of an image processing apparatus provided by an embodiment of the present disclosure. The image processing apparatus 40 shown in FIG. 8 further includes:

确定模块406,用于识别所述待处理图像帧的类型之前,确定所述待处理图像帧中包含视频宏块。The determining module 406 is configured to determine that the to-be-processed image frame contains video macroblocks before identifying the type of the to-be-processed image frame.

图9是本公开实施例提供的一种图像处理装置判断子模块结构图,如图9所示的判断子模块4031包括::FIG. 9 is a structural diagram of a judgment sub-module of an image processing apparatus provided by an embodiment of the present disclosure. The judgment sub-module 4031 shown in FIG. 9 includes:

提取子单元11,用于提取样本图像帧中的关键字/特征值,将提取的关键字/特征值与预设的敏感关键字/敏感特征值进行比对;The extraction subunit 11 is used to extract keywords/feature values in the sample image frame, and compare the extracted keywords/feature values with preset sensitive keywords/sensitive feature values;

确定子单元12,用于如果样本图像帧的关键字/特征值中包含预设的敏感关键字/敏感特征值,则确定所述样本图像帧为敏感类图像。The determination subunit 12 is configured to determine that the sample image frame is a sensitive image if the keyword/feature value of the sample image frame contains a preset sensitive keyword/sensitive feature value.

在上述实施例中,渲染指令是服务器接收终端设备发送的操作消息,根据所述账号信息分配与所述终端设备对应的虚拟机,通过所述虚拟机依次解析所述操作消息,根据解析结果生成的,所述操作消息包括账号信息。In the above-mentioned embodiment, the rendering instruction is that the server receives an operation message sent by a terminal device, allocates a virtual machine corresponding to the terminal device according to the account information, parses the operation message sequentially through the virtual machine, and generates an operation message according to the parsing result. Yes, the operation message includes account information.

图10是本公开实施例提供的一种图像处理装置结构图,如图10所示的图像处理装置40包括获取模块401、渲染模块402、识别模块403、处理模块404、发送模块405和统计模块407,FIG. 10 is a structural diagram of an image processing apparatus provided by an embodiment of the present disclosure. The image processing apparatus 40 shown in FIG. 10 includes an acquisition module 401, a rendering module 402, an identification module 403, a processing module 404, a transmission module 405, and a statistics module 407,

统计模块407,用于根据所述账号信息,统计每个账号信息对应的敏感类图像。The statistics module 407 is configured to count the sensitive images corresponding to each account information according to the account information.

图11是本本公开实施例提供的一种图像处理系统应用结构图,服务器是云端服务器,包括多个虚拟机VM模块,每个VM模块对应一个账号;图像处理装置可以包括GPU POOL模块(相当于上述实施例中的获取模块和渲染模块)、管理S2模块(相当于上述实施例中的识别模块和发送模块)和监控模块(相当于上述实施例中的处理模块),监控模块也可以独立存在,终端设备为一R端。具体的执行流程如图12所示:FIG. 11 is an application structure diagram of an image processing system provided by an embodiment of the present disclosure. The server is a cloud server, including multiple virtual machine VM modules, and each VM module corresponds to an account; the image processing apparatus may include a GPU POOL module (equivalent to a GPU POOL module). The acquisition module and the rendering module in the above-mentioned embodiment), the management S2 module (equivalent to the identification module and the sending module in the above-mentioned embodiment) and the monitoring module (equivalent to the processing module in the above-mentioned embodiment), the monitoring module can also exist independently , the terminal device is an R terminal. The specific execution process is shown in Figure 12:

步骤601、用户在R端进行操作,R端向云端服务器上对应的VM上传操作信息。Step 601 , the user performs an operation on the R terminal, and the R terminal uploads the operation information to the corresponding VM on the cloud server.

其中,操作信息包括键盘的按键事件、鼠标或触控板的移动和点击事件等。The operation information includes key events of the keyboard, movement and click events of the mouse or the touchpad, and the like.

步骤602、VM对操作信息进行解析,根据解析结果生成渲染指令。Step 602: The VM parses the operation information, and generates a rendering instruction according to the parsing result.

步骤603、VM将渲染指令发送到GPU POOL系统中的GPU POOL模块,由GPU POOL模块对该渲染指令进行渲染,得到图像帧;Step 603, the VM sends the rendering instruction to the GPU POOL module in the GPU POOL system, and the rendering instruction is rendered by the GPU POOL module to obtain an image frame;

GPU POOL模块将图像帧发送到S2模块。The GPU POOL module sends the image frames to the S2 module.

步骤604、S2模块对各图像帧,以宏块为单位进行分类。In step 604, the S2 module classifies each image frame in units of macroblocks.

在本步骤中,S2模块将渲染后图像帧拆分为多个宏块,判断各宏块的类型,宏块类型包括unchange宏块和change宏块。In this step, the S2 module divides the rendered image frame into a plurality of macroblocks, and determines the type of each macroblock. The macroblock types include unchange macroblocks and change macroblocks.

具体的判断过程个上述实施例一致,不再赘述。The specific judging process is the same as that in the above-mentioned embodiment, and will not be repeated here.

步骤605、S2将全部的图像帧发送到监控模块,由监控模块对图像帧进行根据分类进行过滤处理。Step 605, S2 sends all the image frames to the monitoring module, and the monitoring module performs filtering processing on the image frames according to the classification.

这里的图像帧包括标识的图像帧和未标识的图像帧。The image frames here include identified image frames and unidentified image frames.

此外,还需要按照VM账号,对各账号拦截的图像帧的帧数和时间进行记录,以便网管统计分析,使得网管能够快速找出所查看视频或图像中包括敏感类图像的数量较多的员工。In addition, it is also necessary to record the frame number and time of the image frames intercepted by each account according to the VM account, so as to facilitate statistical analysis of the network management, so that the network management can quickly find out the video or image viewed by the employee with a large number of sensitive images. .

步骤606、S2模块对处理后的图像帧进行编码,以及将编码数据发送给R端。Step 606, the S2 module encodes the processed image frame, and sends the encoded data to the R terminal.

具体的,由于S2接收到的图像帧中的敏感类图像已被替换掉,且被替换后的图片均标记为Z,因此,S模块可以对标记为Z的图像帧重新进行宏块划分,并确定新划分的宏块的宏块类型,将新确定宏块类型的图像帧替换原先的图像帧。Specifically, since the sensitive images in the image frames received by S2 have been replaced, and the replaced images are marked as Z, the S module can re-define the image frames marked as Z into macroblocks, and The macroblock type of the newly divided macroblock is determined, and the image frame of the newly determined macroblock type is replaced with the original image frame.

S2模块根据各宏块的宏块类型,对图像帧进行编码,并将编码后的编码数据发送给R端。The S2 module encodes the image frame according to the macroblock type of each macroblock, and sends the encoded encoded data to the R terminal.

步骤607、R端对接收到的编码数据进行解码并显示。Step 607: The R terminal decodes and displays the received encoded data.

在本公开中,可以对渲染出的图像帧进行识别和分类,对于不符合企业要求的敏感类图像,进行替换,再将替换后的图像帧发送给终端设备,使得员工在终端设备无法观看到敏感类图像,保证办公场所的良好氛围,提高员工工作效率。In the present disclosure, the rendered image frames can be identified and classified, and the sensitive images that do not meet the requirements of the enterprise can be replaced, and then the replaced image frames can be sent to the terminal device, so that the employees cannot view it on the terminal device. Sensitive images can ensure a good atmosphere in the office and improve the work efficiency of employees.

基于上述图1对应的实施例中所描述的图像处理方法,本公开实施例还提供一种计算机可读存储介质,例如,非临时性计算机可读存储介质可以是只读存储器(英文:ReadOnly Memory,ROM)、随机存取存储器(英文:Random Access Memory,RAM)、CD-ROM、磁带、软盘和光数据存储装置等。该存储介质上存储有计算机指令,用于执行上述图1对应的实施例中所描述的图像处理方法,此处不再赘述。Based on the image processing method described in the above-mentioned embodiment corresponding to FIG. 1 , an embodiment of the present disclosure further provides a computer-readable storage medium. For example, the non-transitory computer-readable storage medium may be a read-only memory (English: ReadOnly Memory). , ROM), random access memory (English: Random Access Memory, RAM), CD-ROM, magnetic tape, floppy disk and optical data storage devices, etc. The storage medium stores computer instructions for executing the image processing method described in the embodiment corresponding to FIG. 1 , which will not be repeated here.

本领域技术人员在考虑说明书及实践这里公开的公开后,将容易想到本公开的其它实施方案。本申请旨在涵盖本公开的任何变型、用途或者适应性变化,这些变型、用途或者适应性变化遵循本公开的一般性原理并包括本公开未公开的本技术领域中的公知常识或惯用技术手段。说明书和实施例仅被视为示例性的,本公开的真正范围和精神由下面的权利要求指出。Other embodiments of the present disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the disclosure disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or techniques in the technical field not disclosed by the present disclosure . The specification and examples are to be regarded as exemplary only, with the true scope and spirit of the disclosure being indicated by the following claims.

Claims (10)

1.一种图像处理方法,应用于图像处理装置,其特征在于,所述方法包括:1. An image processing method, applied to an image processing device, wherein the method comprises: 获取渲染指令,所述渲染指令是根据终端设备的操作消息生成的;Obtaining a rendering instruction, the rendering instruction is generated according to the operation message of the terminal device; 根据所述渲染指令进行渲染,生成待处理图像帧;Rendering according to the rendering instruction to generate a to-be-processed image frame; 识别所述待处理图像帧的类型,待处理图像帧的类型包括变化图像帧和不变图像帧;Identifying the type of the image frame to be processed, the type of the image frame to be processed includes a changing image frame and an invariant image frame; 基于所述待处理图像帧的类型对所述待处理图像帧进行过滤处理,得到目标图像帧;Perform filtering processing on the to-be-processed image frame based on the type of the to-be-processed image frame to obtain a target image frame; 将所述目标图像帧发送给所述终端设备。Send the target image frame to the terminal device. 2.根据权利要求1所述的方法,其特征在于,基于所述待处理图像帧的类型对所述待处理图像帧进行过滤处理,得到目标图像帧包括:2. The method according to claim 1, wherein filtering the to-be-processed image frame based on the type of the to-be-processed image frame to obtain the target image frame comprises: 如果是变化图像帧,则将变化图像帧作为样本图像帧,根据预设图像识别算法判断所述样本图像帧是否为敏感类图像;If it is a changing image frame, the changing image frame is used as a sample image frame, and whether the sample image frame is a sensitive image is judged according to a preset image recognition algorithm; 如果是敏感类图像帧,将所述变化图像帧替换为预设图像帧。If it is a sensitive image frame, replace the changed image frame with a preset image frame. 3.根据权利要求1所述的方法,其特征在于,基于所述待处理图像帧的类型对所述待处理图像帧进行过滤处理,得到目标图像帧包括:3. The method according to claim 1, wherein filtering the to-be-processed image frame based on the type of the to-be-processed image frame to obtain the target image frame comprises: 如果是不变图像帧,则记录所述不变图像帧,并确定下一图像帧的类型,直到检测到变化图像帧;If it is a constant image frame, record the constant image frame, and determine the type of the next image frame, until a changed image frame is detected; 在每M帧不变图像帧中,选取样本图像帧,根据预设图像识别算法判断所述样本图像帧是否为敏感类图像;In every M frames of invariant image frames, select a sample image frame, and determine whether the sample image frame is a sensitive image according to a preset image recognition algorithm; 如果是敏感类图像,将所述M帧不变图像帧替换为预设图像帧。If it is a sensitive image, replace the M frames of invariant image frames with preset image frames. 4.根据权利要求2或3所述的方法,其特征在于,所述待处理图像帧包括至少一个宏块,识别所述待处理图像帧的类型包括:4. The method according to claim 2 or 3, wherein the image frame to be processed comprises at least one macroblock, and identifying the type of the image frame to be processed comprises: 判断所述待处理图像帧中的宏块与前一图像帧中对应位置的宏块的像素点是否完全相同;Judging whether the macroblock in the to-be-processed image frame is exactly the same as the pixel point of the macroblock at the corresponding position in the previous image frame; 如果完全相同,则识别所述待处理图像帧为不变宏块,若不完全相同,则识别所述待处理图像帧为变化宏块;If they are identical, then identify the to-be-processed image frame as a constant macroblock, if not, identify the to-be-processed image frame as a changed macroblock; 统计所述不变宏块的数量,如果所述不变宏块的数量不大于预设阈值,则确定所述待处理图像帧为不变图像帧;如果所述不变宏块的数量小于预设阈值,则确定所述待处理图像帧为变化图像帧。Count the number of unchanged macroblocks, and if the number of unchanged macroblocks is not greater than a preset threshold, then determine that the image frame to be processed is an unchanged image frame; if the number of unchanged macroblocks is less than the predetermined threshold If the threshold is set, it is determined that the image frame to be processed is a changed image frame. 5.根据权利要求4所述的方法,其特征在于,识别所述待处理图像帧的类型之前,所述方法还包括:5. The method according to claim 4, wherein before identifying the type of the image frame to be processed, the method further comprises: 确定所述待处理图像帧中包含视频宏块。It is determined that the to-be-processed image frame contains video macroblocks. 6.根据权利要求5所述的方法,其特征在于,根据预设图像识别算法判断所述样本图像帧是否为敏感类图像包括:6. method according to claim 5, is characterized in that, according to preset image recognition algorithm, judge whether described sample image frame is sensitive class image comprises: 提取所述样本图像帧中的关键字/特征值,将提取的关键字/特征值与预设的敏感关键字/敏感特征值进行比对;Extracting keywords/feature values in the sample image frame, and comparing the extracted keywords/feature values with preset sensitive keywords/sensitive feature values; 如果样本图像帧的关键字/特征值中包含预设的敏感关键字/敏感特征值,则确定所述样本图像帧为敏感类图像。If the keyword/feature value of the sample image frame contains a preset sensitive keyword/sensitive feature value, the sample image frame is determined to be a sensitive image. 7.根据权利要求6所述的方法,其特征在于,所述渲染指令是服务器接收终端设备发送的至少一个操作消息,根据所述账号信息分配与所述终端设备对应的虚拟机,通过所述虚拟机依次解析所述至少一个操作消息,根据解析结果生成的,所述至少一个操作消息包括账号信息。7. The method according to claim 6, wherein the rendering instruction is that a server receives at least one operation message sent by a terminal device, allocates a virtual machine corresponding to the terminal device according to the account information, and uses the The virtual machine sequentially parses the at least one operation message, and is generated according to the parsing result, and the at least one operation message includes account information. 8.根据权利要求7所述的方法,其特征在于,所述方法还包括:8. The method according to claim 7, wherein the method further comprises: 根据所述账号信息,统计每个账号信息对应的敏感类图像。According to the account information, the sensitive images corresponding to each account information are counted. 9.一种图像处理装置,其特征在于,所述装置包括:9. An image processing device, wherein the device comprises: 获取模块,用于获取渲染指令,所述渲染指令是根据终端设备的操作消息生成的;an acquisition module, used for acquiring rendering instructions, the rendering instructions are generated according to the operation message of the terminal device; 渲染模块,用于根据所述渲染指令进行渲染,生成待处理图像帧;a rendering module, configured to perform rendering according to the rendering instruction, and generate a to-be-processed image frame; 处理模块,用于基于所述待处理图像帧的类型对所述待处理图像帧进行过滤处理,得到目标图像帧;a processing module, configured to filter the to-be-processed image frame based on the type of the to-be-processed image frame to obtain a target image frame; 发送模块,用于将所述目标图像帧发送给所述终端设备。A sending module, configured to send the target image frame to the terminal device. 10.根据权利要求9所述的装置,其特征在于,所述处理模块包括:10. The apparatus according to claim 9, wherein the processing module comprises: 识别子模块,用于识别所述待处理图像帧的类型,待处理图像帧的类型包括变化图像帧和不变图像帧;an identification sub-module for identifying the type of the image frame to be processed, the type of the image frame to be processed includes a changing image frame and an invariant image frame; 判断子模块,用于如果是变化图像帧,则将变化图像帧作为样本图像帧,根据预设图像识别算法判断所述样本图像帧是否为敏感类图像;a judging sub-module, configured to use the changed image frame as a sample image frame if it is a changing image frame, and judge whether the sample image frame is a sensitive image according to a preset image recognition algorithm; 第一替换子模块,用于将所述变化图像帧替换为预设图像帧。The first replacement submodule is configured to replace the changed image frame with a preset image frame.
CN202010168017.3A 2020-03-11 2020-03-11 Image processing method and device Active CN111476273B (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN202410557422.2A CN118537603A (en) 2020-03-11 2020-03-11 Image processing method and device
CN202010168017.3A CN111476273B (en) 2020-03-11 2020-03-11 Image processing method and device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010168017.3A CN111476273B (en) 2020-03-11 2020-03-11 Image processing method and device

Related Child Applications (1)

Application Number Title Priority Date Filing Date
CN202410557422.2A Division CN118537603A (en) 2020-03-11 2020-03-11 Image processing method and device

Publications (2)

Publication Number Publication Date
CN111476273A true CN111476273A (en) 2020-07-31
CN111476273B CN111476273B (en) 2024-06-07

Family

ID=71747374

Family Applications (2)

Application Number Title Priority Date Filing Date
CN202010168017.3A Active CN111476273B (en) 2020-03-11 2020-03-11 Image processing method and device
CN202410557422.2A Pending CN118537603A (en) 2020-03-11 2020-03-11 Image processing method and device

Family Applications After (1)

Application Number Title Priority Date Filing Date
CN202410557422.2A Pending CN118537603A (en) 2020-03-11 2020-03-11 Image processing method and device

Country Status (1)

Country Link
CN (2) CN111476273B (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114359451A (en) * 2020-09-28 2022-04-15 逐点半导体(上海)有限公司 Method and system for accelerating image rendering using motion compensation
WO2023093792A1 (en) * 2021-11-26 2023-06-01 华为技术有限公司 Image frame rendering method and related apparatus

Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5754700A (en) * 1995-06-09 1998-05-19 Intel Corporation Method and apparatus for improving the quality of images for non-real time sensitive applications
US7274368B1 (en) * 2000-07-31 2007-09-25 Silicon Graphics, Inc. System method and computer program product for remote graphics processing
CN106470345A (en) * 2015-08-21 2017-03-01 阿里巴巴集团控股有限公司 Video-encryption transmission method and decryption method, apparatus and system
WO2017056230A1 (en) * 2015-09-30 2017-04-06 楽天株式会社 Information processing device, information processing method, and program for information processing device
US20180176597A1 (en) * 2016-12-20 2018-06-21 Axis Ab Encoding a privacy masked image
CN108965982A (en) * 2018-08-28 2018-12-07 百度在线网络技术(北京)有限公司 Video recording method, device, electronic equipment and readable storage medium storing program for executing
CN109040824A (en) * 2018-08-28 2018-12-18 百度在线网络技术(北京)有限公司 Method for processing video frequency, device, electronic equipment and readable storage medium storing program for executing
US20190156123A1 (en) * 2017-11-23 2019-05-23 Institute For Information Industry Method, electronic device and non-transitory computer readable storage medium for image annotation
CN110298862A (en) * 2018-03-21 2019-10-01 广东欧珀移动通信有限公司 Video processing method, video processing device, computer-readable storage medium and computer equipment
CN110312133A (en) * 2019-06-27 2019-10-08 西安万像电子科技有限公司 Image processing method and device
CN110362375A (en) * 2019-07-11 2019-10-22 广州虎牙科技有限公司 Display methods, device, equipment and the storage medium of desktop data

Patent Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5754700A (en) * 1995-06-09 1998-05-19 Intel Corporation Method and apparatus for improving the quality of images for non-real time sensitive applications
US7274368B1 (en) * 2000-07-31 2007-09-25 Silicon Graphics, Inc. System method and computer program product for remote graphics processing
CN106470345A (en) * 2015-08-21 2017-03-01 阿里巴巴集团控股有限公司 Video-encryption transmission method and decryption method, apparatus and system
WO2017056230A1 (en) * 2015-09-30 2017-04-06 楽天株式会社 Information processing device, information processing method, and program for information processing device
US20180176597A1 (en) * 2016-12-20 2018-06-21 Axis Ab Encoding a privacy masked image
US20190156123A1 (en) * 2017-11-23 2019-05-23 Institute For Information Industry Method, electronic device and non-transitory computer readable storage medium for image annotation
CN110298862A (en) * 2018-03-21 2019-10-01 广东欧珀移动通信有限公司 Video processing method, video processing device, computer-readable storage medium and computer equipment
CN108965982A (en) * 2018-08-28 2018-12-07 百度在线网络技术(北京)有限公司 Video recording method, device, electronic equipment and readable storage medium storing program for executing
CN109040824A (en) * 2018-08-28 2018-12-18 百度在线网络技术(北京)有限公司 Method for processing video frequency, device, electronic equipment and readable storage medium storing program for executing
CN110312133A (en) * 2019-06-27 2019-10-08 西安万像电子科技有限公司 Image processing method and device
CN110362375A (en) * 2019-07-11 2019-10-22 广州虎牙科技有限公司 Display methods, device, equipment and the storage medium of desktop data

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
MONA OMIDYEGANEH: "Video Keyframe Analysis Using a Segment-Based Statistical Metric in a Visually Sensitive Parametric Space", 《IEEE TRANSACTIONS ON IMAGE PROCESSING》, vol. 20, no. 10, pages 2730 - 2737, XP011387020, DOI: 10.1109/TIP.2011.2143421 *
ZIHAN ZHOU ET.AL: "Detecting Dominant Vanishing Points in Natural Scenes with Application to Composition-Sensitive Image Retrieval", 《 IEEE TRANSACTIONS ON MULTIMEDIA》, vol. 9, no. 12, 12 May 2017 (2017-05-12), pages 2651 - 2665 *
林海涛: "基于监控视频的信息隐藏与篡改检测技术研究", 《中国优秀硕士学位论文全文数据库 信息科技辑》, vol. 2020, no. 02, pages 136 - 1688 *
梁鹏 等: "基于人体肤色识别征的敏感视频分类方法", 《计算机辅助设计与图形学学报》, vol. 12, no. 13, 31 May 2016 (2016-05-31), pages 181 - 200 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114359451A (en) * 2020-09-28 2022-04-15 逐点半导体(上海)有限公司 Method and system for accelerating image rendering using motion compensation
WO2023093792A1 (en) * 2021-11-26 2023-06-01 华为技术有限公司 Image frame rendering method and related apparatus

Also Published As

Publication number Publication date
CN111476273B (en) 2024-06-07
CN118537603A (en) 2024-08-23

Similar Documents

Publication Publication Date Title
CN112600834B (en) Content security identification method and device, storage medium and electronic equipment
CN108683877A (en) Spark-based distributed massive video analysis system
JP2020515983A (en) Target person search method and device, device, program product and medium
CN113962199B (en) Text recognition method, text recognition device, text recognition equipment, storage medium and program product
CN112633313B (en) Bad information identification method of network terminal and local area network terminal equipment
US12217522B2 (en) Image classification using color profiles
CN112348089A (en) Working state identification method, server, storage medium and device
CN111222571A (en) Image special effect processing method and device, electronic equipment and storage medium
CN110647895B (en) Phishing page identification method based on login box image and related equipment
CN103765421A (en) Content control method, content control apparatus, and program
CN111049786A (en) A network attack detection method, device, equipment and storage medium
CN111476273B (en) Image processing method and device
CN111368128A (en) Target picture identification method and device and computer readable storage medium
CN111445399A (en) Image processing method and system
CN115719423A (en) Similarity-based malicious information detection method and device and processor
CN107688744B (en) Malicious file classification method and device based on image feature matching
CN109033264B (en) Video analysis method and device, electronic equipment and storage medium
CN108334602B (en) Data annotation method and device, electronic equipment and computer storage medium
EP3709666A1 (en) Method for fitting target object in video frame, system, and device
CN118864983A (en) Image classification and recognition method and device
Bailer Face swapping for solving collateral privacy issues in multimedia analytics
CN114781557A (en) Image information acquisition method and device, and computer-readable storage medium
CN113657230B (en) Method for training news video recognition model, method for detecting video and device thereof
RU2671304C1 (en) Method and system for constructing digital print of video content
CN111339367A (en) Video processing method and device, electronic equipment and computer readable storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant