CN111327939A - Distributed teaching video processing system - Google Patents

Abstract

The invention discloses a distributed teaching video processing system, comprising a teaching video file transmission module, a teaching video GPU accelerated processing module and a teaching video automatic human eye mosaicing module, each module can be implemented by C++ language corresponding software module, thereby Implement a teaching video processing system that can run on a server. The teaching video file transfer module is used to download the original video file from the video storage server and upload the processed video file. The GPU-accelerated processing module for teaching videos can perform GPU-accelerated processing for video cropping, video transcoding, and video resolution adjustment. The teaching video automatic human eye mosaic module can realize automatic mosaic of the human eyes in the video. The present invention transfers the local video processing to the server for processing, which speeds up the video processing speed, and provides the video automatic human eye mosaic function that the video processing software does not have, which simplifies and facilitates the video processing flow.

Description

A Distributed Teaching Video Processing System

technical field

The invention relates to the technical field of video processing, in particular to a distributed teaching video processing system.

Background technique

At present, in the learning management system, we usually need to shoot some teaching videos of teachers' lectures and then make them into online courses for students to learn. However, since the original teaching videos shot are usually very large in size and resolution, in order to save bandwidth resources in the learning management system, we usually need to perform some processing on these videos, such as cropping the original video, or adjusting its resolution to reduce Video size, or a series of operations such as transcoding the video. In addition, for some teaching videos of TCM diagnosis, we also need to perform some high-level processing on the videos, such as mosaicking the human eyes to protect the privacy of patients.

Faced with these video processing tasks, we usually use some third-party video processing software, such as Adobe Premiere, for processing. However, for teachers, these video processing tasks are generally very cumbersome and repetitive, and generally these video processing software are usually oriented to professionals, and their functions are relatively basic, lacking high-level functions, and are not friendly to ordinary users. , when users use their personal computers to process these tasks, they rely heavily on single-machine CPU resources, and the processing efficiency is very low.

SUMMARY OF THE INVENTION

The purpose of the present invention is to overcome the shortcomings and deficiencies of the prior art, and proposes a distributed teaching video processing system, which fully exploits the parallel processing capability of the GPU for teaching video processing, and solves the problem that the user's local teaching video processing speed is slow. The user's local processing operation is cumbersome, and it needs to rely on third-party video processing software and provide some video processing functions that the third-party video processing software does not have. The system acts as a teaching video processing engine. On the one hand, it can perform GPU acceleration on traditional teaching video processing, such as video transcoding, resolution, and cropping. On the other hand, this engine provides some high-level teaching video processing functions. For example, the human eye in the teaching video automatically plays mosaic and so on.

To achieve the above purpose, the technical solution provided by the present invention is as follows: a distributed teaching video processing system, comprising:

The teaching video file transmission module is used to download the original video file from the video storage server and upload the video file processed by the video processing system;

The teaching video GPU accelerated processing module is used to adjust the resolution of the video according to the video name and video resolution input by the user, and transcode the video according to the video name and video format input by the user. Time and end time to crop the video;

The teaching video automatic eye mosaic module is used to decompose the teaching video into image frames, detect the human face in the image, detect the human eye feature points in the face, and perform mosaic on the human eye feature point area. image frames and audio into a video.

Further, the teaching video file transmission module includes a user login module, a cookie processing module, a video file download module, and a video file upload module, wherein:

The user login module logs in to the video storage server according to the user name and password input by the user;

The cookie processing module is responsible for saving the cookie returned by the video storage server to the local and loading the cookie from the local;

The video file downloading module is responsible for downloading video files from the video storage server;

The video file uploading module is responsible for uploading the processed video to the video storage server.

Further, the teaching video GPU accelerated processing module includes a video resolution adjustment module, a video transcoding module, and a video cropping module, wherein:

The video resolution adjustment module adjusts the video resolution: by parsing the request parameters in the HTTP POST request, the target resolution of the video is obtained, and then the video processing tool FFMPEG is used to adjust the resolution of the specified video file. In the process, fully exploit the parallel computing capability of GPU, use NVENC and NVDEC provided by NVIDIA to accelerate video encoding and decoding, realize GPU-accelerated video processing, and improve processing efficiency;

The video transcoding module converts the video format: by parsing the request parameters in the HTTP POST request, the target video format is obtained, and then the video processing tool FFMPEG is used to convert the specified video file to the video format. , fully exploit the parallel computing capability of GPU, and use NVENC and NVDEC provided by NVIDIA to accelerate video encoding and decoding to realize GPU-accelerated video processing and improve processing efficiency;

The video trimming module trims the video: by parsing the request parameters in the HTTP POST request, the start time and end time of the video to be trimmed are obtained, and then the video processing tool FFMPEG is used to trim the specified video file.

Further, the automatic human eye mosaicking module of the teaching video includes a face detection module in an image, a human eye feature point detection module in a human face, a human eye feature point tracking module between consecutive frames, a human eye feature point area mosaic module and an image. Frame and audio composite video module, where:

The face detection module in the image automatically detects the coordinate points of the face area in an image: pre-train a neural network model based on deep learning to detect the face, save the weight value of the neural network locally, and perform face detection. During detection, the weight value of the neural network is loaded locally. When an image frame is input into the model, the model will automatically detect the coordinate area of the face in the image;

The eye feature point detection module in the human face automatically detects the coordinate points of the human eye feature points in the human face: pre-training a deep learning-based neural network model for detecting human eye feature points from the human face, and saving the weight of the neural network value to the local. When detecting human eye feature points, the weight value of the neural network is loaded from the local area. When inputting the face area in an image frame to the model, the model will automatically detect the human eye feature points of the human face. ;

The human eye feature point tracking module between consecutive frames tracks the human eye feature points of two consecutive frames of images: pre-training a deep learning-based neural network model for human eye feature point tracking, saving the weight value of the neural network locally, When tracking human eye feature points, the weight value of the neural network is loaded locally. When inputting the face area of the next frame into the model, the tracker first performs human eye feature point tracking. If the tracked human eye feature point results If it meets the expectations, continue to process the next frame of image, otherwise it will be handed over to the human eye feature point detection module for re-detection of human eye feature points;

The human eye feature point region mosaicking module performs mosaicking on the human eye feature point region: after detecting the human eye feature point in the image, the corresponding mosaic region is calculated according to the human eye feature point, and then the corresponding human eye mosaic is generated;

The image frame and the audio synthesizing video module synthesize the image frame and audio after the coding into a video file: after obtaining the image frame after the coding, in conjunction with the audio that was separated before in the video, use the video processing tool FFMPEG to combine the image frame and the audio. The audio is synthesized into the final encoded video.

Compared with the prior art, the present invention has the following advantages and beneficial effects:

1. Transfer the teaching video processing tasks from the local to the server for processing. The server fully exploits the parallel processing capability of the GPU, and combines the video processing tool FFMPEG to accelerate the video transcoding, video resolution adjustment, and video cropping, which greatly improves the performance. The video processing speed simplifies the video processing process, and users do not need to learn and use third-party video processing software.

2. Based on the latest deep learning technology, neural network technology and powerful GPU computing power, the system provides ordinary video processing by deploying a face detection model, a human eye feature point detection model and a human eye feature point tracking model in the system. The software does not have the high-level human eye automatic mosaic function.

3. The video processing system adopts distributed deployment, and the teaching video storage server and the teaching video processing server are separated. The video processing system consists of multiple video processing servers, and the web server Nginx distributes and processes requests to achieve load balancing.

Description of drawings

FIG. 1 is a schematic diagram of the overall architecture of the system of the present invention.

Fig. 2 is the whole flow chart of the video processing process.

Detailed ways

The present invention will be further described below in conjunction with specific embodiments.

The distributed teaching video processing system provided in this embodiment is a teaching video processing system developed by using Visual Studio Code software and C++ and Python languages and running on a server. As shown in Figure 1, the system forwards requests and messages through the web server Nginx. Load balancing, forwards video processing requests to Fast CGI servers for processing. The teaching video processing system includes:

The teaching video file transmission module is used to download the original video file from the video storage server and upload the video file processed by the video processing system;

The teaching video GPU accelerated processing module is used to adjust the resolution of the video according to the video name and video resolution input by the user, transcode the video according to the video name and video format input by the user, and use the video name and start and Crop the video at the end time;

The teaching video automatic eye mosaic module is used to decompose the teaching video into image frames, detect the human face in the image, detect the human eye feature points in the face, and perform mosaic on the human eye feature point area. image frames into a video.

The teaching video file transmission module includes a user login module, a cookie processing module, a video file download module, and a video file upload module, wherein:

The user login module logs in to the video storage server according to the user name and password input by the user;

The cookie processing module is responsible for saving the cookie returned by the video storage server to the local and loading the cookie from the local;

The video file downloading module is responsible for downloading video files from the video storage server;

The video file uploading module is responsible for uploading the processed video to the video storage server.

The teaching video GPU accelerated processing module includes a video resolution adjustment module, a video transcoding module, and a video cropping module, wherein:

The video resolution adjustment module adjusts the video resolution: by parsing the request parameters in the HTTP POST request, the target resolution of the video is obtained, and then the FFMPEG tool is used to adjust the resolution of the specified video file. , fully exploit the parallel computing capability of GPU, and use NVENC and NVDEC provided by NVIDIA to accelerate video encoding and decoding to realize GPU-accelerated video processing and improve processing efficiency;

The video transcoding module converts the video format: by parsing the request parameters in the HTTP POST request, the video format of the target is obtained, and then the specified video file is used to convert the video format using the FFMPEG tool. In the process of conversion, Fully exploit the parallel computing capability of GPU, use NVENC and NVDEC provided by NVIDIA to accelerate video encoding and decoding, realize GPU-accelerated video processing, and improve processing efficiency;

The video cropping module cuts the video: by parsing the request parameter in the HTTP POST request, thereby obtaining the start time and the end time of the video that needs to be cropped, and then using the FFMPEG tool to crop the specified video file;

The automatic human eye mosaicking module of the teaching video includes a face detection module in an image, a human eye feature point detection module in a human face, a human eye feature point region mosaicking module and an image frame and audio synthesis video module, wherein:

The face detection module in the image automatically detects the coordinate points of the face area in an image: pre-trains a deep learning-based neural network model for detecting faces, and the model mainly improves the FaceBoxes face detector. Its network structure and increase the depth and width of the network, and then remove the unneeded indistinguishable samples in the system for training. After training the model, save the weights of the neural network model locally. When performing face detection, load the weight value of the neural network locally. When an image frame is input into the neural network model, the model will automatically detect the coordinate area of the face position in the image, including the upper left of the face area. The corner coordinate value, the coordinate value of the lower right corner of the face area, and the center coordinate value of the face area.

The eye feature point detection module in the face automatically detects the coordinate points of the human eye feature points in the face: pre-training a deep learning-based neural network model for detecting the human eye feature points from the face, the model improves O-Net in the MTCNN model, and adjust the dataset for training. After the model is trained, save the weights of the neural network locally. When detecting human eye feature points, the weight value of the neural network is loaded locally. When inputting the face area in an image frame into the model, the model will automatically detect the human eye feature points of the face;

The human eye feature point tracking module tracks the human eye feature points of two consecutive frames of images: pre-trains a deep learning neural network model based on human eye feature point tracking, and saves the weight value of the neural network locally. When tracking human eye feature points, the weight value of the neural network is loaded locally. When inputting the face area of the next frame into the model, the tracker first performs human eye feature point tracking. If the tracked human eye feature point results If it meets the expectations, continue to process the next frame of image, otherwise it will be handed over to the human eye feature point detection module for re-detection of human eye feature points;

The human eye feature point region mosaicking module performs mosaicking on the human eye feature point region: after detecting the human eye feature point in the image, the mosaic region is calculated according to the human eye feature point, and then the corresponding mosaic is generated;

The image frame and audio synthesis video module synthesizes the coded image frame and audio into a video file: after obtaining the coded image frame, combined with the audio previously separated in the video, the image frame and the audio are synthesized using FFMPEG tools. 's video.

The entire processing flow of the distributed teaching video processing system in this embodiment is shown in Figure 2. First, the video is uploaded from the browser to the video storage server. After the upload is successful, the browser sends a video processing request to the video processing system, and the video is processed. After receiving the request, the system downloads the video from the video storage server and processes it, and then uploads the processed video to the video storage server after the processing is completed.

The above-mentioned embodiments are only preferred embodiments of the present invention, and are not intended to limit the scope of implementation of the present invention. Therefore, any changes made according to the shape and principle of the present invention should be included within the protection scope of the present invention.

Claims (4)

1. A distributed teaching video processing system, comprising:

the teaching video file transmission module is used for downloading an original video file from the video storage server and uploading the video file processed by the video processing system;

the teaching video GPU acceleration processing module is used for adjusting the resolution of a video according to the video name and the video resolution input by a user, transcoding the video according to the video name and the video format input by the user, and cutting the video according to the video name, the starting time and the ending time input by the user;

the automatic human eye mosaic printing module for teaching video is used for decomposing teaching video into image frames, detecting human faces in the images, detecting human eye feature points in the human faces, printing mosaic on human eye feature point areas, and making the image frames and audio after printing into video.

2. A distributed instructional video processing system according to claim 1, wherein: the teaching video file transmission module comprises a user login module, a Cookie processing module, a video file downloading module and a video file uploading module, wherein:

the user login module logs in the video storage server according to a user name and a password input by a user;

the Cookie processing module is responsible for storing Cookies returned by the video storage server to the local and loading the Cookies from the local;

the video file downloading module is responsible for downloading video files from the video storage server;

and the video file uploading module is responsible for uploading the processed video to the video storage server.

3. A distributed instructional video processing system according to claim 1, wherein: the teaching video GPU acceleration processing module comprises a video resolution adjusting module, a video transcoding module and a video cutting module, wherein:

the video resolution adjusting module adjusts the video resolution: analyzing request parameters in the HTTP POST request to obtain a target resolution of a video, then using a video processing tool FFMPEG to adjust the resolution of a specified video file, fully mining the parallel computing capability of a GPU in the adjusting process, and accelerating video coding and decoding by using NVENC and NVDEC provided by NVIDIA to realize GPU accelerated video processing and improve the processing efficiency;

the video transcoding module converts the video format: analyzing request parameters in the HTTP POST request to obtain a target video format, then converting the video format of a specified video file by using a video processing tool FFMPEG, fully mining the parallel computing capability of a GPU in the conversion process, and accelerating video coding and decoding by using NVENC and NVDEC provided by NVIDIA to realize GPU accelerated video processing and improve the processing efficiency;

the video clipping module clips videos: the starting time and the ending time of the video needing to be cut are obtained by analyzing the request parameters in the HTTP POST request, and then the video processing tool FFMPEG is used for cutting the specified video file.

4. A distributed instructional video processing system according to claim 1, wherein: automatic people's eye of teaching video beats mosaic module includes in the image people's eye detection module, people's eye feature point detection module in the people's face, continuous interframe people's eye feature point tracking module, people's eye feature point region beat mosaic module and image frame and audio synthesis video module, wherein:

the in-image face detection module automatically detects coordinate points of a face area in one image: pre-training a neural network model for detecting a human face based on deep learning, storing a weight value of the neural network to a local area, loading the weight value of the neural network from the local area when detecting the human face, and automatically detecting a coordinate area of the human face in an image by the model when inputting an image frame into the model;

the module for detecting the characteristic points of the human eyes in the human face automatically detects the coordinate points of the characteristic points of the human eyes in the human face: pre-training a neural network model based on deep learning for detecting human eye feature points from a human face, storing the weight value of the neural network to the local, loading the weight value of the neural network from the local during human eye feature point detection, and automatically detecting the human eye feature points of the human face by the model when a human face region in an image frame is input into the model;

the continuous inter-frame human eye feature point tracking module tracks the human eye feature points of two continuous frames of images: pre-training a neural network model tracked by human eye feature points and based on deep learning, storing the weight value of the neural network to the local, loading the weight value of the neural network from the local during the tracking of the human eye feature points, when inputting the face area of the next frame into the model, tracking the human eye feature points by a tracker, if the tracked human eye feature point result is in line with expectation, continuing to process the image of the next frame, otherwise, handing the result to a human eye feature point detection module to detect the human eye feature points again;

the human eye characteristic point region mosaic printing module performs mosaic printing on the human eye characteristic point region: after human eye feature points are detected in the image, calculating a corresponding mosaic area according to the human eye feature points, and then generating a corresponding human eye mosaic;

the image frame and audio synthesis video module synthesizes the coded image frame and audio into a video file: after the coded image frames are obtained, the image frames and audio are combined into the final coded video using the video processing tool FFMPEG in combination with the previously separated audio in the video.

Images