CN104105006B - A kind of method of video image processing and system - Google Patents

Abstract

The invention discloses a video image processing method and system. The method includes: the ROI calculation server determines the ROI and non-ROI of the image in the video image data based on a set rule; the ROI calculation server provides the video image data and the ROI division scheme of the video image data to at least one encoding server; encoding The server obtains the encoding requirement and the ROI quality control strategy, and selects the ROI division scheme and the ROI quality control strategy according to the encoding requirement; performs encoding processing on the video image data according to the selected ROI division scheme and the ROI quality control strategy. The present invention provides an architectural pattern that separates ROI calculation from video encoding. ROI division schemes and ROI quality control strategies can be flexibly assembled to better respond to different coding requirements. The ROI calculation and encoding are operated separately, so that the video image data ROI division scheme does not need to be repeatedly calculated, saving ROI calculation resources, improving computing efficiency, and ensuring real-time transmission of video programs.

Description

A video image processing method and system

technical field

Embodiments of the present invention relate to video processing technologies, and in particular, to a video image processing method and system.

Background technique

With the development of video technology and media communication technology, the dissemination and sharing of video image data has become an indispensable means of information sharing in people's work and life.

In the face of massive video image data, in order to reduce the transmission bit rate, it is usually necessary to compress and encode the video image at the data source. Compression will cause a certain loss in the quality of the video image, and it is difficult to take into account the relationship between the transmission bit rate and the video quality. At the same time, for the same video image data provided by the same data source, different receivers will also have different quality requirements and transmission capacity limitations, which also put forward higher requirements for the processing of video image data.

Therefore, if video quality, transmission bit rate and flexible requirements of different receivers can be well taken into account, it becomes one of the problems to be solved in video image processing technology.

Contents of the invention

The invention provides a video image processing method and system to optimize the video image processing technology and improve its flexibility.

In a first aspect, an embodiment of the present invention provides a video image processing method, including:

The region of interest ROI calculation server determines the ROI and non-ROI of the image in the video image data based on the set rules;

The ROI calculation server provides the video image data and the ROI division scheme of the video image data to at least one encoding server;

The encoding server acquires encoding requirements and ROI quality control strategies, and selects the ROI division scheme and ROI quality control strategies according to the encoding requirements;

The encoding server performs encoding processing on the video image data according to the selected ROI division scheme and ROI quality control policy.

In the second aspect, an embodiment of the present invention also provides a video image processing system, including a region of interest ROI calculation server and at least one encoding server, wherein:

The ROI calculation server includes a ROI determination module and a data provision module, the ROI determination module is used to determine the ROI and non-ROI of the image in the video image data based on the set rules, and the data provision module is used to use the video image data , and the ROI division scheme of the video image data, provided to at least one encoding server;

Each of the encoding servers, interacting with the ROI calculation server, includes a strategy selection module and an image encoding module, and the strategy selection module is used to obtain encoding requirements and ROI quality control strategies, and select the An ROI division scheme and an ROI quality control strategy, the image encoding module is configured to encode the video image data according to the selected ROI division scheme and ROI quality control strategy.

Embodiments of the present invention provide an architectural mode that separates ROI calculation from video encoding. Using this model, operators who control each encoding server can flexibly assemble ROI division schemes and ROI quality control strategies to better respond to different encoding requirements. The ROI calculation and encoding are operated separately, so that the video image data ROI division scheme does not need to be repeatedly calculated, saving ROI calculation resources, improving computing efficiency, and ensuring real-time transmission of video programs.

Description of drawings

FIG. 1 is a flowchart of a video image processing method provided by Embodiment 1 of the present invention;

FIG. 2 is a schematic structural diagram of a video image processing system applicable to an embodiment of the present invention;

FIG. 3 is a schematic structural diagram of a video image processing system provided by Embodiment 2 of the present invention.

detailed description

The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present invention, but not to limit the present invention. In addition, it should be noted that, for the convenience of description, only some structures related to the present invention are shown in the drawings but not all structures.

Embodiment one

FIG. 1 is a flow chart of a video image processing method provided by Embodiment 1 of the present invention. This embodiment is applicable to performing Region of Interest (ROI for short) calculation and encoding processing on video image data provided by a data source. In the case of this method, the method is executed by a video image processing system. As shown in FIG. 2 , the system includes an ROI calculation server 210 and at least one encoding server 220. The two servers are preferably physically independent, but may also be physically integrated in the In one server, the logical functions of the two servers are realized respectively.

The method specifically includes the following steps:

Step 110, the ROI calculation server determines the ROI and non-ROI of the image in the video image data based on the set rules;

The above steps are used to determine the ROI and non-ROI of the image in the video image data, which can be determined on a frame-by-frame basis for all or part of the frame images in the video, or can be determined in batches for multiple frame images, and finally form the video image data. The ROI division scheme of the image. There may be various implementation forms for setting the rules, for example, the ROI in the image may be manually marked by the user through human-computer interaction. Or, based on the visual attention mechanism, the salient degree of the features in the image is calculated by establishing a visual attention model, and then the region where the salient features are located in the image is extracted as the ROI. The saliency is the contrast effect of a feature of a pixel in the image with the corresponding features of other pixels in the surrounding environment.

Preferably, the embodiment of the present invention also provides an ROI determination rule with a more concise determination method and a small amount of calculation, that is, the ROI calculation server performs the middle area, face For region and/or subtitle region identification, the identified region is used as the ROI, and the remaining regions are used as non-ROIs. In the above technical solution, when determining the ROI, a method of combining the fixed ROI area and the floating ROI area is adopted. The middle area can be set as a fixed ROI area, and the fixed middle area can be used as the ROI area in each frame. Also set the face area and the subtitle area as the floating ROI area, that is, carry out targeted recognition of the face and subtitle in each frame, and determine the ROI area that floats and changes. In this operation plan, the three types of areas are determined as ROI areas without manual interaction, so the determination method is simple. At the same time, the three types of areas have basically covered the positions that people focus on, so the determination of ROI is relatively accurate. The amount of calculation is also small. Of course, the target area may not be limited to the above three categories, and other target areas with targeted features may also be determined according to requirements. For example, the region where the moving object is located can also be used as a floating ROI region, and the face of an anime character can also be recognized as a human face, such as a mammoth in Ice Age. The identification of moving objects can be determined by correlating the images of several frames before and after.

Step 120, the ROI calculation server provides the video image data and the ROI division scheme of the video image data to at least one encoding server;

The ROI division scheme can be expressed in various ways, for example, the encoding server is notified of the division of ROI and non-ROI in the image by means of region coordinates, pixel coordinates, and the like. Specifically, an identifier of the video image data, such as a program number, may be provided to represent the video image data, and then a frame number and a corresponding ROI division scheme may be provided to determine the ROI division of each frame image. When the setting rules are different, one or more ROI division schemes may be generated for the same image.

Step 130, the encoding server acquires encoding requirements and ROI quality control strategies, and selects the ROI division scheme and ROI quality control strategies according to the encoding requirements;

For any encoding server, it can determine the encoding requirements according to its own situation or the situation of the video image receiver. For example, the encoding server may need to provide high-definition, standard-definition or smooth images, or the encoding rate may need to be adjusted due to changes in its transmission bandwidth. The encoding scheme can be determined from the above factors. The ROI quality control strategy is a quality processing strategy for determining how to distinguish between ROIs and non-ROIs based on the ROI division scheme. There are one or more control strategies, specified by the third-party controller, or multiple types are pre-configured on the encoding server, or specified by the ROI calculation server. The encoding server may select an ROI division scheme and a corresponding ROI quality control strategy according to encoding requirements.

Step 140, the encoding server encodes the video image data according to the selected ROI division scheme and ROI quality control strategy.

Preferably, in the ROI quality control strategy, the processed image quality of the ROI is higher than the processed image quality of the non-ROI. Usually, the ROI is an area that people will pay attention to, so the image quality of the ROI can be controlled to be higher than that of the non-ROI, for example, the definition of the ROI is high, and the non-ROI is blurred. The image quality control is typically reflected in the compression quality, that is, the encoding server performs an encoding process on the video image data according to the selected ROI division scheme and ROI quality control strategy, which may be specifically: the encoding According to the selected ROI division scheme and ROI quality control strategy, the server uses the first compression algorithm to compress the ROI, and uses the second compression algorithm to compress the non-ROI, wherein the image of the first compression algorithm The loss rate is lower than the image loss rate of the second compression algorithm. For example, the QP (Quoted-printable) method can be used for compression encoding, and the lossy parameters in the lossy compression algorithm can be controlled to achieve different image qualities, then the ROI quality control strategy is the QP difference Strategy.

Embodiments of the present invention provide an architectural mode that separates ROI calculation from video encoding. Using this model, operators who control each encoding server can flexibly assemble ROI division schemes and ROI quality control strategies to better respond to different encoding requirements. The ROI calculation and encoding are operated separately, so that the video image data ROI division scheme does not need to be repeatedly calculated, saving ROI calculation resources, improving computing efficiency, and ensuring real-time transmission of video programs. In practical application, this scheme can realize real-time transmission of 70 channels of programs in 1 channel of 38M digital bandwidth.

Embodiment two

FIG. 3 is a schematic diagram of the architecture of a video image processing system provided by Embodiment 2 of the present invention. The system includes an ROI calculation server 210 and at least one encoding server 220. FIG. 3 shows a situation including an encoding server 220. FIG. The case where multiple encoding servers 220 are included is shown. The ROI calculation server 210 and the encoding server 220 are physically set independently, and perform data interaction through wired or wireless means.

The architecture mode of the system is not limited to those shown in Figure 2 and Figure 3, and there are also multiple implementations, for example, the ROI calculation server and the encoding server can also be physically integrated in one server to implement their logic respectively Function. The number of ROI calculation servers can be multiple, and there are many interactive connection relationships with each encoding server, so that the ROI calculation can be processed in a distributed manner.

The ROI calculation server 210 includes a ROI determination module 211 and a data provision module 212, the ROI determination module 211 is used to determine the ROI and non-ROI of the image in the video image data based on a set rule, and the data provision module 212 is used to use The video image data, and the ROI division scheme of the video image data, are provided to at least one encoding server 220;

Each of the encoding servers 220, interacting with the ROI calculation server 210, includes a strategy selection module 221 and an image encoding module 222, and the strategy selection module 221 is used to obtain encoding requirements and ROI quality control strategies, and according to the The encoding requirement selects the ROI division scheme and ROI quality control strategy, and the image encoding module 222 is configured to encode the video image data according to the selected ROI division scheme and ROI quality control strategy.

On the basis of the above solution, in the preferred ROI quality control strategy, the processed image quality of the ROI is higher than the processed image quality of the non-ROI.

The strategy selection module 221 can specifically be configured to compress the ROI using a first compression algorithm and compress the non-ROI using a second compression algorithm according to the selected ROI division scheme and ROI quality control strategy, wherein, The image loss rate of the first compression algorithm is lower than the image loss rate of the second compression algorithm.

The ROI determining module 211 can be specifically configured to identify the middle area, face area and/or subtitle area of the image in the video image data, and use the identified area as the ROI, and the remaining areas as non-ROIs.

The above-mentioned product can execute the method provided by any embodiment of the present invention, and has corresponding functional modules and beneficial effects for executing the method.

Embodiments of the present invention provide an architectural mode that separates ROI calculation from video encoding. Using this model, operators who control each encoding server can flexibly assemble ROI division schemes and ROI quality control strategies to better respond to different encoding requirements. The ROI calculation and encoding are operated separately, so that the video image data ROI division scheme does not need to be repeatedly calculated, saving ROI calculation resources, improving computing efficiency, and ensuring real-time transmission of video programs.

Note that the above are only preferred embodiments of the present invention and applied technical principles. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the protection scope of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and can also include more other equivalent embodiments without departing from the concept of the present invention, and the present invention The scope is determined by the scope of the appended claims.

Claims (9)

1. A kind of 1. method of video image processing, it is characterised in that including：

   Region of interest ROI calculation server determines the ROI and non-ROI of image in vedio data based on setting rule, its In, the quantity of the ROI calculation servers is multiple；

   The ROI calculation servers carry the ROI splitting schemes of the vedio data, and the vedio data Supply at least one encoder server；

   The encoder server obtains code requirement and ROI Quality Control Strategies, and is counted according to the code requirement from multiple ROI Calculate and ROI splitting schemes are selected in multiple ROI splitting schemes that server provides and are selected from multiple ROI Quality Control Strategies ROI Quality Control Strategies, to realize flexible assembling ROI splitting schemes and ROI Quality Control Strategies；

   The encoder server enters according to the ROI splitting schemes and ROI Quality Control Strategies of selection to the vedio data Row coded treatment.
2. 2. according to the method for claim 1, it is characterised in that：In the ROI Quality Control Strategies, the processing of the ROI Picture quality after processing of the picture quality higher than the non-ROI afterwards.
3. 3. according to the method for claim 2, it is characterised in that the encoder server divides according to the ROI of selection Scheme and ROI Quality Control Strategies carry out coded treatment to the vedio data to be included：

   The encoder server is according to the ROI splitting schemes and ROI Quality Control Strategies of selection, to the ROI using the One compression algorithm is compressed, and the non-ROI is compressed using the second compression algorithm, wherein, first compression algorithm Image impairment rate be less than second compression algorithm image impairment rate.
4. 4. according to any described methods of claim 1-3, it is characterised in that the ROI calculation servers are based on setting rule Determine that the ROI of image and non-ROI include in vedio data：

   The ROI calculation servers, in the vedio data for image carry out intermediate region, human face region and/or Subtitle region identifies that, using the region recognized as the ROI, remaining area is as non-ROI.
5. 5. a kind of video image processing system, it is characterised in that including multiple semi-cylindrical hills ROI calculation servers and at least one Individual encoder server, wherein：

   The ROI calculation servers include ROI determining modules and data providing module, and the ROI determining modules are used to be based on setting Set pattern then determines the ROI and non-ROI of image in vedio data, and the data providing module is used for the video image number According to, and the ROI splitting schemes of the vedio data, there is provided at least one encoder server；

   Each encoder server, interacts with the ROI calculation servers, including strategy selection module and Image Coding mould Block, the strategy selection module are used to obtain code requirement and ROI Quality Control Strategies, and according to the code requirement from multiple ROI splitting schemes are selected in multiple ROI splitting schemes that ROI calculation servers provide and from multiple ROI Quality Control Strategies Middle selection ROI Quality Control Strategies, to realize flexible assembling ROI splitting schemes and ROI Quality Control Strategies, described image coding Module is used to carry out coded treatment to the vedio data according to the ROI splitting schemes and ROI Quality Control Strategies of selection.
6. 6. system according to claim 5, it is characterised in that in the ROI Quality Control Strategies, the processing of the ROI Picture quality after processing of the picture quality higher than the non-ROI afterwards.
7. 7. system according to claim 6, it is characterised in that the strategy selection module is specifically used for the institute according to selection ROI splitting schemes and ROI Quality Control Strategies are stated, the ROI is compressed using the first compression algorithm, to the non-ROI It is compressed using the second compression algorithm, wherein, the image impairment rate of first compression algorithm is calculated less than the described second compression The image impairment rate of method.
8. 8. according to any described systems of claim 5-7, it is characterised in that the ROI determining modules are specifically used for described In vedio data for image carry out intermediate region, human face region and/or subtitle region identification, using the region recognized as The ROI, remaining area is as non-ROI.
9. 9. system according to claim 6, it is characterised in that the ROI calculation servers and the encoding service implements It is independently arranged in reason, data interaction is carried out by wired or wireless way.