CN102905200B - Video interesting area double-stream coding transmission method and system - Google Patents

Abstract

The invention discloses a video interesting area double-stream coding transmission method and a system, wherein the method adopts higher code rate to carry out enhanced coding on the interesting area, while the non-interesting area adopts general code rate to carry out coding, and the generated basic flow and the enhanced flow can complete transmission under the condition of limited bandwidth through distribution on a time domain. The invention determines the information certainty factor according to the occurrence probability of the specific semantic event, and carries out the interested coding facing the user, so as to effectively remove the information redundancy which is not interested by the user in the monitored video and greatly improve the overall coding efficiency of the monitored video. According to the method, the information certainty of the monitoring video is researched, a double-flow packaging method for the certainty self-adaptive scalable coding model and the optimal distribution of channel space domain-time domain resources is established, the self-adaptive coding of the interesting information of the monitoring video is realized, and the coding efficiency of the monitoring video is improved.

Description

Method and system for dual-stream encoding and transmission of video region of interest

technical field

The present invention relates to a method and system in the technical field of video encoding and transmission, in particular to a method for enhancing encoding for an area of interest and then performing dual-stream transmission on code streams.

Background technique

With the rapid development of politics and economy in today's society, video surveillance, as an effective preventive measure and post-event investigation method, has received more and more attention. Users' requirements for video surveillance systems are different from traditional entertainment videos. There are two main requirements: real-time requirements and high-quality requirements (also called after-the-fact access requirements).

H.264 is the tenth part of MPEG-4, which is proposed by the Joint Video Team (JVT, Joint Video Team) jointly composed of ITU-T Video Coding Experts Group (VCEG) and ISO/IEC Motion Picture Experts Group (MPEG). Compressed digital video codec standard. H264 has many excellent features: higher coding efficiency, high-quality video images, higher network adaptability, hybrid coding structure, H.24 can be applied in different occasions, error recovery function, and higher complexity. In addition, the H264 standard also supports Flexible Macroblock Order (FMO) and Scalable Coding (SVC) functions.

At present, based on the scalable feature in H.264, it can be applied to the different needs of different terminals, but there are few studies on applying H.264 to the field of video surveillance and performing adaptive coding. In the paper "Research on Multi-stream Coding and Transmission of Surveillance Video" (Master's Thesis of Shanghai Jiaotong University, 2012), a scheme for reasonably allocating bandwidth in the time domain is proposed, which effectively improves the coding efficiency and eliminates information redundancy in the time domain. However, the paper does not perform enhanced coding according to the spatial domain of interest to the user in the video, so there is a certain degree of spatial information redundancy. In the paper "Scalable ROI Algorithm for H.264/SVC-Based VideoStreaming" (IEEE International Conference on Consumer Electronics (ICCE), 2011), the center area of the video is enhanced with ROI encoding, but the division of the region of interest is based on the human eye model. It is not proposed for surveillance video sequences, and it does not eliminate redundant information in the time domain.

Aiming at the problem of mass data transmission and storage encountered by the rapid development of the current surveillance video industry, and according to the characteristics of users' needs for surveillance video that are different from traditional entertainment videos, this paper proposes a video double-layer ROI region spatio-temporal joint coding applied to video surveillance systems The transmission method performs high-quality coding on the region of interest in the abnormal event time period, and reasonably allocates the time-domain bandwidth of the code stream transmission. At the same time, considering that using a single PSNR value of the entire image to evaluate the video quality in a video sequence cannot reflect the difference between the ROI area and the non-ROI area, a utility function is proposed as an objective indicator to evaluate the video quality in the specific field of surveillance systems. Conduct objective evaluation and comprehensively consider the video in combination with subjective indicators.

Contents of the invention

The purpose of the present invention is to overcome the deficiencies of the prior art, and propose a method and system for enhancing encoding for the region of interest and then performing dual-stream transmission on the code stream, so that it can be based on the information of the region of interest of the user in the video The area of interest is enhanced to encode, and the temporal redundancy and spatial redundancy in the video are eliminated to achieve efficient utilization of bandwidth.

The present invention is achieved through the following technical solutions:

The present invention provides a dual-stream encoding transmission method for an area of interest. The dual-stream encoding transmission for an area of interest means that the area of interest uses a higher code rate for enhanced encoding, and the area of non-interest uses a general code rate for encoding. Coding, the generated basic stream and the enhanced stream are distributed through the time domain, so that it can complete the transmission under the condition of limited bandwidth; the method includes the following steps:

Step 1, on the basis of obtaining the information of the region of interest and abnormal information, obtain the specific encoding parameters of the basic video stream and the enhanced stream through the utility function, and obtain the optimal bandwidth allocation strategy at the same time;

The utility function refers to: fully considering the difference in the importance of information contained in the information of different regions in different periods of time in the video sequence, wherein the region of interest in the abnormal period has the highest degree of importance, and also has the highest weight in the utility function , the information in the non-abnormal time period is less important, and the corresponding weight assigned is also smaller.

Step 2, on the basis of step 1, perform enhanced encoding on the video content of the region of interest in the abnormal time period;

This method is mainly aimed at encoding the ROI area in the field of video surveillance. In the field of video surveillance, more attention is paid to license plates, zebra crossing areas in traffic, etc., which is different from the human eye focus model. Because the focus of this article is on the implementation of the ROI region coding algorithm, not on the extraction of the ROI region, the acquisition of the ROI region is a known condition.

The ROI area information is obtained by using the ROI extraction algorithm, and the second mode in FMO, that is, the foreground and background mode, is used to divide the image into ROI slice groups and background slice groups according to the FMO characteristics and ROI area information. Since the ROI areas of different slice groups are encoded independently, by prohibiting the prediction between slice groups, the code stream of the generated ROI area and the code stream of the non-ROI area can be independent of each other. In this way, the ROI area information obtained in advance is used as the foreground information in the second mode in FMO during encoding. Set the real-time stream bit rate for the basic layer and set the QP value for the enhancement layer, so that two parts of the stream can be obtained in the enhancement layer, that is, the enhanced stream in the ROI area and the enhanced stream in the non-ROI area. By deleting the enhanced code stream in the non-ROI area, only the basic layer code stream and the enhanced code stream in the ROI area remain, and decoding the two parts of the code stream can obtain a video sequence with higher image quality in the ROI area.

And because the ROI area information changes with time, that is, the ROI area is dynamic, so when the ROI area information changes, it is necessary to re-modify the parameters of the MBA setting, encode the video in the new time, and remove the enhancement layer after encoding The code stream of the non-ROI area is sufficient. The finally obtained multiple code streams are decoded and spliced together to achieve the effect of dynamic ROI area enhancement.

Step 3, perform dual-stream transmission according to the bandwidth policy allocated in Step 1;

The specific method of the dual-stream transmission is as follows: when an abnormal event occurs, only the basic layer code stream is transmitted; after the abnormal event occurs, the code rate of the basic layer code stream is reduced, so that the excess bandwidth is used to transmit the enhancement in the abnormal event Layer code stream, whose base enhancement layer code stream only contains the enhancement information of the region of interest. When the event detection module determines the occurrence of an abnormal event, and obtains the abnormal area information in the abnormal time period. The event detection module sends abnormal events and area information to the enhanced stream encoder. At the same time, there will be a corresponding algorithm to adaptively obtain the best code rate allocation scheme according to the size of the ROI area, and pass the real-time stream code rate and QP value encoded in the scheme to the real-time encoder and enhanced encoder. The enhanced encoder will perform enhanced encoding on the abnormal area of the abnormal event, and store the code stream. When the abnormal event ends, the system enters into the process of no abnormal event. At this time, encode according to the real-time encoding bit rate obtained in the solution just now, reduce the encoding bit rate of the real-time streaming video, and reserve bandwidth for enhanced stream transmission. After the high-quality streaming video transmission is completed, the encoding bit rate of the real-time streaming encoder is increased, and all available network transmission bandwidth is used for real-time streaming video transmission.

The optimal code rate allocation scheme adaptively obtained according to the size of the ROI area refers to: what the front end detects is the probability of whether an area is an area of interest to the user, and thus the probability determines whether it is an area of interest through a dual-stream ROI utility function region, and adaptively adjust the encoding parameters of the base layer and the enhancement layer, and the transmission bandwidth allocation of the base stream and the enhancement stream in the utility function.

The present invention also provides a system for realizing the above-mentioned double-stream coding transmission method for the region of interest, the system includes: a monitoring video sending end, a network channel and a monitoring video receiving end. in:

The sending end includes: event and area detection module, real-time stream encoder, enhanced stream encoder, real-time transmission module and non-real-time transmission module; real-time stream encoder and enhanced stream encoder respectively generate the aforementioned real-time stream and enhanced stream, The real-time transmission module and the non-real-time transmission module package and transmit the real-time stream and the enhanced stream respectively. The event and area detection module is used to detect the time and area of abnormal events, and use the obtained information to control the enhanced stream encoding and real-time stream encoding.

The receiving end includes: a real-time stream decoder, a storage module and a display module; the real-time stream decoder and display module provide users with real-time display services; the storage module stores real-time streams and enhanced streams to form high-quality streams, and together with the display device provides The user provides the service of checking after the fact.

Further, the real-time stream encoder generates a real-time stream according to the detection module information: for real-time browsing, the image quality is not high, but the real-time performance is high, and the video needs to be encoded in real time and immediately transmitted to the backend.

Further, the enhanced stream encoder generates the enhanced stream according to the information of the detection module: the sending end uses the event detection module to detect the time period in which an event occurs and the area in which the event occurs in the time period (ie, the region of interest) , the encoding end will perform enhanced encoding on the specific area of the time period. Since the enhancement stream is mainly to meet the role of post-event investigation and evidence collection, the real-time requirements are not high, but the image quality requirements for specific areas are high. Therefore, it is sufficient to schedule the enhanced stream to be transmitted within a period of time after the occurrence of the abnormal event. The enhanced stream contains the code stream of a specific time period and specific region, as well as time period and region information.

Compared with the prior art, the present invention has the following beneficial effects: the method of the present invention can not only eliminate redundant time information in the video sequence, but also eliminate redundant spatial information in the video sequence, thereby realizing the region of interest in abnormal events Enhanced encoding to improve bandwidth utilization. Using this solution, the user's interest information that cannot be clearly seen in some videos under a certain bandwidth can be seen clearly.

Description of drawings

Figure 1 Structural diagram of the dual-stream transmission system in the area of interest;

Figure 2 Image segmentation without using FMO;

Figure 3 FMO for the foreground and background sheet group mapping;

Figure 4 The 192nd frame of the original Tcross sequence;

Figure 5 Tcross double-layer ROI frame 192;

Figure 6 Tcross double-layer non-ROI frame 192;

Figure 7 Frame 192 of Tcross single stream.

Detailed ways

The embodiments of the present invention are described in detail below in conjunction with the accompanying drawings: this embodiment is implemented on the premise of the technical solution of the present invention, and detailed implementation methods and specific operating procedures are provided, but the protection scope of the present invention is not limited to the following the described embodiment.

In this embodiment, based on the JSVM platform, enhanced coding is performed on the region of interest in the abnormal time period, and then the obtained code stream is divided into a basic layer code stream and an enhanced layer code stream for dual-stream transmission. Under the limited bandwidth, the content of the region of interest in the abnormal time period can be seen more clearly in the video restored by the backend.

The ROI dual-stream coding transmission method involved in this embodiment includes the following specific steps:

Step 1, on the basis of obtaining the information of the region of interest and abnormal information, based on the degree of information certainty and through the utility function, the specific encoding parameters of the basic video stream and the enhanced stream are obtained, and the optimal bandwidth allocation strategy is obtained at the same time;

The main idea of research on high-efficiency coding of surveillance video adaptive to information certainty is: According to the definition of inverse Shannon information: information is the increase of certainty; the measure of information volume is changed from negative entropy to certainty. In surveillance video, the probability of occurrence of user-defined high-level semantics (specific events such as abnormal intrusion), that is, the certainty of information, becomes an effective measure to measure the importance of the video information, which directly reflects the user's interest in the information. . The degree of information certainty is introduced into video coding, and differential coding is adaptively carried out according to the degree of certainty of information in the video, so as to achieve the purpose of effectively compressing information redundancy. Based on the degree of information certainty, it guides the scalable coding of surveillance video in dimensions of time, space, and quality, that is, adaptive scalable coding of information certainty. Information is the increase of certainty; the measure of information volume changes from negative entropy to certainty.

The utility function was originally a concept in economics. There are two key issues in decision analysis: one is to quantify the uncertainty of the state of the phenomenon studied; the other is to assign values to various possible consequences. Generally speaking, the uncertainty of the state is described by the probability of various states, and the utility theory is used to study the value of the consequences. The utility function is introduced into the video research, and the video is evaluated through the utility function.

Finally, according to the optimal solution of the utility function and the limitation condition of the bandwidth, the specific coding parameters and the bandwidth allocation ratio are obtained by solving.

The system structure adopted in this embodiment is shown in Figure 1, including a monitoring video sending end, a network channel and a monitoring video receiving end.

After dual-stream encapsulation of surveillance video, network bandwidth resources need to be allocated in the process of encapsulating code stream transmission. The specific distribution is as follows:

When the event detection module determines the occurrence of an abnormal event, and obtains the abnormal area information in the abnormal time period. The event detection module sends abnormal events and area information to the enhanced stream encoder. At the same time, there will be a corresponding algorithm to adaptively obtain the best code rate allocation scheme according to the size of the ROI area, and pass the real-time stream code rate and QP value encoded in the scheme to the real-time encoder and enhanced encoder. The enhanced encoder will perform enhanced encoding on the abnormal area of the abnormal event, and store the code stream.

When the abnormal event ends, the system enters into the process of no abnormal event. At this time, encode according to the real-time encoding bit rate obtained in the solution just now, reduce the encoding bit rate of the real-time streaming video, and reserve bandwidth for enhanced stream transmission. After the high-quality streaming video transmission is completed, the encoding bit rate of the real-time streaming encoder is increased, and all available network transmission bandwidth is used for real-time streaming video transmission.

For the derivation process of "Research on Multi-stream Coding and Transmission of Surveillance Video" (Master's Thesis of Shanghai Jiao Tong University, 2012), this embodiment provides a mathematical expression under the meaning of code stream different from this document.

During abnormal events: the bit rate and transmission occupied bandwidth parameters of real-time streams and high-quality streams can be summarized by the following formula:

$\left\{\begin{array}{c}{\mathrm{<span\; class="notranslate">\; R</span>}}_{\mathrm{<span\; class="notranslate">\; r</span>}}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; B</span>}}_{\mathrm{<span\; class="notranslate">\; r</span>}}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; B</span>}\\ {\mathrm{<span\; class="notranslate">\; B</span>}}_{\mathrm{<span\; class="notranslate">\; h</span>}}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 0</span>}\end{array}\right.$

In the above formula, R _r is the code rate of real-time streaming video, B _r is the transmission bandwidth occupied by real-time streaming, and B is the total bandwidth available for the video surveillance system. B _h is the occupied transmission bandwidth of the high-quality stream.

When the abnormal event is over, the code rate of the real-time stream encoding is reduced, the transmission bandwidth occupied by the real-time stream is reduced, and the extra bandwidth is used for the transmission of the stored high-quality stream. It can be represented by the following formula.

B _r +B _h =B

In addition, the variable K=T _d /T _a is defined, which is the time ratio of the non-abnormal event process and the abnormal event process. Then the code rate and corresponding bandwidth of each time period need to satisfy the following formula. Among them, B _{r, AE} represents the bandwidth of the real-time stream after the abnormal event, R _{h, error} represents the code rate generated in the high-level ROI area during the abnormal event, R _{r, error} represents the real-time stream code rate during the abnormal event, and B _{r, error} represents the abnormal event Live streaming bandwidth at event time.

${\mathrm{<span\; class="notranslate">\; B</span>}}_{\mathrm{<span\; class="notranslate">\; r</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; AE</span>}}<span\; class="notranslate">\; +</span>\frac{{\mathrm{<span\; class="notranslate">\; R</span>}}_{\mathrm{<span\; class="notranslate">\; h</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; error</span>}}}{\mathrm{<span\; class="notranslate">\; K</span>}}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; B</span>}$

R _{r, AE} = B _{r, AE} < B

R _r,error =B _r,error =B

According to the code rate of the SVC standard coding for the QP value proposed in the existing literature, the relationship between the image distortion and the quantization step value is as follows. The subscript 0 corresponds to the base layer, the subscript 1 corresponds to the enhancement layer, D is the distortion (MSE), R is the code rate of the code stream, and Q is the coding quantization step size.

${\mathrm{<span\; class="notranslate">\; D.</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; Q</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; b</span>}<span\; class="notranslate">\; \&Center\; Dot;</span>{\mathrm{<span\; class="notranslate">\; Q</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}^{\mathrm{<span\; class="notranslate">\; \&beta;</span>}}$

D ₁ (Q ₀ ,Q ₁ )≈D ₀ (Q ₁ )

${\mathrm{<span\; class="notranslate">\; R</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; Q</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; a</span>}{\mathrm{<span\; class="notranslate">\; Q</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}^{\mathrm{<span\; class="notranslate">\; \&alpha;</span>}}$

R ₁ (Q ₀ ,Q ₁ )＝R ₀ (Q ₁ )-cR ₀ (Q ₀ )

Combined with the above formula, and according to the algorithm introduced below, the code rate optimization allocation scheme of R _r in the process of non-abnormal events and R _r and Q _h in the process of abnormal events can be obtained.

Information is the increase of certainty; the measure of information volume changes from negative entropy to degree of certainty I _r , as shown in the following formula:

I _r = log P

Evaluation of video quality is generally divided into two aspects, objective quality evaluation and subjective quality evaluation. This embodiment proposes a relatively reasonable objective index for evaluating video quality in a specific context of surveillance video, and this objective index is the ROI dual-stream utility function.

The relationship between PNSR and distortion is shown in the following formula:

$\mathrm{<span\; class="notranslate">\; u</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; PSNR</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 10</span>}<span\; class="notranslate">\; \&CenterDot;</span>\mathrm{<span\; class="notranslate">\; log</span>}<span\; class="notranslate">\; (</span>\frac{{\mathrm{<span\; class="notranslate">\; 255</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}{\mathrm{<span\; class="notranslate">\; MSE</span>}}<span\; class="notranslate">\; )</span>$

In order to make it appear that the second-order derivative is less than zero in terms of mathematical performance (that is, the improvement of video quality will appear as a marginal effect in the eyes of the human eye), the following two functions are constructed to deal with the initial PSNR of the video accordingly:

u _q,r = U _q,r (psnr _r )

u _q,h = U _q,h (psnr _h )

Among them, u _{q, r} is the video quality utility value of the real-time stream, psnr _r is the PSNR value of the real-time stream, u _{q, r} is the video quality utility value of the high-quality stream, psnr _h is the PSNR value of the high-quality stream, U _{q ,r} (·) is the video quality utility function of real-time streaming, and U _q,h (·) is the video quality utility function of high-quality streaming.

Use the function of y=x ^(1/n) to construct a function whose first-order derivative is greater than zero and whose second-order derivative is less than zero. The larger the value of n, the more obvious the marginal effect. Therefore, it can be obtained (in the example, n _r and n _h are 3 and 2 respectively):

${\mathrm{<span\; class="notranslate">\; u</span>}}_{\mathrm{<span\; class="notranslate">\; q</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; r</span>}}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; u</span>}}_{\mathrm{<span\; class="notranslate">\; q</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; r</span>}}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; psnr</span>}}_{\mathrm{<span\; class="notranslate">\; r</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; psnr</span>}}_{\mathrm{<span\; class="notranslate">\; r</span>}}^{\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; /</span>{\mathrm{<span\; class="notranslate">\; no</span>}}_{\mathrm{<span\; class="notranslate">\; r</span>}}}$

${\mathrm{<span\; class="notranslate">\; u</span>}}_{\mathrm{<span\; class="notranslate">\; q</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; h</span>}}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; u</span>}}_{\mathrm{<span\; class="notranslate">\; q</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; h</span>}}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; psnr</span>}}_{\mathrm{<span\; class="notranslate">\; h</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; psnr</span>}}_{\mathrm{<span\; class="notranslate">\; h</span>}}^{\mathrm{<span\; class="notranslate">\; 1</span>}<span\; class="notranslate">\; /</span>{\mathrm{<span\; class="notranslate">\; no</span>}}_{\mathrm{<span\; class="notranslate">\; h</span>}}}$

The final single-frame total utility function of the dual-stream ROI transmission method can be expressed as the following formula:

u _i ＝w _q,r *u _q,r,i +w _q,h,roi *u _q,h,roi,i *s _roi,i +w _q,h,nroi *u _{q,h,nroi, i} *s _nroi,i

Among them, w _{q, r} represent the weight of the utility of the real-time code stream in the actual monitoring without abnormal events, indicating the degree of attention to the video when no abnormal events occur, w _{q, h, roi} , w _{q, h, nroi} respectively represent the actual The weight of the effectiveness of the high-quality code stream ROI area and non-ROI area in the monitoring of abnormal events represents the difference in the degree of attention to the area of interest and the area of non-interest in the actual monitoring of abnormal events. The specific value will be shown in the following formula give. The values of u are the same as those introduced in the previous formulas, which represent the corresponding images encoded by certain encoding parameters that can be brought to the user (u _{q, r} correspond to the coding part without abnormal events, u _{q, h, roi} correspond to the sense of abnormal events ROI coding part, u _{q, h, nroi} correspond to the full graph utility value of abnormal event non-ROI coding part). Among them, s _{roi, i} and s _{nroi, i} represent the percentage of the total area of the roi area and the non-roi area in the i-th frame, respectively, which is reflected in the actual monitoring system as the space occupied by the region of interest in the i-th frame of the video in the overall video Proportion. When the event detection module determines whether an area is an ROI, it needs to be determined according to the degree of information certainty that the area is indeed an ROI area. As shown in the following formula, where Squre _roi,i represents the area of the i-th frame roi area, and Squre _whole represents the total area of the i-th frame video, where C is the threshold value for determining whether it is the information certainty of the ROI area, which means that only When the information certainty degree of a certain area is greater than a certain threshold value, the user will regard it as an area of interest. In practical applications, it needs to be determined according to different needs of users. Among them, I is the information certainty degree of the area. This patent is based on the information certainty degree. Judgment, but not the focus of this patent research. The selection of the specific value of C is also related to the algorithm adopted by the value of I.

${\mathrm{<span\; class="notranslate">\; S</span>}}_{\mathrm{<span\; class="notranslate">\; roi</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; =</span>\left\{\begin{array}{cc}\frac{{\mathrm{<span\; class="notranslate">\; Squre</span>}}_{\mathrm{<span\; class="notranslate">\; roi</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; i</span>}}}{{\mathrm{<span\; class="notranslate">\; Squre</span>}}_{\mathrm{<span\; class="notranslate">\; whole</span>}}}<span\; class="notranslate">\; ,</span>& {\mathrm{<span\; class="notranslate">\; I</span>}}_{\mathrm{<span\; class="notranslate">\; r</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; roi</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; ></span>\mathrm{<span\; class="notranslate">\; C</span>}\\ \mathrm{<span\; class="notranslate">\; 0</span>}<span\; class="notranslate">\; ,</span>& {\mathrm{<span\; class="notranslate">\; I</span>}}_{\mathrm{<span\; class="notranslate">\; r</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; roi</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; <</span>\mathrm{<span\; class="notranslate">\; C</span>}\end{array}\right\}$

Finally, each frame of a video stream is weighted to obtain the final video utility function, as shown in the following formula. Among them, N represents the total number of frames contained in an actual surveillance video, and the meanings of other variables are the same as the previous description of the total utility function of a single frame. Among them, E is the average utility value of a surveillance video, which is a reference evaluation index for the video quality of a surveillance video under certain bandwidth constraints.

$\begin{array}{c}\mathrm{<span\; class="notranslate">\; E.</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; u</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; N</span>}<span\; class="notranslate">\; \&Right\; Arrow;</span><span\; class="notranslate">\; +</span><span\; class="notranslate">\; \&infin;</span>}{\mathrm{<span\; class="notranslate">\; lim</span>}}\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; N</span>}}\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; N</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; u</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}\\ <span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; N</span>}<span\; class="notranslate">\; \&Right\; Arrow;</span><span\; class="notranslate">\; +</span><span\; class="notranslate">\; \&infin;</span>}{\mathrm{<span\; class="notranslate">\; lim</span>}}\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; N</span>}}\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; N</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; w</span>}}_{\mathrm{<span\; class="notranslate">\; q</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; r</span>}}<span\; class="notranslate">\; *</span>{\mathrm{<span\; class="notranslate">\; u</span>}}_{\mathrm{<span\; class="notranslate">\; q</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; r</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; w</span>}}_{\mathrm{<span\; class="notranslate">\; q</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; h</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; roi</span>}}<span\; class="notranslate">\; *</span>{\mathrm{<span\; class="notranslate">\; u</span>}}_{\mathrm{<span\; class="notranslate">\; q</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; h</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; roi</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; *</span>{\mathrm{<span\; class="notranslate">\; the\; s</span>}}_{\mathrm{<span\; class="notranslate">\; roi</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; w</span>}}_{\mathrm{<span\; class="notranslate">\; q</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; h</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; nroi</span>}}<span\; class="notranslate">\; *</span>{\mathrm{<span\; class="notranslate">\; u</span>}}_{\mathrm{<span\; class="notranslate">\; q</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; h</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; nroi</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; *</span>{\mathrm{<span\; class="notranslate">\; the\; s</span>}}_{\mathrm{<span\; class="notranslate">\; nroi</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; i</span>}}\end{array}$

It can be known from subjective experience that the smaller the region of interest, the greater the degree of attention people pay to the region, and when the region of interest expands to the entire image, the degree of attention to it decreases accordingly. In this method, the value of w _{q, h, roi} is shown in the following formula, which is a positive fixed value for a certain video sequence.

${\mathrm{<span\; class="notranslate">\; w</span>}}_{\mathrm{<span\; class="notranslate">\; q</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; h</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; roi</span>}}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 4</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 3</span>}<span\; class="notranslate">\; *</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{{\mathrm{<span\; class="notranslate">\; T</span>}}_{\mathrm{<span\; class="notranslate">\; error</span>}}}<span\; class="notranslate">\; *</span>\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{{\mathrm{<span\; class="notranslate">\; T</span>}}_{\mathrm{<span\; class="notranslate">\; error</span>}}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; S</span>}}_{\mathrm{<span\; class="notranslate">\; nroi</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; i</span>}}$

In the formula, T _error represents the number of frames of abnormal events in the actual surveillance video, and a weighted average is taken in the formula to obtain an average value of the area ratio of the region of interest. Among them, the meanings of parameters w _{q, h, roi} and S _{nroi, i} have been given in the description of the single-frame total utility function. That is, the weight of the region of interest changes from 4 to 1, and its value is inversely correlated with the value of the average area ratio of the region of interest. In addition, w _q,r and w _q,h,nroi all take 1 in this embodiment.

To sum up, the final expression of -E(u) can be obtained, and then the Fibonacci method is used to perform a one-dimensional search to solve R _r,AE (the code rate value of the real-time code stream after the abnormal event).

Step 2. On the basis of step 1, strengthen the encoding of the video content of the region of interest in the abnormal time period: this method is mainly aimed at encoding the ROI area in the field of video surveillance. In the field of video surveillance, more attention is paid to license plates, traffic In the zebra crossing area, etc., it is different from the human eye focus model.

For image slices, mapping information from macroblocks to slices is required. Usually, a frame of image is divided into one or more slices, and each slice contains several macroblocks, and the macroblocks are arranged in raster scanning order. As shown in Figure 2.

In addition, H.264 provides another macroblock mapping mode, namely Flexible Macro-block Ordering (FMO, FlexibleMacro-block Ordering). This mode uses the concept of slice groups, and the macroblocks contained in each slice group are determined by the macroblock slice group mapping table in the parameter set. There is a slice group belonging number for each macroblock in the macroblock slice group mapping table, which indicates which slice group the macroblock belongs to. Each slice group can have one or more slices, and the macroblocks in each slice are arranged in raster scan order in the corresponding slice. In addition, when the FMO method is not used, it can also be regarded as a special case of FMO, that is, the case where the entire image only contains one slice group.

The encoding of the ROI region can be implemented based on the FMO characteristics. Choose a certain ROI extraction algorithm to obtain ROI area information, and use the second mode in FMO (see Figure 3), that is, the foreground and background mode. According to the FMO characteristics and ROI area information, divide the image into ROI slice groups and background slices Group. Since the ROI areas of different slice groups are encoded independently, by prohibiting the prediction between slice groups, the code stream of the generated ROI area and the code stream of the non-ROI area can be independent of each other. In this way, the ROI area information obtained in advance is used as the foreground information in the second mode in FMO during encoding. Set the real-time stream bit rate for the basic layer and set the QP value for the enhancement layer, so that two parts of the stream can be obtained in the enhancement layer, that is, the enhanced stream in the ROI area and the enhanced stream in the non-ROI area. By deleting the enhanced code stream in the non-ROI area, only the basic layer code stream and the enhanced code stream in the ROI area remain, and decoding the two parts of the code stream can obtain a video sequence with higher image quality in the ROI area.

Step 3: Perform dual-channel encoding on the same surveillance video source to generate real-time streaming video and high-quality streaming video.

Real-time streaming: It is used for real-time browsing. It has low requirements for image quality and high requirements for real-time performance. It needs to encode the video in real time and transmit it to the backend immediately.

Enhanced flow: The sending end uses the event detection module to detect the time period where an event occurred and the area where the event occurred in this time period (ie, the area of interest), and the encoding end will perform enhanced encoding on the specific area of the time period. Since the enhancement stream is mainly to meet the role of post-event investigation and evidence collection, the real-time requirements are not high, but the image quality requirements for specific areas are high. Therefore, it is sufficient to schedule the enhanced stream to be transmitted within a period of time after the occurrence of the abnormal event. The enhanced stream contains the code stream of a specific time period and specific region, as well as time period and region information.

As shown in Figure 1, the structure of the dual-stream transmission system includes a monitoring video sending end, a network channel and a monitoring video receiving end.

The sending end includes: an event and area detection module, a real-time stream encoder, an enhanced stream encoder, a real-time transmission module, and a non-real-time transmission module. The two encoders respectively generate the aforementioned real-time stream and the enhanced stream, and the two transmission modules respectively encapsulate and transmit the real-time stream and the enhanced stream. The event and area detection module is used to detect the time and area of abnormal events, and use the obtained information to control enhanced stream coding and real-time stream coding, and quickly adjust the source code rate to be consistent with the effective transmission bandwidth of the channel.

The receiving end includes: a real-time stream decoder, a storage module and a display module. The real-time stream decoder and display module provide users with real-time display services. The storage module will store the real-time stream and the enhanced stream to form a high-quality stream, and together with the display device, it will provide the user with a post-mortem service.

The described rapid adjustment of the source code rate to be consistent with the effective transmission bandwidth of the channel refers to: firstly, the cleaning of the sender's cache, discarding the packets polluted by lost packets in the cache, and avoiding the transmission of invalid data and wasting bandwidth; The second is to control the encoding bit rate of the encoding module, which is jointly realized by modifying quantization parameters, image frame rate and image resolution, so that the encoding bit rate of the encoding module matches the estimated target bit rate.

After dual-stream encapsulation of surveillance video, network bandwidth resources need to be allocated in the process of encapsulating code stream transmission. The specific allocation strategy is as follows:

When the event detection module determines the occurrence of an abnormal event, and obtains the abnormal area information in the abnormal time period. The event detection module sends abnormal events and area information to the enhanced stream encoder. At the same time, there will be a corresponding algorithm to adaptively obtain the best code rate allocation scheme according to the size of the ROI area, and pass the real-time stream code rate and QP value encoded in the scheme to the real-time encoder and enhanced encoder. The enhanced encoder will perform enhanced encoding on the abnormal area of the abnormal event, and store the code stream.

When the abnormal event ends, the system enters into the process of no abnormal event. At this time, encode according to the real-time encoding bit rate obtained in the solution just now, reduce the encoding bit rate of the real-time streaming video, and reserve bandwidth for enhanced stream transmission. After the high-quality streaming video transmission is completed, the encoding bit rate of the real-time streaming encoder is increased, and all available network transmission bandwidth is used for real-time streaming video transmission.

Specific bandwidth parameters in allocation are determined according to Step 1.

Table 1 The test results of using this scheme in the abnormal event of T-cross sequence in this implementation

The specific images can be seen in the comparison of Figures 4, 5, 6, and 7.

It can be seen from the comparison that by adopting the method of the present invention, a more reasonable allocation of code rates in the time domain and the space domain can be realized, and the image quality of the picture during an abnormal event can be improved, so that the region of interest has a higher image quality and better It satisfies the needs of post-event forensics under the condition of limited bandwidth.

Although the content of the present invention has been described in detail through the above preferred embodiments, it should be understood that the above description should not be considered as limiting the present invention. Various modifications and alterations to the present invention will become apparent to those skilled in the art upon reading the above disclosure. Therefore, the protection scope of the present invention should be defined by the appended claims.

Claims (9)

1. A dual-stream encoding transmission method for an area of interest, characterized in that, the dual-stream encoding transmission for the area of interest refers to: the area of interest adopts a higher code rate for enhanced encoding, while the area of non-interest adopts a general code rate coded at a code rate, and the generated basic stream and enhanced stream are distributed in the time domain so that they can be transmitted under limited bandwidth conditions; the method includes the following steps: Step 1, on the basis of obtaining the information of the region of interest and abnormal information, obtain the specific encoding parameters of the basic video stream and the enhanced stream through the utility function, and obtain the optimal bandwidth allocation strategy at the same time; Step 2, on the basis of step 1, perform enhanced encoding on the video content of the region of interest in the abnormal time period; Step 3, perform dual-stream transmission according to the bandwidth policy allocated in Step 1; The specific method of the dual-stream transmission is as follows: when an abnormal event occurs, only the basic layer code stream is transmitted; after the abnormal event occurs, the code rate of the basic layer code stream is reduced, so that the excess bandwidth is used to transmit the enhancement in the abnormal event Layer code stream, whose base enhancement layer code stream only contains the enhanced information of the area of interest; when the event detection module determines the occurrence of an abnormal event and obtains the abnormal area information in the abnormal time period, the event detection module sends the abnormal event and area information to To the enhanced stream encoder, at the same time, according to the size of the ROI area, the optimal code rate allocation scheme is adaptively obtained, and the real-time stream code rate and QP value encoded in the scheme are passed to the real-time encoder and the enhanced encoder, and the enhanced encoder Enhanced encoding will be performed on the abnormal area of the abnormal event, and the code stream will be stored; when the abnormal event ends, the system will enter the process of no abnormal event. The encoding bit rate is reduced, and the bandwidth is reserved for enhanced stream transmission; when the high-quality streaming video transmission is completed, the encoding bit rate of the real-time streaming encoder is increased, and all available network transmission bandwidth is used for real-time streaming video transmission. 2. The method for dual-stream encoding and transmission of an area of interest according to claim 1, wherein the utility function refers to: fully considering the difference in the importance of information contained in the information of different regions in different time periods in the video sequence , in which the region of interest in the abnormal period has the highest importance, and also has the highest weight in the utility function, while the information in the non-abnormal period is less important, and the corresponding assigned weight is also smaller. 3. The ROI dual-stream encoding transmission method according to claim 1, wherein said adaptively obtaining the best code rate allocation scheme according to the size of the ROI area refers to whether the front end detects whether a piece of area is The probability of the user's region of interest, from which the probability determines whether it is a region of interest through the dual-stream ROI utility function, and adaptively adjusts the encoding parameters of the base layer and the enhancement layer, the transmission of the base stream and the enhancement stream in the utility function bandwidth allocation. 4. the region of interest dual-stream coding transmission method according to claim 3, is characterized in that, described dual-stream ROI utility function is specifically: The final single-frame total utility function of the dual-stream ROI transmission method is expressed as the following formula: u _i ＝w _q,r *u _q,r,i +w _q,h,roi *u _q,h,roi,i *s _roi,i +w _q,h,nroi *u _{q,h,nroi, i} *s _nroi,i Among them, w _{q, r} represent the weight of the utility of the real-time code stream in the actual monitoring without abnormal events, indicating the degree of attention to the video when no abnormal events occur, w _{q, h, roi} , w _{q, h, nroi} respectively represent the actual The weight of the effectiveness of the high-quality code stream ROI area and non-ROI area in the monitoring of abnormal events represents the difference in the degree of attention to the area of interest and the area of non-interest in the actual monitoring of abnormal events; each u value represents a certain encoding parameter The corresponding full-image utility value that the encoded picture can bring to the user, u _{q, r} correspond to the coding part of no abnormal event, u _{q, h, roi} correspond to the coding part of the region of interest of the abnormal event, u _{q, h, nroi} correspond to the abnormal event Event non-interest region coding part; s _roi,i and s _nroi,i represent the percentages of the i-th frame roi area and non-roi area in the total area, which is reflected in the actual monitoring system as the spatial proportion of the i-th frame of interest in the video as a whole ; And when judging whether a region is ROI by the event detection module, it needs to be determined according to the information certainty degree of the ROI region according to the region; Squre _{roi, i} represents the i-th frame roi region area, and Squre _whole represents the i-th frame video total area; C is to judge whether it is the domain value of the information certainty degree of ROI region, and I is the information certainty degree of the region; ${\mathrm{<span\; class="notranslate">\; S</span>}}_{\mathrm{<span\; class="notranslate">\; roi</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; =</span>\left\{\begin{array}{cc}\frac{{\mathrm{<span\; class="notranslate">\; Squre</span>}}_{\mathrm{<span\; class="notranslate">\; roi</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; i</span>}}}{{\mathrm{<span\; class="notranslate">\; Squre</span>}}_{\mathrm{<span\; class="notranslate">\; whole</span>}}}<span\; class="notranslate">\; ,</span>& {\mathrm{<span\; class="notranslate">\; I</span>}}_{\mathrm{<span\; class="notranslate">\; r</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; roi</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; ></span>\mathrm{<span\; class="notranslate">\; C</span>}\\ \mathrm{<span\; class="notranslate">\; 0</span>}<span\; class="notranslate">\; ,</span>& {\mathrm{<span\; class="notranslate">\; I</span>}}_{\mathrm{<span\; class="notranslate">\; r</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; roi</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; <</span>\mathrm{<span\; class="notranslate">\; C</span>}\end{array}\right\}$ Finally, each frame of a video stream is weighted to obtain the final video utility function, as shown in the following formula, where N represents the total number of frames contained in an actual surveillance video, and the meanings of other variables are the same as the total utility of a single frame before The description of the function is the same, E is the average utility value of a surveillance video, which is a reference evaluation index for the video quality of a surveillance video under certain bandwidth constraints; $\begin{array}{c}\mathrm{<span\; class="notranslate">\; E.</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; u</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; N</span>}<span\; class="notranslate">\; \&Right\; Arrow;</span><span\; class="notranslate">\; +</span><span\; class="notranslate">\; \&infin;</span>}{\mathrm{<span\; class="notranslate">\; lim</span>}}\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; N</span>}}\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; N</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; u</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}\\ <span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; N</span>}<span\; class="notranslate">\; \&Right\; Arrow;</span><span\; class="notranslate">\; +</span><span\; class="notranslate">\; \&infin;</span>}{\mathrm{<span\; class="notranslate">\; lim</span>}}\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; N</span>}}\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; N</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; w</span>}}_{\mathrm{<span\; class="notranslate">\; q</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; r</span>}}<span\; class="notranslate">\; *</span>{\mathrm{<span\; class="notranslate">\; u</span>}}_{\mathrm{<span\; class="notranslate">\; q</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; r</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; w</span>}}_{\mathrm{<span\; class="notranslate">\; q</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; h</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; roi</span>}}<span\; class="notranslate">\; *</span>{\mathrm{<span\; class="notranslate">\; u</span>}}_{\mathrm{<span\; class="notranslate">\; q</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; h</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; roi</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; *</span>{\mathrm{<span\; class="notranslate">\; the\; s</span>}}_{\mathrm{<span\; class="notranslate">\; roi</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; w</span>}}_{\mathrm{<span\; class="notranslate">\; q</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; h</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; nroi</span>}}<span\; class="notranslate">\; *</span>{\mathrm{<span\; class="notranslate">\; u</span>}}_{\mathrm{<span\; class="notranslate">\; q</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; h</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; nroi</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; *</span>{\mathrm{<span\; class="notranslate">\; the\; s</span>}}_{\mathrm{<span\; class="notranslate">\; nroi</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; i</span>}}\end{array}$ The value of w _{q, h, roi} is shown in the following formula, which is a positive fixed value for a certain video sequence, ${\mathrm{<span\; class="notranslate">\; w</span>}}_{\mathrm{<span\; class="notranslate">\; q</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; h</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; roi</span>}}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 4</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 3</span>}<span\; class="notranslate">\; *</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{{\mathrm{<span\; class="notranslate">\; T</span>}}_{\mathrm{<span\; class="notranslate">\; error</span>}}}<span\; class="notranslate">\; *</span>\underset{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{{\mathrm{<span\; class="notranslate">\; T</span>}}_{\mathrm{<span\; class="notranslate">\; error</span>}}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}{\mathrm{<span\; class="notranslate">\; S</span>}}_{\mathrm{<span\; class="notranslate">\; nroi</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; i</span>}}$ In the formula, T _error represents the number of frames of abnormal events in the actual surveillance video. A weighted average is used in the formula to obtain an average value of the area ratio of the region of interest, that is, the weight of the region of interest is changed from 4 to 1. Its value is inversely correlated with the value of the area ratio of the average region of interest; In summary, the final expression of -E(u) can be obtained, and then the Fibonacci method is used to perform one-dimensional search to solve R _r,AE , where R _r,AE is the code rate value of the real-time code stream after the abnormal event. 5. The ROI dual-stream encoding transmission method according to any one of claims 1-4, characterized in that, the enhanced encoding of the video content of the ROI in the abnormal time period is specifically: obtaining the area information of the ROI , using the second mode in FMO, that is, the foreground and background mode, according to the FMO characteristics and ROI area information, the image is divided into ROI slice groups and background slice groups; the ROI areas of different slice groups are independently coded, by prohibiting The prediction can realize that the generated ROI area code stream and non-ROI area code stream are independent of each other, so that the ROI area information obtained in advance can be used as the foreground information of the second mode in FMO during encoding; set the real-time stream code rate and enhancement for the basic layer If the QP value is set in the enhancement layer, two parts of the code stream can be obtained in the enhancement layer, that is, the enhanced code stream in the ROI area and the enhanced code stream in the non-ROI area. By deleting the enhanced code stream in the non-ROI area, only the basic layer code stream and The code stream is enhanced in the ROI area, and the two-part code stream is decoded to obtain a video sequence with higher image quality in the ROI area. 6. The method for dual-stream encoding and transmission of regions of interest according to claim 5, wherein the information of the ROI area changes with time, that is, the ROI area is dynamic, and needs to be re-modified when the information of the ROI area changes Set the parameters of MBA, encode the video in the new time, remove the code stream of the non-ROI area of the enhancement layer after encoding, and finally obtain the video sequence after decoding multiple code streams to achieve the effect of dynamic ROI area enhancement . 7. An ROI dual-stream coded transmission system for the method according to any one of claims 1-6, characterized in that it comprises: a monitoring video sending end, a network channel and a monitoring video receiving end, wherein: The monitoring video sending end includes: event and area detection module, real-time stream encoder, enhanced stream encoder, real-time transmission module and non-real-time transmission module; real-time stream encoder and enhanced stream encoder generate the aforementioned real-time stream and enhanced stream respectively The stream, real-time transmission module and non-real-time transmission module encapsulate and transmit the real-time stream and the enhanced stream respectively; the event and area detection module is used to detect the time and area of abnormal events, and use the obtained information to control the enhanced stream encoding and real-time stream encoding; The network channel and surveillance video receiving end include: real-time stream decoder, storage module and display module; real-time stream decoder and display module provide users with real-time display services; the storage module will store real-time stream and enhanced stream to form high-quality stream, Together with the display device, it provides users with the service of after-the-fact access. 8. The dual-stream encoding and transmission system for an area of interest according to claim 7, wherein the real-time stream encoder generates a real-time stream according to the information of the detection module, for real-time browsing, and does not require high image quality, but real-time The performance requirements are high, and the video needs to be encoded in real time and transmitted to the backend immediately. 9. The ROI dual-stream encoding transmission system according to claim 7, wherein the enhanced stream encoder generates the enhanced stream according to the information of the detection module, and the sending end uses the event detection module to detect the occurrence of a certain event The time period and the area where the event occurred in the time period is the area of interest. The encoding end will strengthen the encoding of the specific area of the time period, and arrange the enhanced stream to be transmitted within a period of time after the occurrence of the abnormal event. The enhanced stream includes a specific period of time. Code stream for a specific region, time period and region information.