TWI504244B - Method, system, controlling device and processing device for distributively processing video stream - Google Patents

Description

Video stream dispersion processing method, distributed processing system, control host And computing host

The present invention relates to a distributed processing method, a distributed processing system, a control host, and a computing host, and particularly relates to a distributed processing method for a video stream, a distributed processing system, a control host, and a computing host.

In recent years, the demand for multimedia applications and video surveillance has increased, resulting in a huge increase in the amount of video data. The demand for using computers to process large amounts of video data has also increased. For example: (1) for video surveillance video, license plate recognition or flow counting functions to assist in statistical analysis; (2) quality enhancement processing for video screens to improve their visibility; (3) for video screens Video enrichment to reduce the time it takes to access the video; (4) to retrieve the time and place of a particular object from a large number of videos; or (5) to compress the format for high resolution video to reduce the amount of data for profit Store applications such as transfer. The above-mentioned applications for identification, statistics, enhancement, enrichment, retrieval or transcoding of video content are generally referred to as Video Processing (VP). With the continuous improvement of video resolution, the traditional stand-alone computing architecture has been unable to load the computational demands of video processing.

This case relates to a distributed processing method of video streaming, a distributed processing system, a control host, and a computing host.

According to the first aspect of the present invention, a distributed processing of video streams is proposed. method. The distributed processing method of the video stream includes the following steps. A video stream is cut to generate at least one group of pictures (GOP). Generate a work unit (task). The work unit records the picture group and its corresponding one of the processing functions. Determining one of the computing units corresponding to the working unit, the action determines the computing host based at least on the dependency of the processing function. Transfer the work unit to the corresponding computing host.

According to a second aspect of the present invention, a distributed processing system for video streaming is proposed. The video streaming decentralized processing system includes a control host and at least one computing host. The control host includes a video parser, a task generator, and a task scheduler. The video cutter cuts a video stream to generate at least one group of pictures (GOP). The work unit generator generates a work unit (task). The work unit records the picture group and its corresponding one of the processing functions. The work unit corresponds to one of the computing hosts. The work unit scheduler determines the computing host based on the dependencies of the processing functions.

According to a third aspect of the present invention, a control host for video streaming is proposed. The video stream control host includes a video parser, a task generator, and a task scheduler. The video cutter cuts a video stream to generate at least one group of pictures (GOP). The work unit generator generates a work unit (task). The work unit records the picture group and its corresponding one of the processing functions. The work unit scheduler determines one of the computing units corresponding to the work unit according to the dependency of the processing function.

According to a fourth aspect of the present invention, a computing host for video streaming is proposed. The video stream computing host is coupled to a task scheduler. A video stream is cut into at least one group of pictures (GOP). The work unit records the picture group and its corresponding one of the processing functions. The work unit corresponds to one of the computing hosts. The work unit scheduler determines the computing host based on the dependencies of the processing functions. The computing host includes a task processor and a result processor. The work unit processor processes the picture group according to the processing function. The result processor displays or stores the processing results of the picture group.

In order to better understand the above and other aspects of the present invention, the following specific embodiments, together with the drawings, are described in detail below:

Please refer to FIG. 1 to FIG. 2 . FIG. 1 is a flowchart of a distributed processing method of a video stream VS (indicated in FIG. 2 ), and FIG. 2 illustrates a distributed processing system 200 of a video stream VS. Schematic diagram. The distributed processing method of the video stream VS includes a controller side processing flow CS and a processor side processing flow PS.

The control side processing flow CS is executed by the control host 200C of the distributed processing system 200 of the video stream VS. The control host 200C performs work scheduling and distribution of data, such as a server, a personal computer, a firmware circuit, a processing chip, or a storage medium storing the array code.

The control host 200C includes a video parser 210, a task generator 220, and a data unit. Library 230 and a task scheduler 240. The video cutter 210 performs cutting of the video material. The work unit generator 220 creates a work unit 300 (shown in Figure 3). The database 230 stores various materials. The work unit scheduler 240 is arranged for each work unit 300. The video cutter 210, the work unit generator 220, and the work unit scheduler 240 are, for example, a firmware, an integrated circuit, a processing chip, or a storage medium for storing array code. The database 230 is, for example, a hard disk, a memory or a compact disc.

The arithmetic end processing flow PS is executed by the computing host 200P. The computing host 200P executes various video data computing programs, such as a server, a personal computer, a smart phone, a firmware, an integrated circuit, a processing chip, or a storage medium for storing array code.

The computing host 200P includes at least one work unit processor 250 and a result processor 260. The work unit processor 250 performs various processes for the scheduled work unit 300. The result processor 260 performs the storage or output of the results.

Hereinafter, the distributed processing system of the video stream VS of Fig. 1 will be described in detail with reference to the distributed processing system 200 of the video stream VS of Fig. 2. After receiving the operation request RS and the video stream VS, the control host 200C starts the distributed processing method of the video stream VS.

In step 102, the video cutter 210 cuts the video stream VS to generate at least one group of pictures (GOP). In step 102, the video cutter 210 performs cutting, for example, by using an Intra frame (I-frame) and a Predicted Frame (P-frame), and inserts all the frames between the two key frames. Cut the grid into one Picture group. Alternatively, the video cutter 210 can also cut the video stream VS according to the operating frequency.

In step 104, the work unit generator 220 generates a work unit 300. Please refer to FIG. 3 , which shows a schematic diagram of the working unit 300 . Each work unit 300 records the number T _{i,j of the} work unit 300 (corresponding to the jth picture group of the i-th video stream VS), the number of the picture group D _i,j (ie, the i-th video stream VS The jth picture group) and the corresponding number L _k of the processing function. The number D _{i,j of the} corresponding picture group can be known from the number T _{i,j of the} work unit 300. Then, the corresponding group of pictures can be retrieved from the picture group record table 312 of the video library 310. In addition, the number L _{k of the} corresponding processing function can also be known from the number T _{i,j of the} working unit 300. The corresponding processing function can be retrieved from the processing function record table 322 of the processing function library 320.

In step 106, the work unit scheduler 240 determines the computing host 200P to which the work unit 300 corresponds. This step determines the computing host 200P corresponding to the working unit 300 according to the dependency of the processing function, the reusability of the processing function, and the reusability of the picture group.

In step 108, the control host 200C transfers the work unit 300 to the corresponding computing host 200P. After the working unit 300 is transferred to the computing host 200P, the processing flow PS is entered.

In step 118, the work unit processor 250 processes the picture group in accordance with the processing function corresponding to the work unit 300.

In step 120, the computing host 200P returns the state of the computing host 200P to the controlling host 200C, and the result processor 260 of the computing host 200P displays or stores the processing result of the group of pictures.

In step 110, the database 230 updates the status of the computing host 200P. The state, for example, is that the recording operation unit 200P has executed the work unit 300.

In step 106 of FIG. 1, the work unit scheduler 240 determines the computing host 200P corresponding to the working unit 300 according to the dependency of the processing function, the reusability of the processing function, and the reusability of the picture group. This step can be done in several sub-steps. Please refer to FIG. 4, which shows a detailed flow chart of step 106 of FIG. 1. In Fig. 4, the input is the work unit 300, and the output is the number p of the most suitable computing host 200P. The work unit 300 records the number D _{i,j of} the picture group (ie _{, the jth} picture group of the i-th video stream VS) and the number L _{k of the} processing function.

In step 410, the work unit scheduler 240 determines whether the combination of the numbers i and k corresponding to the work unit 300 falls into a specific processor list. If so, it means that the number L _k processing function required by the i-th video stream VS of the work unit 300 is completely dependent, and has been assigned to the computing host 200P of the specific host list.

In step 420, the number p is set to one of a particular host list to cause the work unit 300 to be executed by the designated computing host 200P of the particular host list.

In step 410, if the combination of the numbers i and k corresponding to the work unit 300 does not fall into the specific host list, then step 430 is entered.

In step 430, the work unit scheduler 240 integrates all of the operational hosts 200P that can be used.

In step 440, the work unit scheduler 240 calculates a pairing score F ( p ) for each of the computing hosts 200P that can be used with respect to the work unit 300. The pairing score F ( p ) satisfies the following formula: F ( p )= w ₁ F _L ( p )+ w ₂ F _G ( p ).........(1)

w ₁ and w ₂ are preset weights. F _L ( p ) represents the reusability of the processing function. If the processing function is reusable with respect to the computing host 200P, it is suitable for assigning to the computing host 200P and giving a score; otherwise, if the processing function is relative to this operation The host 200P does not have reusability, and is not suitable for being assigned to the computing host 200P without giving a score.

F _G ( p ) represents the reusability of the image group. If the picture group is reusable with respect to the operation host 200P, it is suitable to be allocated to the operation host 200P, and the score is given; otherwise, if the picture group is not reusable with respect to the operation host 200P, it is not suitable for the operation. Host 200P, without giving a score.

Among them, the dependency of the processing function is of the highest importance. Therefore, in step 410, the work unit scheduler 240 determines whether the combination of the numbers i and k corresponding to the work unit 300 falls into a specific processor list. If so, it means that the processing function of the i-th video stream VS and the number L _k of the work unit 300 is completely dependent, and has been assigned to the computing host 200P of the specific host list. In step 420, the number p is set to one of a particular host list to cause the work unit 300 to be executed by the designated computing host 200P of the particular host list.

Furthermore, when there is no dependency on the processing function, the importance of the reusability of the processing function is higher than the reusability of the image group. Therefore, when w ₁ > _{w 2 is set} , it can be ensured that the processing host 200P can obtain a higher score when the processing function is reusable with respect to the computing host 200P.

For example, w ₁ can be set to 2 and w ₂ can be set to 1. Alternatively, w ₁ can be set to 3 and w ₂ can be set to 1.

In step 450, the work unit scheduler 240 selects the computing host 200P having the highest score of the pairing score F ( p ).

In step 460, the work unit scheduler 240 determines if the work unit 300 is fully dependent. If the work unit 300 is fully dependent, then step 470 is entered; if the work unit 300 is not fully dependent, the process ends.

In step 470, the selected computing host 200P is logged to a particular host list. In this way, other groups of pictures for the processing function of the i-th video stream VS and the number L _k will be directly assigned to the computing host 200P of the specific host list directly in steps 410 and 420.

In step 118 of FIG. 1, the work unit processor 250 processes the picture group in accordance with the work unit 300 in a corresponding processing function. This step can be done in several sub-steps. Please refer to FIG. 5, which shows a detailed flow chart of step 118 of FIG. 1.

In step 510, the work unit processor 250 determines whether the picture group exists in the cache. If the group of pictures has been stored in the cache memory, proceed to step 512; if the group of pictures is not stored in the cache, proceed to step 522.

In step 512, the group of pictures is read from the cache.

In step 514, if the processing function is not stored in the cache memory, the processing function is transferred to the cache memory of the computing host 200P.

In step 522, if the processing function is not stored in the cache memory, Then, the processing function is transferred to the cache memory of the computing host 200P.

In step 524, the picture group is transferred to the cache memory of the computing host 200P.

In step 526, the group of pictures is read from the cache.

In step 530, the decoded picture group is a number of pictures.

In step 540, the group of pictures is processed in accordance with the processing function of the unit of work 300.

Due to the versatility of the processing functions, the processing results are, for example, uncompressed video data, recompressed video data, key pictures containing specific objects, or event markers. The result processing unit 260 can further store the processing result to a file, display it on a screen, reply to a remote device, or store it in a database for access.

The distributed processing method and processing system 200 of the video stream VS of this embodiment can perform dynamic processing. For example, when the number of video streams VS increases, the video cutter 210 can cut the new video stream VS, and the work unit generator 220 can generate a new work unit 300 according to the new group of pictures. In another case, when the computing host 200P is incremented, the work unit scheduler 240 can re-pair the computing host 200P with the work unit 300.

Please refer to FIG. 6 , which is a schematic diagram showing the application of the distributed processing method of the video stream VS to the dynamic change situation. The above table is the time-varying curve C11~C14 of the computational load of the computing host 200P, and the table below is the time-varying curve R11 of the residual workload. The time-varying curves C11 to C14 of the calculation load sequentially indicate the calculation loads of the first to fourth arithmetic masters 200P, respectively. In stage A11, one video stream is processed by two computing hosts 200P. VS. In phase A12, two more video streams VS are added, and three video streams VS are added. In phase A13, two more computing hosts 200P are further added to the four computing hosts 200P. In Figure 6, all of the video streams VS use processing functions are not dependent.

In phase A11, one video stream VS is processed by two computing hosts 200P. At this time, the first computing host 200P and the second computing host 200P alternately process the video stream VS. Therefore, the time-varying curves C11 and C12 of the calculation load are alternately high and low. And the time-varying curve R11 of the residual workload is maintained at a low level.

In phase A12, the three video streams VS are processed by two computing hosts 200P. At this time, the first to second computing units 200P continuously process the three video streams VS without interruption. Therefore, the time-varying curves C11 and C12 of the calculation load are all full. And the time-varying curve R11 of the residual workload gradually increases.

In phase A13, the three video streams VS are processed by four computing hosts 200P. At this time, the first to fourth computing hosts process three video streams VS in turn. Therefore, the time-varying curves C11, C12, C13, and C14 of the calculation load are alternately high and low. And the time-varying curve R11 of the residual workload gradually decreases.

The processing functions used by the three video streams VS in FIG. 6 have no dependency, so the three video streams VS can be arbitrarily allocated to each of the computing hosts 200P. In another implementation, when the processing function used by one of the video streams VS has dependencies, then it needs to be scheduled to the particular computing host 200P. Please refer to FIG. 7 , which illustrates a partial dispersion processing method using video processing VS with a dependency processing function applied to dynamic changes. Schematic diagram of the situation. The above table is the time-varying curve C21~C24 of the computing load of the computing host 200P, and the following table is the time-varying curve R21 of the residual workload without dependency and the residual workload with dependency. Time-varying curve R22. The time-varying curves C21 to C24 of the calculation load sequentially indicate the calculation loads of the first to fourth arithmetic masters 200P, respectively.

In phase A21, one video stream VS having no dependency processing function is processed by two computing hosts 200P. In the phase A22, one video stream VS with a dependency processing function and one video stream without a dependency processing function are further added to the three video streams VS. In phase A23, two more computing hosts 200P are further added to the four computing hosts 200P.

In phase A21, one video stream VS having no dependency processing function is processed by two computing hosts 200P. At this time, the first computing host 200P and the second computing host 200P alternately process the video stream VS having no dependency processing function. Therefore, the time-varying curves C21 and C22 of the calculation load are alternately high and low. And the time-varying curve R21 of the residual workload without dependence is maintained at a low level.

In stage A22, one video stream VS with a dependency processing function is fixed by the first computing host 200P, and the two video streams VS without the dependency processing function are from the second computing host 200P. Process it. Therefore, the time-varying curve C21 of the computing load is at 60-80% level, and the time-varying curve C22 of the computing load is full. And the time-varying curve R21 of the residual workload without dependence is gradually increased, and the time-varying curve R22 of the residual workload with dependence is maintained at a low level.

In stage A23, a video stream VS with a dependency processing function VS The processing is performed by the first computing host 200P, and the two video streams VS without the dependency processing function are processed in turn by the second to fourth computing hosts. Therefore, the time-varying curve C21 of the calculation load is located at 60 to 80%, and the time-varying curves C22, C23, and C24 of the calculation load are alternately high and low. And the time-varying curve R21 of the residual workload without dependence is gradually decreased, and the time-varying curve R22 of the residual workload with dependence is maintained at a low level.

In summary, although the present invention has been disclosed above by way of example, it is not intended to limit the present invention. Those who have ordinary knowledge in the technical field of the present invention can make various changes and refinements without departing from the spirit and scope of the present case. Therefore, the scope of protection of this case is subject to the definition of the scope of the patent application attached.

102, 104, 106, 108, 110, 118, 120, 410, 420, 430, 440, 450, 460, 470, 510, 512, 514, 530, 540, 522, 524, 526 ‧ ‧ process steps

200‧‧‧Distributed processing system for video streaming

200C‧‧‧Control host

200P‧‧‧ computing host

210‧‧‧Video Cutter

220‧‧‧Workcell generator

230‧‧‧Database

240‧‧‧Working unit scheduler

250‧‧‧Working unit processor

260‧‧‧ result processor

300‧‧‧Working unit

310‧‧‧Video Library

312‧‧‧Photo group record

320‧‧‧Processing library

322‧‧‧ Processing function record

A11, A12, A13, A21, A22, A23‧‧

Time-varying curves of C11, C12, C13, C14, C21, C22, C23, C24‧‧‧ computing loads

CS‧‧‧Control process

D _i,j , D _{i,j +1} , D _{i,j +2} ‧‧‧Number of the picture group

L _k , L _{k +1} , L _{k +2} ‧‧‧ processing function number

PS‧‧‧Operation processing

Time-varying curve of R11, R21, R22‧‧‧ residual workload

RS‧‧‧ computing request

T _i,j ‧‧‧Working unit number

VS‧‧‧ video streaming

FIG. 1 is a flow chart showing a method for distributing video stream.

Figure 2 is a schematic diagram showing a distributed processing system for video streaming.

Figure 3 is a schematic diagram showing the working unit.

Figure 4 is a flow chart showing the details of step 106 of Figure 1.

Figure 5 is a flow chart showing the details of step 118 of Figure 1.

FIG. 6 is a schematic diagram showing the application of the distributed processing method of the video stream to the dynamic change situation.

FIG. 7 is a schematic diagram showing a method for applying a distributed processing method of a video stream having a dependency processing function to a dynamic change.

102, 104, 106, 108, 110, 118, 120‧‧‧ process steps

CS‧‧‧Control process

PS‧‧‧Operation processing

Claims (33)

A method for processing a video stream, comprising: cutting a video stream to generate at least one picture group; generating a work unit, the work unit recording the picture group and a corresponding one of the processing functions; determining the working unit Corresponding to one of the computing hosts, wherein the action determines the computing host according to at least the dependency of the processing function; and transmits the working unit to the corresponding computing host. The method for processing a video stream as described in claim 1, further comprising: processing the group of pictures according to the processing function. The method for dispersing a video stream according to the second aspect of the invention, wherein before the act of processing the group of pictures according to the processing function, the method for distributing the video stream further comprises: decoding the group of pictures to at least a picture. The method for distributing video stream according to claim 1, wherein in determining the operation of the computing host, the computing host is further determined according to the reusability of the processing function. The method for distributing video stream according to claim 1, wherein in determining the operation of the computing host, the computing host is further determined according to the reusability of the group of pictures. The method for processing a video stream as described in claim 1, wherein in determining the operation of the computing host, calculating a pairing score F ( p ) of the computing host relative to the working unit, the pairing score F ( p ) satisfies the following formula: F ( p )= w ₁ F _L ( p )+ w ₂ F _G ( p ), w ₁ and w ₂ are preset weights; The method for processing a video stream as described in claim 6, wherein w ₁ > w ₂ . The method for distributing video stream according to claim 6, wherein in determining the operation of the computing host, the computing host is further determined according to the pairing score F ( p ). The method for dispersing a video stream as described in claim 1, wherein the method for distributing the video stream further comprises: transmitting the group of pictures to the The computing host. The method for dispersing a video stream as described in claim 1, wherein the method for distributing the video stream further comprises: transmitting the processing function to the operation of transmitting the working unit to the corresponding computing host The computing host. The method for distributing video stream according to claim 1, further comprising: if the working unit is completely dependent, recording the selected computing host to a specific host list. The method for dispersing a video stream as described in claim 1 further includes: storing the processing result of the computing host to a file and displaying it in a firefly Screen, reply to a remote device or stored in a database. A video stream distributed processing system includes: a control host, comprising: a video cutter, the video cutter cutting a video stream to generate at least one picture group; a work unit generator, the work unit generator Generating a work unit, the work unit records the picture group and a corresponding one of the processing functions; and a work unit scheduler; and at least one operation host, the work unit corresponding to one of the operation units, the work unit row The program determines the computing host according to the dependency of the processing function. The distributed processing system for video streaming according to claim 13 , wherein the computing host comprises: a working unit processor, and the working unit processor processes the group of pictures according to the processing function. The distributed processing system for video streaming according to claim 14, wherein the work unit processor further decodes the picture group into a plurality of pictures. The distributed processing system for video streaming according to claim 13 , wherein the work unit scheduler further determines the computing host according to the reusability of the processing function. The distributed processing system for video streaming according to claim 13 , wherein the work unit scheduler further determines the computing host according to the reusability of the image group. The distributed processing system for video streaming according to claim 13, wherein the work unit scheduler further calculates a pairing score of the computing host relative to the working unit, and the pairing score satisfies the following calculation formula: F ( p )= w ₁ F _L ( p )+ w ₂ F _G ( p ), w ₁ and w ₂ are preset weights; The video streaming processing system of claim 18, wherein w ₁ > w ₂ . The distributed processing system for video streaming according to claim 18, wherein the work unit scheduler further determines the computing host according to the pairing score F ( p ). The distributed processing system for video streaming according to claim 13, wherein the computing host further receives the group of pictures. The distributed processing system for video streaming according to claim 13, wherein the computing host further receives the processing function. The distributed processing system for video streaming according to claim 13 , wherein the computing host further comprises: a result processor, wherein the result processor displays or stores the processing result of the group of pictures. A control host for video streaming, comprising: a video cutter, the video cutter cutting a video stream to generate at least one picture group; a work unit generator, the work unit generator generating a work unit, the work unit recording the picture group and a corresponding one of the processing functions; and a work unit scheduler, the work unit scheduler according to the processing The dependence of the equation determines one of the computing hosts corresponding to the unit of work. The control host of the video stream as described in claim 24, wherein the work unit scheduler further determines the computing host according to the reusability of the processing function. The control host of the video stream as described in claim 24, wherein the work unit scheduler further determines the computing host according to the reusability of the picture group. The control host of the video stream according to claim 24, wherein the work unit scheduler further calculates a pairing score of the computing host relative to the working unit, and the pairing score satisfies the following calculation formula: F ( p ) = w ₁ F _L ( p )+ w ₂ F _G ( p ), w ₁ and w ₂ are preset weights; The control host of the video stream as described in claim 27, wherein w ₁ > w ₂ . The control host of the video stream as described in claim 27, wherein the work unit scheduler further determines the computing host according to the pairing score F ( p ). A video stream computing host coupled to a working unit row a video stream is cut into at least one picture group, and a work unit records the picture group and a corresponding one of the processing functions, the work unit corresponding to the operation host, and the work unit scheduler according to the processing function The dependency of the equation determines the computing host, the computing host includes: a work unit processor, the work unit processor processes the image group according to the processing function; and a result processor, the result processor displays or stores the image The result of the group processing. The computing host of the video stream as described in claim 30, wherein the computing host further receives the group of pictures. The computing host of the video stream as described in claim 30, wherein the computing host further receives the processing function. The computing host of the video stream as described in claim 30, wherein the work unit processor further decodes the picture group into a plurality of pictures.