[go: up one dir, main page]

HK1186327B - A method for processing error correction requests, a video server and a video conference system - Google Patents

A method for processing error correction requests, a video server and a video conference system Download PDF

Info

Publication number
HK1186327B
HK1186327B HK13113553.7A HK13113553A HK1186327B HK 1186327 B HK1186327 B HK 1186327B HK 13113553 A HK13113553 A HK 13113553A HK 1186327 B HK1186327 B HK 1186327B
Authority
HK
Hong Kong
Prior art keywords
video
threshold
weight value
client
error correction
Prior art date
Application number
HK13113553.7A
Other languages
Chinese (zh)
Other versions
HK1186327A1 (en
Inventor
燕必东
Original Assignee
Polycom通讯技术(北京)有限公司
Filing date
Publication date
Priority claimed from CN201110402969.8A external-priority patent/CN103152545B/en
Application filed by Polycom通讯技术(北京)有限公司 filed Critical Polycom通讯技术(北京)有限公司
Publication of HK1186327A1 publication Critical patent/HK1186327A1/en
Publication of HK1186327B publication Critical patent/HK1186327B/en

Links

Description

Method for processing error correction request, video server and video conference system
Technical Field
The present invention relates to the field of video communication, and in particular to a method for communicating in a video conference system comprising a video server and a video client, and to a corresponding video server and video conference system.
Background
Video conferencing enables parties remote from each other to conduct a face-to-face conference using audio and video signals. A video conferencing system may include one or more video conferencing endpoints and a video server. Participants of a video conference interact with participants at other locations through video conference endpoints. A video conference endpoint may be a video terminal on a network capable of real-time, bi-directional audio/video/data communication with other video terminals and/or video servers. Of course, a video conference endpoint may provide voice only, voice and video, or voice, data and video communications, among others. Video conferencing endpoints typically include a display unit that can display video images from one or more remote sites and speakers that play audio signals from the remote sites. In operation, a videoconferencing endpoint transmits audio, video and/or data from a local site to one or more remote sites, and displays the video and/or data received from the one or more remote sites on its display unit and outputs the audio received from the one or more remote sites on its speakers. Various video conference endpoints exist in the market at present, for example, the video conference endpoints may comprise POLYCOM VSX and HDX series of endpoint products obtained from Polycom, Inc.
Video conferences are sometimes managed by a video server, which may be, for example, a Multipoint Control Unit (MCU). The MCU may be a conference control entity that may be located in a network node, a terminal, or any other location. The MCU may receive and process several media channels from access ports according to certain criteria and distribute signals to connected channels through other ports. In some cases, the MCU may also be considered an endpoint. Some MCUs include two logic units: a Media Controller (MC) and a Media Processor (MP). There are various MCUs on the market today, for example, MCUs may comprise the RMX series of products available from Polycom, inc. More detailed definitions of video endpoints and MCUs can be found in international telecommunications union ("ITU") standards such as, but not limited to, the h.320, h.324, and h.323 standards.
In the existing video conference, because the video endpoints are at different positions, the network communication conditions between the video endpoints and a video server (MCU) may also be different, the communication conditions between some video endpoints and the MCU are good, and the communication conditions between other video endpoints and the MCU are poor and packet loss may occur during transmission. If unfortunately a video endpoint with poor communication conditions loses critical packets, e.g. packets that cannot be recovered by other received packets, or packets that are necessary for displaying the video, e.g. packets that constitute I-frames in a GOF (group of frames), the video endpoint may send an error correction request, such as a request to retransmit the lost packets (or to retransmit I-frames), to the MCU. The MCU will typically process the error correction request and retransmit the lost packets or I-frames, etc. so that the video endpoint can continue the normal conference.
Due to the real-time nature of video conferencing, it is often desirable to try to keep consistent among all the video endpoints participating in the video conference. Thus, if one video endpoint needs to have error correction, the MCU may temporarily stop the video display of the other video endpoints, or send a command to the other video endpoints (e.g., retransmit I-frames to all video endpoints) to synchronize the conference progress on these video endpoints. However, if the network condition between one video endpoint and the MCU is poor and the error correction request is sent more frequently, the conference process of other video endpoints participating in the video conference is frequently and suddenly stopped according to the existing processing method, so that conference participants at these other video endpoints feel uncomfortable. In addition, the MCU consumes too much resources to process these error correction requests, thereby reducing the efficiency of the MCU.
Therefore, there is a need for a new technique for handling error correction requests from video endpoints in a video conferencing system that can balance the satisfaction of all video conference participants to the conference and at the same time can improve the efficiency of the MCU.
Disclosure of Invention
In view of the above, there is disclosed herein a method of handling error correction requests from a video client in a video conferencing system, a video server and a video conferencing system that overcome or at least partially solve or mitigate the above-mentioned problems.
According to an aspect of the present invention, there is provided a method of processing an error correction request from a video client in a video conference system including a video server and at least one video client, comprising: assigning a weight to each of the at least one video client, the weight value being determined based on the communication conditions of the respective video client and the video server; determining one or more thresholds according to the number of video clients and the assigned weight value; and receiving an error correction request from the video client, and processing the error correction request from the video client according to the weight value assigned to the video client and the determined one or more thresholds.
Optionally, in the method of processing an error correction request, a plurality of response time periods are determined, each response time period corresponding to one of a range of thresholds divided by one or more thresholds. Thus, processing the error correction request from the video client further comprises: determining a threshold range in which the video client is located by comparing the weight value assigned to the video client with the determined one or more thresholds, and processing the error correction request from the video client according to a response time period corresponding to the determined threshold range.
The method of processing error correction requests from video clients may process error correction requests for video clients with different network conditions at different response times. Therefore, some video clients in poor network conditions frequently send error correction requests and can process the requests at longer response time intervals, so that interference to other video clients is avoided, and the overall satisfaction of conference participants on a video conference system is improved. In addition, by processing the error correction requests at different response time intervals, the load of the video server for processing the error correction requests is reduced, and the operating efficiency of the video server is improved.
Optionally, when assigning the weight to each video client, the weight value may also be determined based on a time duration for each video client to join the video conference system, and when determining the threshold, the time duration for each video client to join the video conference system may also be further considered. Because the network link is still unstable and/or the newly added conference participant is worth waiting by other conference participants when the video client just joins the video conference system, the video client just joining the video conference system can be assigned with more important weight, so that the error correction request of the video client can be processed in a shorter time, and the satisfaction degree of the conference participants on the video conference system can be further improved.
Optionally, the determined threshold comprises a first threshold corresponding to good communication conditions and a second threshold corresponding to poor communication conditions. The determined response time period includes a first response time period, a second response time period, and a third response time period divided by a first threshold and a second threshold. The first response period may be shorter than the second response period, while the third response period may be very long or infinitely long. Alternatively, the length of the third response period may be twice the length of the second response period. Thus, the frequent error correction requests from the video client with poor communication condition can be processed without processing, thereby avoiding the video client from seriously affecting other conference participants, and ensuring the satisfaction degree of the video conference as a whole.
Optionally, the communication condition of the corresponding client and the video server is reflected by a packet loss rate in the communication process of the corresponding client and the video server. The step of assigning a weight to each video client further comprises: distributing a first weight value to a video client which is added into the video conference system and has the time length less than the preset time; distributing a second weight value smaller than the first weight value to the video client side which does not lose packets in the communication process with the video server; and assigning a weight value which is dependent on the packet loss rate and a third weight value smaller than the second weight value to the video client which has the packet loss during the communication process, wherein the assigned weight value is decreased along with the increase of the packet loss rate. And determining the first threshold such that the first weight value is greater than the first threshold; and determining the second threshold such that the second weight value is greater than the second threshold. Therefore, the weight of each video client can be determined according to the packet loss rate during the communication between the video client and the video service and the time length of the video client joining the video conference, the error correction request of the video client which just joins the video conference can be ensured to be processed quickly, and the error correction request of the video client which does not generally have the packet loss can also be requested quickly.
Alternatively, the steps of assigning a weight value to each video client and determining one or more thresholds may be repeated at predetermined time intervals. In this way, the error correction request processing policy can be updated periodically according to the network condition of the video client.
Alternatively, the error correction request may be a retransmit I-frame request.
There may also be provided a video server comprising: a communication interface configured to communicate with one or more video clients to compose a video conference; an audio and video processor configured to process audio and video signals from video clients in a video conference and forward the audio and video signals between the video clients; a controller configured to manage the video conference and further configured to process an error correction request from the video client. The controller further comprises a weight assignment component adapted to assign a weight to each video client, wherein the magnitude of the weight value is determined based on the communication conditions of the respective client and the video server. The controller further comprises a threshold determination component adapted to determine one or more thresholds depending on the number of video clients and the assigned weight value; and an error correction request processing section adapted to receive an error correction request from the video client, process the error correction request from the video client by a weight value assigned to the video client and the determined one or more thresholds.
A video conferencing system may also be provided comprising a video server according to the invention and one or more video clients. The video clients send and receive audio and/or video signals to each other via the video server, and the video clients send an I-frame request to the video server for processing by the video server when failing to correctly acquire I-frame data.
Drawings
Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the invention. Also, like reference numerals are used to refer to like parts throughout the drawings. In the drawings:
FIG. 1 shows a block diagram of a video conferencing system according to an embodiment of the invention;
FIG. 2 shows a block diagram of a video server according to one embodiment of the invention;
FIG. 3 shows a block diagram of a controller in a video server according to an embodiment of the invention; and
fig. 4 shows a flow diagram of a method for processing an error correction request from a video client according to one embodiment of the invention.
Detailed Description
The invention is further described with reference to the following figures and detailed description of embodiments.
Fig. 1 shows a block diagram of a video conferencing system 100 according to an embodiment of the invention. The video conferencing system 100 includes a video server (MCU) 120 and one or more video Endpoints (EPs) 130A-130N, which are linked to each other via a network 110.
Network 110 may be a single network or a combination of two or more networks and may be any type of network including a packet switched network, a circuit switched network, an Integrated Services Digital Network (ISDN), a PSTN, an Asynchronous Transfer Mode (ATM), the internet, a local area network, and any other type of network over which data may be transferred.
In general, the video conferencing system 100 may conduct multimedia communications between the video endpoints 130A-E and the video server 120 via the network 110 according to various multimedia communication protocols. These multimedia communication protocols may be any communication protocols such as h.320, h.324, h.323 and STP.
Video endpoints 130 may be any entity on a network capable of real-time, bi-directional audio/video/data communication with other video endpoints 130 and/or video servers 120. For example, video endpoint 130 may be implemented as a computer, a PDA (personal digital assistant), a cellular telephone, a television with a microphone, a camera, and so on. The video endpoints may have a microphone and speaker that allow users on the video endpoints to interact with each other by voice during the conference, and a display and camera that allow users to interact with each other by video during the conference. Video endpoint 130 may provide voice, data, video, signaling, control, or a combination of these signals.
The video server (MCU) 120 may be adapted to manage a video conference, e.g., when a video conference is established between one or more video endpoints 130A-130E, the video endpoints 130 may transmit and receive audio and/or video signals via the MCU 120, and may also transmit signaling, control information, and data signals.
In the video conferencing system 100, the communication quality of the network links between the various video endpoints 130A-130E and the video server 120 may vary because they are in different locations. For example, as shown in fig. 1, the link between video endpoint 130A and video server 120 has a better communication quality, while the link between video endpoint 130E and video server 120 has a poorer communication quality. A packet loss is likely to occur in the communication between video endpoint 130E and video server 120, thereby preventing video endpoint 130E from properly videoconferencing. In some embodiments, video endpoint 130E may not be able to properly retrieve I frame data to decode and render the conference video due to the loss of packets that make up the I frames in the GOF (group of video frames). At this point, video endpoint 130E may send an error correction request, such as a request to resend an I-frame, to video server 120 for processing by video server 120.
It should be noted that the above description of the video conference system 100 only includes elements that are part of the embodiments of the present invention, and that the video conference system 100 may also include other non-described parts depending on the arrangement and application requirements without departing from the scope of the appended claims.
Fig. 2 shows a block diagram of the video server 120. The video server 120 may be, for example, a Multipoint Control Unit (MCU). The video server 120 may include a communication interface 210, an audio processor 220, a video processor 230, and a controller 240. It should be noted that only the components associated with the embodiments presented herein are shown in fig. 2. The video server 120 may also include other components and the audio processor 220 and the video processor 230 may be combined into an audio video processor, all within the scope of the present invention.
The communication interface 210 may receive communications from multiple video endpoints 130A-F over the network 110. Communication interface 210 may communicate in accordance with various communication protocols including h.320, h.321, h.323, h.324, and SIP protocols. Further, communication interface 210 may handle communications in accordance with various compression standards, including compression standards such as H.261, H.263, H.264, G.711, G.722, and MPEG. The communication interface 210 may receive control and data information from and transmit control and data information to other video servers and video endpoints to communicate with one or more video endpoints to form a video conferencing system. Communication interface 210 may multiplex or de-multiplex various signals and/or "signaling and control" transmitted between video endpoints 130A-F and video server 120. For example, the communication interface 210 may send compressed audio signals to the audio processor 220 and receive compressed audio signals from the audio processor 220. The communication interface 210 may transmit a compressed video signal to the video processor 230 and receive a compressed video signal from the video processor 230. Communication interface 210 may send "control and signaling" to controller 240 and receive "control and signaling" from controller 240.
The audio processor 220 may receive compressed audio streams from the plurality of video endpoints 130A-F via the communication interface 210 and over the audio communication 222. The audio processor 220 may decode the compressed audio streams, analyze the decoded streams, select certain streams, and mix the selected streams. The mixed streams may be compressed and sent back to the communication interface 210 and then sent by the communication interface 210 to the different video endpoints 130A-F. The audio streams sent to different video endpoints may be different. For example, the audio stream may be formatted according to different communication standards and the needs of the video endpoints.
The video processor 230 may receive compressed video streams sent from the plurality of video endpoints 130A-F to the video server 120 and received and processed through the communication interface 210. In one embodiment, video processor 230 may include one or more input modules 232A-X, one or more output modules 234A-X, and a video common interface 236. The input modules 232A-X may process compressed input video streams from one or more video endpoints 130A-F. The compressed output video stream may be composed of several input streams to form a video stream for a conference presented at a specified video endpoint. Optionally, each input module 232A-X includes a decoder 2322 that decodes the compressed input video stream. Output modules 234A-X may generate compressed video outputs for video endpoints 130A-F. Optionally, each output module 234A-X includes an encoder 2342 for encoding the output video stream to generate a conference video presentation specific to the video endpoints 130A-F. In one embodiment, one output module 234 may serve multiple video endpoints, and may even serve all video endpoints participating in a conference. The output modules 234 receive video data from the respective input modules 232 via the common interface 236. Common interface 236 may comprise any suitable type of interface, including a Time Division Multiplexed (TDM) interface, a packet-based interface, and/or a shared memory, among others. The data on the common interface 236 may be fully decompressed or partially decompressed.
It should be noted that the above description of the audio processor 220 and the video processor 230 is merely exemplary and not limiting. The present invention is not limited by the specific structure of audio processor 220 and video processor 230, and all audio processors and video processors configured to process audio and video signals from video clients in the video conference and forward the audio and video signals between video clients are within the scope of the present invention.
The controller 240 may control the operation of the video server 120 and the operation of its internal components, including but not limited to the audio processor 220, video processing 230, etc. Conventional control functions performed by controller 240 are not described in detail herein, and details of the various controllers 230 are disclosed in the chinese patent application No. 201110077748.8, incorporated herein by reference.
The components involved in the controller 240 processing error correction requests from video endpoints are described in detail below with reference to fig. 3. As described above, the error correction request refers to a request sent by the video endpoint to the video server to solve a conference failure when the video endpoint cannot present the video conference content due to an error or a communication problem in the course of the video conference. Since failures during a video conference are typically caused by lost data packets during the communication, the error correction request typically includes a request to the video server to retransmit the lost data packets. A video endpoint may not be able to correctly fetch I frame data to decode and render a conference video stream due to the loss of packets that make up the I frames in a GOF (group of video frames). At this point, video endpoint 130E sends a request, such as a resend I-frame, to video server 120 for processing by video server 120, and in particular controller 240 in video server 120, and resends the I-frame. As shown in fig. 3, the controller 240 includes a weight assignment section 310, a threshold value determination section 320, a response time period determination section 330, and an error correction request processing section 340.
The weight assignment component 310 assigns a weight to each video endpoint participating in the video conference. The weight value of each video endpoint determines the priority level at which the error correction request of the video endpoint is processed, and therefore, the size of the weight value is generally determined according to the communication condition between the video endpoint and the video server. For video endpoints with better communication conditions, a weight value may be assigned to them that causes the error correction process to be processed faster, and vice versa.
There may be multiple indicators reflecting the communication conditions between the video endpoint and the video server, such as the average data transfer rate between the video endpoint and the video server. The communication condition can be reflected by the packet loss rate in the communication process of the video endpoint and the video server. If no packet is lost in the communication process, the communication condition is very good, and the communication condition is worse when the packet loss rate is higher. No particular indicator is required and combinations of indicators may be used, all of which are within the scope of the invention.
In addition, according to an embodiment of the present invention, the weight value assignment unit 310 may also need to consider a duration of time for each video endpoint to join the video conference when assigning a weight to each video endpoint. Since the interaction between a participant at that location and other conference participants may be important when a video endpoint has just joined the video conference, it may be desirable to assign a better weight to the video endpoint that has just joined the video conference so that the video endpoint's error correction request can be processed as soon as possible. There are a number of ways to define such just-joined video endpoints. For example, a video endpoint that has joined a video conference for less than 5 seconds may be determined as the video endpoint that has just joined the conference. Of course, other time periods may be defined, such as 10 seconds, 15 seconds, etc., all of which are within the scope of the present invention.
Threshold determination component 320 can be used to determine one or more thresholds. These thresholds may be used to categorize video endpoints. Video endpoints in the same threshold range interval can be classified by comparing their weight values to the thresholds determined by threshold determination component 320. There are many ways to determine the threshold, and the principle of threshold determination is that video endpoints can be explicitly categorized. For example, video endpoints with very good communication conditions may be classified into one type (e.g., video endpoints that do not lose packets during communication), video endpoints with good communication conditions may be classified into one type (e.g., video endpoints with packet loss rate lower than 5%), and video endpoints with poor communication conditions may be classified into one type (e.g., video endpoints with packet loss rate greater than 30%).
In addition, if weight assignment section 310 also takes into account that the length of time each video endpoint joined the video conference is assigning a weight to each video endpoint, threshold determination section 320 may also desire to determine a threshold that facilitates handling of grouping video endpoints that have just joined the video conference into a category when determining the threshold.
It should be appreciated that based on the above description, a variety of ways for determining the threshold value can be obtained and employed in threshold value determining component 320. All threshold determination ways of determining the threshold based on the weight assigned to each video endpoint and categorizing the video endpoints participating in the conference according to the threshold are within the scope of the present invention.
The response time period determining section 330 may determine a plurality of response time periods each corresponding to a threshold range each divided by the threshold determined by the threshold determining section 320. For example, if the threshold value determining section 320 determines N threshold values of an integer number, the threshold value range divided by the N threshold values is N +1, and therefore the number of response time periods determined by the response time period determining section 330 should also be N +1 corresponding to the number of threshold value ranges. As described above, the threshold determined by threshold determination component 320 may be suitable for categorizing video endpoints in a video conference, and thus, the response time period corresponding to each threshold range may also be associated with the categorized video endpoint. The response time period corresponding to a video endpoint whose error correction request needs to be processed as soon as possible for the category should be short, and vice versa.
When the video server 120 receives an error correction request from a video endpoint, the network interface 210 may send the request to the controller 240 for processing, and the controller 240 may process the request by the error correction request processing component 340. Error correction request processing section 340 receives the weight that can be obtained from weight redistribution section 310 to be assigned to the video client that sent the error correction request when the error correction request is received. Then, a threshold range in which the weight value is located may be determined by comparing the acquired weight value with one or more thresholds determined by the threshold determination section 320. Then, a response time period corresponding to the determined threshold range may be acquired from the response time period determining section 330. Finally, the error correction request for the video endpoint may be processed according to the determined response time period. E.g., receiving multiple error correction requests from the video endpoint within the determined response time period, only the first error correction request may be processed. Alternatively, after processing an error correction request from the video endpoint, the error correction request may be ignored if a subsequent error correction request from the same video endpoint is received again within the determined response time period.
The controller 340 may assign a weight to each video endpoint in the conference and may process error correction requests from the video endpoints at different response times depending on the magnitude of the weight values. Thus, some error correction requests that need to be processed as soon as possible (e.g., video endpoints that have just joined the conference) and that are occasionally erroneous due to accidents (e.g., video endpoints with good communication conditions) may be processed as soon as possible. And the error correction requests of the video endpoints which frequently make errors due to poor communication conditions are ignored, so that other video conference participants do not feel uncomfortable due to the processing of the error correction requests by the video server. According to the processing mode, all participants in the video conference can be considered, so that the overall satisfaction degree of the video conference is improved. In addition, the video server does not process each error correction request any more, so the load of the video server for processing the error correction request can be greatly reduced.
As the communication conditions between the video endpoints and the video server may change over time. Thus, alternatively, the weight assignment section 310 and the threshold determination section 320 may periodically check the communication condition between the video endpoints and the video server, and update the weight assigned to each video endpoint and the threshold size determined thereby accordingly.
In general, the better the communication between a video endpoint and a video server, the higher the weight value assigned to the video endpoint, and the better the communication between the video endpoint and the video server, the lower the weight value assigned to the video endpoint. The present invention is described by taking this as an example, and optionally, an opposite weight value assignment manner, that is, the better the communication condition between the video endpoint and the video server is, the lower the assigned weight value for the video endpoint is, may also be adopted when assigning a weight to each video endpoint.
In some applications, it may not be necessary to classify video endpoints into more categories in view of the number of video endpoints participating in the video conference, and therefore, threshold determining component 320 may determine only two thresholds, a first threshold corresponding to good communication conditions and a second threshold corresponding to poor communication conditions. The first threshold is higher than the second threshold.
The response time period determination section 330 may also determine three response time periods, a first response time period corresponding to a threshold range determined by a first threshold value and higher values, a second response time period corresponding to a threshold range determined by the first threshold value and a second threshold value, and a third response time period corresponding to a threshold range determined by the second threshold value and lower values. The first response period is shorter than the second response period, and the second response period is shorter than the third response period.
For an error correction request sent from a video endpoint with poor communication conditions (i.e., a video endpoint with a weight value less than the second threshold), the video server may process the request for a considerable amount of time, i.e., the length of the third response period is very long. Optionally, the length of the third response period is twice the length of the second response period.
The video server according to an embodiment of the present invention is further described below by taking an example in which the communication status of the video endpoint and the video server is reflected by the packet loss rate in the communication process between the video endpoint and the video server.
The weight value assigning section 310 may assign a first weight value to a video endpoint that has just joined the video conference. For example, the first weight value may be 500000. The weight value assigning section 310 may also assign a second weight value to a video endpoint that does not lose packets during communication with the video server. The second weight value may be generally smaller than the first weight value, for example, the second weight value may be 1000.
For a video endpoint where packet loss occurs during communication with the video server, the weight value assigning section 310 assigns a weight value depending on the packet loss rate and the third weight value. The third weight value may be smaller than the second weight value, and the weight value assigned to each video endpoint decreases as the packet loss rate increases. For example, the third weight value W3 is set to 100. And the weight values assigned to each video endpoint are shown in table 1 below:
table 1
Packet loss ratio (%) Weighted value
≤1% W3×1.5
≤3% W3
≤5% W3×0.8
≤8% W3×0.5
≤10% W3×0.5
≤15% W3×0.3
≤20% W3×0.2
≤30% W3×0.15
≤50% W3×0.10
>50% W3×0.05
At video endpoints that employ LPR (packet loss recovery) techniques, video endpoints may themselves recover some of the lost packets, and therefore are assigned higher weight values than video endpoints that do not employ LPR. The weight values assigned to each video endpoint employing LPR are shown in table 2.
Table 2
Packet loss ratio (%) Weighted value
≤1% W3×2.5
≤3% W3×2
≤5% W3×1.5
≤8% W3
≤10% W3
≤15% W3×0.8
≤20% W3×0.5
≤30% W3×0.5
≤50% W3×0.2
>50% W3×0.05
Technology relating to LPR is available in U.S. Pat. No. US7876685, the contents of which are incorporated herein by reference.
After the weight assignment section 310 assigns a weight to each video endpoint, the threshold determination section 320 may determine the first threshold H1 and the second threshold H2. According to one embodiment, the first threshold H1 and the second threshold H2 may be defined as follows:
a first threshold H1= (number of video endpoints × 0.4) × second weight; and
the second threshold H2= (number of video endpoints × 0.5) × the third weight.
The threshold value determining part 320 may define the first threshold value and the second threshold value in various ways as long as the first threshold value is determined such that the first weight value is greater than the first threshold value and the second threshold value is determined such that the second weight value is greater than the second threshold value. That is, the weight of the video endpoint that just joins the video conference should be greater than the first threshold, and the weight value of the video endpoint with good communication condition, that is, the video endpoint with the packet loss rate of 0, may be at least greater than the second threshold.
The first response period determined by the response period determining part 330 may be 3 seconds, the second response period may be 30 seconds, and the third response period may be 60 seconds. In this way, the error correction request processing section 340 may obtain, from the weight allocating section 310, a weight allocated to the video client sending the error correction request when receiving the error correction request from the video endpoint, may compare the weight value with the first threshold and the second threshold, and may process the error correction request according to the first response time period (i.e., 3 seconds) if the weight value is greater than the first threshold, that is, may process the error correction request if a previous error correction request processed from the same video endpoint is before the first response time period, and otherwise may ignore the request. If the weight value is between the first threshold and the second threshold, the error correction request may be processed according to a second response time period (i.e., 30 seconds). If the weight value is less than the second threshold, the error correction request may be processed according to a third response time period (i.e., 60 seconds).
Fig. 4 shows a flow diagram of a method 400 for processing an error correction request from the video client described above, according to one embodiment of the invention. The method may be adapted to be performed in the video server 120 described with reference to fig. 2, in particular in the controller 240 of the video server 120 shown in fig. 3.
As described above, the error correction request may refer to a request sent by the video endpoint to the video server to solve a conference failure when the video endpoint cannot present the video conference content due to an error itself or a communication problem during the video conference. The error correction request may include a request to the video server to retransmit the lost data packet. More specifically, the error correction request typically includes a request to retransmit an I-frame
The illustrated method 400 begins at step S410, where a weight is assigned to each video endpoint participating in the video conferencing system. The weight value of each video endpoint determines the priority level at which error correction requests for that video endpoint are processed. The weight value may be determined according to the communication condition between the video endpoint and the video server. For video endpoints with better communication conditions, a weight value may be assigned to them that causes the error correction process to be processed faster, and vice versa.
As described above, the communication condition between the video endpoint and the video server may be reflected by the packet loss rate during the communication between the video endpoint and the video server. It is to be understood that the present invention is not limited to specific criteria and, as noted above, various combinations of criteria may also be employed, all of which are within the scope of the present invention.
Further, optionally, in assigning a weight to each video endpoint in step S410, it may be desirable to consider the duration of time each video endpoint joined the video conference. Since the interaction between a participant at that location and other conference participants may be important when a video endpoint has just joined the video conference, it may be desirable to assign a better weight to the video endpoint that has just joined the video conference so that the video endpoint's error correction request can be processed as soon as possible. There are a number of ways to define such just-joined video endpoints. For example, a video endpoint that has joined a video conference for less than 5 seconds may be determined as the video endpoint that has just joined the conference. Of course, other time periods may be defined, such as 10 seconds, 15 seconds, etc., all of which are within the scope of the present invention. Step S410 may be generally performed in the weight value assigning section 310 of the controller 240.
Subsequently, in step S420, one or more thresholds are determined. These thresholds may be used to categorize video endpoints. Video endpoints whose weight values determined in step S410 are in the same threshold range interval divided by these thresholds are classified into one class. There are many ways to determine the threshold, and the principle of threshold determination is that video endpoints can be explicitly categorized. For example, video endpoints with very good communication conditions may be classified into one type (e.g., video endpoints that do not lose packets during communication), video endpoints with good communication conditions may be classified into one type (e.g., video endpoints with packet loss rate lower than 5%), and video endpoints with poor communication conditions may be classified into one type (e.g., video endpoints with packet loss rate greater than 30%).
In addition, if it is also considered in step S410 that the length of time that each video endpoint joins the video conference system is weighting each video endpoint, then in determining the threshold in step S420, it may be desirable to determine a threshold that facilitates the grouping of video endpoints that have just joined the video conference into one class.
It should be appreciated that based on the above description, it can be appreciated that a variety of ways for determining the threshold value can be employed in step S420. All threshold determination ways of determining the threshold based on the weight assigned to each video endpoint and categorizing the video endpoints participating in the conference according to the threshold are within the scope of the present invention. Step S420 may be performed in the threshold determination part 320 of the controller 340.
In step S430, a plurality of response time periods may be determined, each corresponding to a threshold range into which the threshold determined in step S420 is divided. For example, if N thresholds are determined in step S420, the threshold range divided by the N thresholds is N +1, and thus the number of response time periods determined in step S430 should also be N +1 corresponding to the number of threshold ranges. The response time period determined in step S430 may also be associated with the categorized video endpoint. The response time period corresponding to a video endpoint whose error correction request needs to be processed as soon as possible for the category should be short, and vice versa. Step S430 may be performed in the response period determination part 330 of the controller 240.
Subsequently, when an error correction request from a video endpoint is received in step S440, the weight assigned to the video client in step S410 is acquired in step S450, the acquired weight value is compared with the threshold determined in step S420 to determine a threshold range in which the weight value is located in step S460, and the response time corresponding to the threshold range determined in step S430 is acquired in step S470, and then the error correction request of the video endpoint may be processed according to the determined response time period in step S480. E.g., receiving multiple error correction requests from the video endpoint within the determined response time period, only the first error correction request may be processed. Or, after processing an error correction request from the video endpoint, if a subsequent error correction request from the same video endpoint is received again within the determined response time period, the error correction request is ignored.
After the error correction request for one video endpoint completes the processing of steps S440-S480 above, the method may proceed to step S490, where it waits for a new error correction request (which may be from the same video endpoint or a different video endpoint), and may repeat the processing of steps S440-S480 above for the new error correction request. Steps S440-S480 may be performed in the error correction request processing part 340 of the controller 240
According to the error correction request processing method 400 of the present invention, error correction requests from video endpoints can be processed with different response times by assigning a weight to each video endpoint in the conference system and according to the magnitude of the weight value. Thus, some error correction requests that need to be processed as soon as possible (e.g., video endpoints that have just joined the conference) and that are occasionally erroneous due to accidents (e.g., video endpoints with good communication conditions) may be processed as soon as possible. And the error correction requests of the video endpoints which frequently make errors due to poor communication conditions are ignored, so that other video conference participants do not feel uncomfortable due to the processing of the error correction requests by the video server. According to the processing mode, all participants in the video conference can be considered, so that the overall satisfaction degree of the video conference is improved.
Since the communication conditions between the video endpoints and the video server may change over time, optionally, in method S400, steps S410 and S420 are also repeated periodically to update the weight assigned to each video endpoint and the determined threshold size.
In general, the better the communication between a video endpoint and a video server, the higher the weight value assigned to the video endpoint may be, and the better the communication between the video endpoint and the video server, the lower the weight value assigned to the video endpoint may be. The present invention is described by taking this as an example, and of course, the opposite way of assigning the weight values, that is, the better the communication condition between the video endpoints and the video server is, the lower the weight value assigned to the video endpoint may be, and the lower the weight value assigned to each video endpoint may also be used when assigning the weight to each video endpoint.
In some applications, it is not necessary to classify the video endpoints into more categories in consideration of the number of video endpoints participating in the video conference, and therefore, according to one embodiment of the present invention, two thresholds, i.e., a first threshold corresponding to good communication conditions and a second threshold corresponding to poor communication conditions, may be determined in step S420. The first threshold is higher than the second threshold.
Step S430 may determine three response time periods, a first response time period corresponding to a threshold range determined by a first threshold and higher values, a second response time period corresponding to a threshold range determined by the first threshold and a second threshold, and a third response time period corresponding to a threshold range determined by the second threshold and lower values. The first response period may be shorter than the second response period, and the second response period may be shorter than the third response period.
The error correction request processing method 400 according to an embodiment of the present invention is further described below by taking an example in which the communication status of the video endpoint and the video server is reflected by the packet loss rate in the communication process between the video endpoint and the video server.
In step S410, a first weight value may be assigned to the video endpoint that just joined the video conference. For example, the first weight value may be 500000. In step S410, a second weight value may also be assigned to a video endpoint that does not lose packets during communication with the video server. The second weight value may be smaller than the first weight value, for example, the second weight value may be 1000.
In step S410, for a video endpoint where packet loss occurs during communication with the video server, a weight value depending on the packet loss rate and the third weight value may be assigned. The third weight value may be smaller than the second weight value, and the weight value assigned to each video endpoint may decrease as the packet loss rate increases. For example, the third weight value W3 may be set to 100. And the weight values assigned to each video endpoint are consistent with table 1 given above. Alternatively, at video endpoints that employ LPR (packet loss recovery) techniques, the video endpoints may be assigned higher weight values than those assigned to video endpoints that do not have LPR, since the video endpoints themselves may recover partially lost packets. The weight values assigned to each video endpoint utilizing LPR are shown in table 2 above. Technology relating to LPR is available in U.S. Pat. No. US7876685, the contents of which are incorporated herein by reference.
After each video endpoint is assigned a weight at step S410, a first threshold H1 and a second threshold H2 may be determined at step S420. According to one embodiment, the first threshold H1 and the second threshold H2 may be defined as follows:
a first threshold H1= (number of video endpoints × 0.4) × second weight; and
the second threshold H2= (number of video endpoints × 0.5) × the third weight.
The first threshold and the second threshold may be defined in various ways in step S420 as long as the first threshold is determined such that the first weight value is greater than the first threshold and the second threshold is determined such that the second weight value is greater than the second threshold. That is, the weight of the video endpoint that just joins the video conference should be greater than the first threshold, and the weight value of the video endpoint with good communication condition, that is, the video endpoint with the packet loss rate of 0, should be at least greater than the second threshold.
The first response period determined in step S430 is 3 seconds, the second response period is 30 seconds, and the third response period is 60 seconds.
The processing of steps S440-S480 is the same as above, and will not be described here.
It should be noted that, in the respective components of the controller described above, the components therein are logically divided according to the functions to be implemented, but the present invention is not limited thereto, and the respective components may be re-divided or combined as needed, for example, some components may be combined into a single component, or some components may be further decomposed into more sub-components.
The methods, apparatus and systems described herein may be implemented in hardware, or in software modules running on one or more processors, or a combination thereof. Those skilled in the art will appreciate that a microprocessor or Digital Signal Processor (DSP) may be used in practice to implement some or all of the functions of some or all of the components in a controller according to embodiments of the present invention. The methods, apparatus and systems described herein may also be implemented as apparatus or device programs (e.g., computer programs and computer program products) for performing a portion or all of the methods described herein. Such programs implementing the present invention may be stored on computer-readable media or may be in the form of one or more signals. Such a signal may be downloaded from an internet website or provided on a carrier signal or in any other form.
It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by one and the same item of hardware. The usage of the words first, second and third, etcetera do not indicate any ordering. These words may be interpreted as names.

Claims (21)

1. A method of processing error correction requests from a video client in a video conference including a video server and at least one video client, comprising:
assigning a weight value to each of the at least one video client, the weight value being determined based on communication conditions of the respective video client and the video server;
determining one or more thresholds according to the number of video clients in the video conference and the size of the assigned weight value;
determining a plurality of response time periods, each corresponding to one of the threshold ranges divided by the one or more thresholds;
receiving an error correction request from a video client, and processing the error correction request from the video client by:
determining a threshold range in which the video client is located by comparing the weight value assigned to the video client with the determined one or more thresholds, an
Processing an error correction request from the video client according to a response time period corresponding to the determined threshold range.
2. The method of claim 1, wherein in assigning a weight value to each video client, the weight value is further sized according to how long each video client joined the video conference.
3. The method of claim 2, wherein the one or more thresholds are determined in further consideration of a duration of time each video client joined the video conference.
4. The method of claim 1, wherein the one or more thresholds include a first threshold corresponding to good communication conditions and a second threshold corresponding to poor communication conditions,
the response time period includes a first response time period corresponding to a first threshold range determined by a first threshold and a weight value corresponding to a communication condition better than the first threshold, a second response time period corresponding to a second threshold range determined by the first threshold and a second threshold, and a third response time period corresponding to a third threshold range determined by the second threshold and a weight value corresponding to a communication condition worse than the second threshold, wherein the first response time period is shorter than the second response time period, and the second response time period is shorter than the third response time period.
5. The method of claim 4, wherein the third response period is twice as long as the second response period.
6. The method according to any of claims 2-5, wherein the communication condition of the respective client and video server comprises a packet loss rate during communication of the respective client and video server.
7. The method of claim 6, wherein assigning a weight to each video client further comprises:
distributing a first weight value to the video client with the time length of joining the video conference less than the preset time;
distributing a second weight value to a video client side which does not lose packets in the communication process with the video server, wherein the second weight value is smaller than the first weight value;
and allocating a weight value with a size depending on the packet loss rate and a third weight value to the video client with the packet loss occurring in the communication process with the video server, wherein the size of the third weight value is smaller than that of the second weight value, and the size of the allocated weight value is reduced along with the increase of the packet loss rate.
8. The method of claim 7, wherein the determining one or more thresholds comprises:
determining a first threshold such that the first weight value magnitude is greater than the first threshold; and
determining a second threshold such that the second weight value magnitude is greater than the second threshold.
9. The method of claim 1, wherein the error correction request is a retransmit I-frame request.
10. The method of claim 1, further comprising:
the assigning of the weight and the determining of the one or more thresholds to each video client is repeated at predetermined time intervals.
11. A video server, comprising:
a communication interface configured to communicate with one or more video clients to compose a video conference;
an audio and video processor configured to process audio and/or video signals from video clients in the video conference and forward the audio and/or video signals between video clients;
a controller configured to manage the video conference and further configured to process error correction requests from a video client, the controller further comprising:
a weight assignment component adapted to assign a weight value to each of the one or more video clients, the weight value being determined based on communication conditions of the respective client and the video server;
a threshold determination component adapted to determine one or more thresholds according to the number of video clients and the assigned weight value;
a response time period determination component adapted to determine a plurality of response time periods, each corresponding to one of the threshold ranges divided by the one or more thresholds;
an error correction request processing component adapted to receive an error correction request from a video client, determine a threshold range in which the video client is located by comparing a weight value assigned to the video client with the determined one or more thresholds, and process the error correction request from the video client with a response time period corresponding to the determined threshold range.
12. The video server of claim 11, wherein the weight assignment component is configured to assign a weight to each video client further based on how long each video client joined the video conference.
13. The video server of claim 12, wherein the threshold determination component is configured to determine the one or more thresholds further taking into account a length of time each video client joined the video conference.
14. The video server of claim 11, wherein the threshold determined by the threshold determining component comprises a first threshold corresponding to good communication conditions and a second threshold corresponding to poor communication conditions, and
the response period determined by the response period determination means includes a first response period corresponding to a first threshold range determined by a first threshold and a weight value corresponding to a communication condition better than the first threshold, a second response period corresponding to a second threshold range determined by the first threshold and a second threshold, and a third response period corresponding to a third threshold range determined by the second threshold and a weight value corresponding to a communication condition worse than the second threshold, wherein the first response period is shorter than the second response period, and the second response period is shorter than the third response period.
15. The video server of claim 14, wherein the third response period is twice as long as the second response period.
16. The video server of any of claims 12-15, wherein the communication condition of the respective video client and video server comprises a packet loss rate during communication of the respective video client and video server.
17. The video server of claim 16, wherein the weight assignment component is configured to:
distributing a first weight value to the video client with the time length of joining the video conference less than the preset time;
distributing a second weight value to a video client side which does not lose packets in the communication process with the video server, wherein the size of the second weight value is smaller than that of the first weight value;
and allocating a weight value with a size depending on the packet loss rate and a third weight value to the video client with the packet loss occurring in the communication process with the video server, wherein the size of the third weight value is smaller than that of the second weight value, and the size of the allocated weight value is reduced along with the increase of the packet loss rate.
18. The video server of claim 17, wherein the threshold determination component is configured to:
determining a first threshold such that the first weight value magnitude is greater than the first threshold; and
determining a second threshold such that the second weight value magnitude is greater than the second threshold.
19. The video server of claim 11, wherein the error correction request is a retransmit I-frame request.
20. The video server of claim 11, wherein the weight assignment component and the threshold determination component repeatedly assign a weight value to each video client and determine a respective threshold at predetermined time intervals.
21. A video conferencing system, comprising:
the video server of any of claims 11-20; and
one or more video clients transmitting and receiving audio and/or video signals to and from each other via the video server,
and when the video client fails to acquire the I frame data correctly, the video client sends a resending I frame request to the video server so as to process the I frame request by the video server.
HK13113553.7A 2013-12-05 A method for processing error correction requests, a video server and a video conference system HK1186327B (en)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201110402969.8A CN103152545B (en) 2011-12-07 2011-12-07 A kind of method, video server and video conferencing system processing error correction request

Publications (2)

Publication Number Publication Date
HK1186327A1 HK1186327A1 (en) 2014-03-07
HK1186327B true HK1186327B (en) 2016-10-21

Family

ID=

Similar Documents

Publication Publication Date Title
US9210380B2 (en) Multi-participant conference setup
US7949117B2 (en) Heterogeneous video conferencing
US8861701B2 (en) Multi-participant conference adjustments
US8441517B2 (en) Data processing apparatus and method, and network system
US9596433B2 (en) System and method for a hybrid topology media conferencing system
EP1875769B1 (en) Multi-participant conferencing
US9369671B2 (en) Method and system for handling content in videoconferencing
US8760490B2 (en) Techniques for a rate-adaptive video conference bridge
US9237306B2 (en) Distributed audio/video bridging for conferencing endpoints
US8208003B2 (en) Minimizing fast video update requests in a video conferencing system
CN101262587A (en) A method for realizing multi-screen video conference and multi-point control unit
CN115209189B (en) Video stream transmission method, system, server and storage medium
US10645128B2 (en) Media session processing method, related device, and communications system
CN108574840B (en) Method and device for evaluating video experience quality
CN103152545B (en) A kind of method, video server and video conferencing system processing error correction request
US20110247004A1 (en) Information Processing Apparatus
CN106998328B (en) Video transmission method and device
HK1186327B (en) A method for processing error correction requests, a video server and a video conference system
CN116436870A (en) Bandwidth adaptive method, device and storage medium
CN111404908B (en) Data interaction method and device, electronic equipment and readable storage medium
CN112770077B (en) Video conference I frame coding method and device