WO2012072276A1

WO2012072276A1 - Transport bit-rate adaptation in a multi-user multi-media conference system

Info

Publication number: WO2012072276A1
Application number: PCT/EP2011/053780
Authority: WO
Inventors: Magnus Westerlund; Bo Burman
Original assignee: Telefonaktiebolaget LM Ericsson AB
Current assignee: Telefonaktiebolaget LM Ericsson AB
Priority date: 2010-11-30
Filing date: 2011-03-14
Publication date: 2012-06-07
Anticipated expiration: 2013-05-30

Abstract

One aspect provides a client apparatus and method of adapting the bit-rates of a plurality of media streams transmitted by a client entity in a multi-media conference. The media streams have associated quality levels, at least one of the media streams having a plurality of associated quality levels. A priority value is assigned, on a scale of priority values, to each quality level of each media stream. A list is determined, in order of priority value, comprising all of the assigned priority values and for each assigned priority value specifying the quality levels of the media streams to which the priority value has been assigned. Feedback is obtained for each media stream indicating a QoS for the media stream. Based on the feedback, the bit-rate of one or more of the media streams is adjusted in accordance with the priorities on the list. Another aspect provides a mixer apparatus and method of adapting media streams distributed by a mixer in a multi-media conference.

Description

Transport Bit-rate Adaptation in a Multi-user Multi-media Conference System Field of the Invention

The present invention relates to a method and apparatus for adapting the bit-rates of media streams in a multi-media telecommunications conference.

Background

Figure 1 depicts a multi-user, multi-media conference system 10 consisting of n participating clients, 12a, 12b...12n and a central mixer 14 providing a conference focus. The clients 12a, 12b...12n each have a terminal, or User Equipment, UE, and in such a system these may be heterogeneous in that they may have varying capabilities, such as in screen size/resolution, encoding and decoding capacity, capturing devices and connectivity. For example a highly capable client may be able both to capture and play out High Definition, HD, video, while there might be less capable clients that can only render VGA resolution.

To maximize the user experience it is desirable to render video and audio from multiple clients at the same time. However many clients will not be capable of rendering all media at the highest received resolution, or there may be other factors that limit the user experience at any given time, such as the availability of network resources. To account for this, the mixer 14 may be configured to prioritise the media streams, selecting one or more of the most active media and providing them in high resolution while the remaining media are presented in lower resolution, or not at all. In addition, the mixer may prioritise sending higher resolution media to the most active participants. This means that the mixer 14 needs to send different quality versions to different participants according to each client's individual capabilities.

To enable the mixer 14 to provide different media quality levels to different participants, a session participant will send one or more quality versions to the mixer. The mixer selects between the media streams from the session participants based on user activity and distributes a number of streams to each participant. The audio from the different participants is often mixed together so that the same audio is sent to all participants. Multi-user, multi-media conference systems as described above will involve the transmission of large amounts of data. The bit-rate required for the different media, including audio and more than one version of video, can be very high and above what is available in the network paths between the participants and the mixer.

The bit-rate consumed is different depending on the path. The path from a client to the mixer is equivalent to a point to point session, and bit-rate adaptation for this situation is a well known problem. One can, for example, use Real-time Transmission Control Protocol (RTCP) Receiver reports provided by the mixer to the client. These indicate a current Quality of Service, (QoS) for the media. The media encoders in the client can then adjust the bit-rate of each encoding and determine how it is packetized into RTP packets (see Real-time Transmission Protocol (RTP), I ETF RFC 3550). However, in the multi-user system, in addition to providing multiple media, for some media such as video, the client may need to deliver several quality levels of the same media. This creates a multi-dimensional adaptation problem.

The path from the mixer to any given client presents a more difficult case. Ideally, in the conference application the mixer 14 should deliver up to n video streams based on each clients' decoding and display capabilities. Thus the mixer 14 needs to select the n most important video streams and if feasible deliver them in the best possible quality. The mixer needs to determine if the combined bit-rate of all these streams can be delivered and what to do if that is not possible. The network path and the number n vary for each client taking part in the session. A system could rely on transcoding to provide adaptation to clients, but this is demanding of resources in the mixer/conference focus and also negatively affects the quality relative to used bandwidth resources. Thus we can disregard transcoding as the best solution to the problem. Independent of whether the media sent from the client to the mixer is delivered as multiple independent media streams at different quality levels, or uses a scalable encoding such as SVC (see Scalable Video Coding (SVC), ITU-T Recommendation H.264 Annex G), both the client and the mixer have a multidimensional prioritization problem in choosing the best trade-off between delivering different types of media and different media quality levels. Summary

According to a first aspect of the present invention there is provided a method of adapting the bit-rates of a plurality of media streams transmitted by a client entity in a multi-media conference. The media streams have associated quality levels, at least one of the media streams having a plurality of associated quality levels. The method comprises assigning a priority value on a scale of priority values to each quality level of each media stream. A list is determined, in order of priority value, comprising all of the assigned priority values and for each assigned priority value specifying the quality levels of the media streams to which the priority value has been assigned. Feedback is obtained for each media stream indicating a QoS for the media stream. Based on the feedback, the bit-rate of one or more of the media streams is adjusted in accordance with the priorities on the list. Adjusting the bit-rate may comprise either reducing the bit-rate of a stream to maintain the quality levels indicated by a higher priority on the list, or increasing the bit rate of one or more of the media streams to include a quality level indicated by a lower priority on the list. Obtaining feedback may also include probing to identify whether resources are available to increase the bit-rate of a media stream.

According to a second aspect of the present invention there is provided a method of adapting media streams distributed by a mixer in a multi-media conference. At least one of the media has a plurality of associated quality levels, and at least one of the media can be distributed to a variable number of participating clients. The method comprises assigning a priority value, on a scale of priority values, to each quality level of each media and for each number of the variable number of participating clients to which the media can be distributed. A list is determined, in order of priority value, comprising all of the assigned priority values and for each assigned priority value specifying the media quality levels to which the priority value has been assigned and numbers of additional participating clients to which the media can be distributed. Feedback indicating the QoS of the media streams being distributed is received. Based on the feedback, the distribution of the media streams is adjusted in accordance with the priorities on the list. The adjusting may include adjusting the number of participating clients to which a media is being distributed and/or signalling a source of one or more of the media to provide the media at a changed quality level.

The adjusting may include adjusting the number of and/or quality level of the media streams being distributed to individual participating clients. The adjusting may comprise selecting one or more participating clients for receiving one or more media streams based on a list that prioritizes the importance of each participating client. Alternatively, the adjusting may be performed in response to instructions based on signalling between the mixer and the participating clients.

The distribution of the media streams may be adjusted to maintain the quality levels of the media streams and/or number of participants receiving the media streams indicated by a higher priority on the list, or to include quality levels of the media streams and/or number of participants receiving the media streams indicated by a lower priority on the list.

The priority list may be constructed such that the bit rate required for the media streams increases with a decreasing priority.

For each media at least three quality threshold levels may be defined, the list comprising the media threshold levels in order of priority. The threshold levels may comprise: a minimum quality threshold, a fair quality threshold and a maximum quality threshold. Obtaining feedback may also include probing to identify whether resources are available to increase the bit-rate of a media stream. The method may further comprise increasing the bit-rate of a media stream above a threshold level towards the next threshold level when the probing indicates resources are available. According to a third aspect of the present invention there is provided apparatus for participating in a multi-media conference. The apparatus comprises an input/output for transmitting a plurality of media streams, each media stream having one or more associated quality levels. A priority value on a scale of priority values has been assigned to each quality level of each media stream. The apparatus is configured to determine a list, in order of priority, comprising all of the assigned priority values and specifying the quality levels of the media streams to which each priority value has been assigned. The apparatus is configured to obtain feedback relating to each media stream indicating a QoS for the media stream, and, responsive to the feedback, to adjust the bit-rate of one or more of the media streams in accordance with the priorities on the list.

According to a fourth aspect of the present invention there is provided a mixer apparatus for distributing streams of media to clients participating in a multi-media conference. At least one of the media has a plurality of associated quality levels. At least one of the media can be distributed to a variable number of participating clients. A priority value, on a scale of priority values, has been assigned to each quality level of each media and for each number of the variable number of participating clients to which the media can be distributed. The mixer apparatus is configured to determine a list, in order of priority value, comprising all of the assigned priority values and specifying the media quality levels and numbers of additional participating clients to which the media can be distributed to which each priority value has been assigned. The mixer apparatus is configured to obtain feedback indicating the QoS of the media streams being distributed, and, based on the feedback, to adjust the distribution of the media streams in accordance with the priorities on the list.

The methods and apparatus advantageously provide for both the client and the mixer to perform a multidimensional prioritization when choosing between delivering different types of media and different media quality levels. Brief Description of the Drawings

Figure 1 is a schematic representation of a multi-user, multi-media conference system.

Figure 2 is a flow diagram illustrating the principal steps in a procedure for adapting the bit-rate at a client terminal in a multimedia conference.

Figure 3 is a flow diagram illustrating the principal steps in a procedure for adapting the bit-rate at a mixer/conference focus in a multimedia conference.

Figure 4 is a schematic illustration showing the principal functional components in a mixer/conference focus

Figure 5 is a schematic illustration showing the principal functional components in a client terminal. Detailed description

The basic concept is a method to map the multiple dimensions that affect bit rate adaptation onto a single dimension to produce a one-dimensional priority list against which the available bit rate can be mapped. The multiple dimensions include the number of media, their quality levels and individual encoding quality, and, in the case of the mixer, the number of participants receiving a media stream. This also enables customization and service adaptation for optimizing the user experience on the available client device.

The different dimensions are identified in the system at the point (client or mixer) where adaptation needs to occur. A multi-dimensional matrix is created expressing the relative priority of each cell of the matrix - i.e. each particular combination of the different dimensions that intersect at that cell. This matrix may look different for each client attached to the same session. To simplify the potentially complex decision of prioritizing the dimensions, priority values in the cells are serialized by sorting them in descending order, forming a priority list. The point (i.e. client or mixer) performing the adaptation, and which is operating at a position or level in the list, measures, for the aggregate of the media flows, the current path performance, in terms of its Quality of Service, QoS. If this indicates either that the media QoS at that level are not able to be maintained, or that the available bit-rate is not fully utilised, then it adapts the bit-rate by taking a step either up or down to the next level in the list and determining the required bit-rate for the media at that level. The rate of change needs to be controlled to find a suitable trade-off between user annoyance caused by the change and the required speed of change to avoid persistent congestion. In other words, a balance needs to be found because frequent or rapid changes, while beneficial for optimising the system performance, are likely to adversely affect the user experience.

Some dimensions, such as video quality related to bit-rate for an individual stream, are almost continuous rather than having a low number of discrete steps. In some such cases, e.g. for adaptation of media streamed from a client to the mixer, and where there is close to a linear relationship between bit rate and media quality over a particular range, then it may be possible to perform an interpolation of the priority values along that dimension. However, care needs to be taken to avoid causing unnecessary or frequent jumps in other dimensions. We consider first the adaptation at a client 12, such as one of clients 12a, 12b, 12n in Figure 1 , for media streams being sent to the mixer/conference focus 14. The client 12 prioritizes the different media streams, which might typically include audio, video at various different quality levels and any additional media. Usually, audio is prioritized highest, followed by the lower, or lowest, quality video and then higher video quality levels. The available bit-rate might be sufficient for all streams at their maximum quality, but in general this will not be the case and so some adaptation is needed. In this example, for each media three quality thresholds are created: a minimum quality threshold, where if the bit-rate is below it the media become virtually unusable; a fair threshold where the quality is good enough to be truly usable (i.e. an acceptable quality for all users) and the maximum quality where providing additional bit-rate for the media stream would not provide a quality increase worth the cost. These three thresholds represent different points of prioritization on the axis of the media quality dimension.

Thus we can create a prioritization matrix for the client's transmission to the server. This has on one axis the different media streams, and on the other axis their individual media quality expressed at the threshold values. An example of such a matrix is shown in Table 1 .

Table 1

The values in the cells of the matrix are priority values representing the relative importance (on a scale of 0-1 ) of including the media in the session (note that for the purpose of this discussion each different level of video resolution is considered to be a different media). Note also that what matters is the relative position of each of the media on the scale, not the absolute magnitude of the priority value. The mixer 14 sends RTCP feedback to the client for each media stream relating to the QoS of the stream. The client 12 also utilizes any transport level feedback (e.g. DCCP - Datagram Congestion Control Protocol (DCCP), I ETF RFC 4340 - if used). Examples of QoS-related feedback include indications of:

• (complete or) partial loss of the stream before reaching the receiver (e.g. packet loss),

• data corruption of the stream,

• stream delay (too long end-to-end delay could be perceived as bad quality),

• stream delay variation (too much variation could be perceived as bad quality), and each of these may be expressed as one or more measured values. Based on this feedback the client 12 determines if it should maintain current bit-rate or increase it or decrease it. An increase or a decrease will slide its operational point with respect to used media streams towards the lower or higher priority by modifying the encoding bit- rate towards the thresholds.

The above 2-dimensional matrix can be translated into a one-dimensional list in decreasing order of priority and corresponding media activity. Table 2 shows the list for the example of the matrix of Table 1 above.

Table 2

1 .0 Minimal Quality Audio

0.95 Fair quality audio and minimal quality low

resolution (LR) video

0.9 Maximum Quality Audio and between minimal and

fair quality LR video

0.85 Max Q Audio, Fair Q LR Video and Minimal Q

Medium Resolution (MR) Video

0.7 Max Q Audio, Max Q LR Video and between

minimal and fair Q MR Video

0.5 Max Q Audio, Max Q LR Video, Fair Q MR Video

and Minimal Q High Resolution (HR) Video

0.3 Max Q Audio, Max Q LR Video, Between Fair and

Max Q MR Video, and Fair Q HR Video 0.2 Max Q Audio, Max Q LR Video, Max Q MR Video,

and between Fair and Max Q HR Video

0.1 Max Q Audio, Max Q LR Video, Max Q MR Video

and Max Q HR Video

It should be noted that there is no real correspondence between the priority value and the actual bit-rate consumption. The highest priority (1 ) can be mapped to the lowest bit-rate where communication with the highest priority media is at all feasible. The lowest (non-zero) priority can be mapped to the bit-rate where all included media reached their full quality. Intermediate priorities do not, in general, map linearly to bit-rate. Thus, table 2 can be seen as a ladder, or scale, of operating points with descending priority values and ascending bit rates.

Table 2 above is constructed in such a way that the bit-rate will increase for an operating point with lower priority compared to one with higher priority. However, it is clear that certain operating points will result in significant jumps in bit-rate. A good example is the operating point with priority 0.5. At this point the client 12 adds a high resolution video stream. Even at its minimal quality a high resolution video stream can utilise as much bandwidth as all of the previous (higher priority) streams together.

The priority table (Table 2) can also be used to provide weights between media for applying fine-grained changes between operating points by increasing the quality for individual media. Such fine-grained control of media enable many small step increases in bit-rate. For example, the 0.85 priority value operating point includes both fair quality low- resolution video and minimal quality medium-resolution video. Between the 0.85 and 0.7 priority value operating points the Low-resolution Video is expected to go from Fair to Maximum quality. Therefore, when bit-rate becomes available the quality of this media stream could be increased towards the maximum quality. However, the quality of the Medium-resolution video could also be increased between minimal and fair. As indicated, maximum quality low-resolution video occurs at the 0.7 priority value operating point, whereas fair quality medium-resolution video is only reached at the 0.5 priority point. Thus, the priority values indicate the relative importance of the different media streams for allocating resources when moving in between the operating points. As stated above, the mixer 14 sends RTCP feedback to the client 12 for each media stream and the client 12 can also utilise other transport level feedback. However, rather than just waiting for this feedback, the client 12 can probe to find out whether there is any available bit-rate. Probing for the additional bit-rate required to move towards the next operating point is quite straightforward when the next operating point only requires a higher quality for an existing media stream, and the encoding of these media support fine-grained control. It is more difficult when adding additional media streams or for media that only have coarse-grained control (i.e. only allow a few large step changes in bit-rate). In such cases the following operational behaviour is recommended if possible.

If another media stream has fine grained control and has not yet reached maximum quality, or has the possibility to provide additional bits, then these can be used to function as probe traffic until it can be determined that there is sufficient bit-rate available for the new media stream, or to increase the bit-rate to the next step in the coarse grained controlled media. Using the example of Tables 1 and 2 above, when attempting to reach the 0.5 operating point, the client 12 can consider what is intended to happen with the other media to reach the next operating point. So before the high- resolution video is added, the medium-resolution video stream can be further increased in bit-rate towards its maximum quality level, a quality level not intended to be reached until the 0.2 priority value operating point. When the client 12 determines that the bit- rate necessary for the high-resolution video stream at minimal quality is present, that stream is added while the medium-resolution video stream has its bit-rate reduced back to the level necessary for the fair quality. The bit-rate threshold for fair quality medium-resolution video should be selected such that the quality degradation when dropping from close to maximum quality back to fair quality is not too disturbing for the user experience. It is here further assumed that the process of probing for higher bit-rates does not have a severe impact, such as major loss of available bit-rate, on existing media streams. That is however a separate technical problem not discussed in detail herein.

The main steps in the method are shown in Figure 2. At step 201 a priority value is assigned to each quality level of each media stream in the conference. At step 202 a list is determined, in order of priority, of the media quality levels. The list includes all of the assigned priority values and specifies the quality levels of the media streams assigned to each priority value. At step 203 feedback is obtained for each media stream indicating the current QoS (i.e. bit-rate) for the media stream. Optionally, at step 204, the client sends a probing signal, to ascertain if there is additional bit-rate capacity available. At step 205, based on the feedback the client determines if a reduction in bit-rate of one or more of the media streams is required to maintain the quality levels indicated by a higher priority on the list. At step 206 the appropriate adjustment to the bit-rate is made. Alternatively, at step 207, the client determines from the feedback and/or probing that the bit rate of one or more of the media streams may be increased towards a quality level indicated by a lower priority on the list, or that an additional media stream may be added.

The flow-chart of Figure 2 is presented as a single timeline. However a "real" system implementing the method will in general be dynamic in the sense that certain steps may be may be repeated more than once as the bandwidth will vary dynamically over time and through the process. Also, Figure 2 does not make a distinction between finegrained adjustment and step changes between operating points, as discussed above. Clearly any increase or decrease in bit-rate may be either a fine-grained adjustment of one or more media streams (if such is possible), or a jump up or down between operating points.

Referring back to Figure 1 , we consider next the situation of adaptation at the mixer 14. A mixer in a multi-party multi-media conference only has direct control over the bit-rate created by the media encoders for streams that are locally encoded. In a system that avoids using transcoding (see above), it is possible that only the audio mix created by the mixer 14 is encoded in the mixer. The rest of the media streams are only selected from those the clients 12a, 12b. 12c... etc. deliver to the mixer 14. The following tools are available for adaptation:

1 . Selection of the quality level for a particular media when more than one quality level is available from the client.

2. Selection of which media to deliver to the client(s).

3. Selection of the number of session participants to be distributed to a particular client for a particular media and quality level. 4. Using some signalling means, e.g. TMMBR (see Codec Control Messages (CCM), IETF RFC 5104), to request that a client changes the bit-rate for a particular media encoding. If we disregard the locally encoded content that provides fine-grained bit-rate control, we here have a system with several dimensions of adaptation.

In the illustrative example below, we consider an application that plays back an audio mix, one Main video and up to 5 additional videos. Thus there are three points - audio, Main video and additional video - on the "media" dimension. This example only has one audio stream as it is mixed together. The Main video also has only one stream (due to limitations of the clients). More capable clients that could handle multiple Main video feeds and/or have multiple screens would involve the use of more streams. The additional video streams show the conference participants not currently selected for the Main video. Due to screen estate a limit of 5 simultaneous video streams has been selected in this example.

The different media streams in a conference have different purposes. Thus, the axis of another of the dimensions is the quality level of the individual media from a client used for a particular purpose. However, some media may be used in conjunction with others for a single purpose. For example medium-resolution video and high-resolution video may be used for the same purpose (Main video in this example) and therefore effectively only constitute different quality levels of the same media. This only provides a coarse-grained adaptation. With scalable encoding, such as SVC, it might be possible to get a few more steps within one dimension of adaptation. However, without transcoding or fine-grained scalability, it is not possible to provide fine-grained control in this dimension.

This illustrative example presents a solution with priorities that is applicable to any use case which has multi-dimensional selections. The priorities should be tailored to the goals of the application as well as to the conference system designer preferences and possibly also to end-user preferences.

Table 3 below shows an example matrix over the three dimension axes. The first axis is the "media" axis referred to above. The second axis is the number of streams. The third axis, individual media quality, is expressed within the individual cells of the matrix in Table 3, where applicable. In this example, only the Main video is available in different quality levels for the mixer. The quality levels are medium (MR) and high (HR) resolution and this is expressed in the form "MR/HR" priority values.

Table 3

As for the client adaptation case described above, the information in table 3 can be reinterpreted into the following list in order of descending priority values, as shown in Table 4.

Table 4

1 .0 Audio Only

0.9 Audio and Main Video at MR

0.8 Audio, Main Video at MR and 1 additional video

0.7 Audio, Main Video at MR and 2 additional videos

0.65 Audio, Main Video at HR and 2 additional videos

0.6 Audio, Main Video at HR and 3 additional videos

0.5 Audio, Main Video at HR and 4 additional videos

0.4 Audio, Main Video at HR and 5 additional videos The table 4 priority list can be used for selection of the quality level for a particular media when more than one quality level is available from the client and for selection of which media to deliver to the client(s) - i.e. the first two adaptation tools referred to above - in the same way as described above for client adaptation.

For the third tool - selection of the number of session participants to be distributed to a particular client for a particular media and quality level - the mixer must determine which streams to send to a client, both in the case of the single Main video and the number of additional videos. This can be determined from a list of the session participants that prioritizes (in descending order) the importance of each participant. This list may be expected to be quite dynamic and regularly updated to reflect changing participant activity, or other measure of the importance of each participant for the conference. An alternative, depending on the particular conference application capabilities, would be to have manual control using signalling between the mixer and the participants, or a controller.

In general, a mixer will not have as good information about quality thresholds for the different media streams, when compared with the client that encoded them. This limits the mixer's ability to use the fourth tool - using a signalling means such as TMMBR to request that a client changes the bit-rate for a particular media encoding. TMMBR requests from the mixer to the client should be limited and only be used for the more prioritized media streams. When a stream is distributed to several clients it is important to ensure that most benefit from adjusting the bit-rate. The clients' possibility to comply with the TMMBR request might also be limited because the mixer may request a bit- rate below what the client deems to be minimal quality.

In the case where there is severe congestion of media sent to a certain client, for video media means may be provided to request intra pictures, such as PLI (see Audio-Visual Profile with Feedback (AVPF), IETF RFC 4585) or FIR (see Codec Control Messages (CCM), IETF RFC 5104), as well as the mixer having the ability to selectively filter them out. In that case, rather than a complete video stream, a slide-show of intra pictures could be sent to the seriously congested client, possibly as a last resort before dropping the stream entirely. The advantage with this approach would be that it does not affect other receiving clients in the same way as bit-rate reduction at the source. This can be applied in addition or as an alternative to the bit-rate reduction (e.g. TMMBR) described above.

As a way to optimize the entire conference, the mixer can collect all available transport feedback from RTP/RTCP and the underlying transport protocol, and consider the available bit-rate based on the whole aggregate of the streams for the conference service. This is because the combined feedback indicates how well the whole aggregate is being transported. For example, packet losses in the most prioritized audio stream, indicating congestion, should result in a bit-rate adaptation in the least prioritized media to improve the situation.

Because the video streams that are selected will vary in bit-rate and also see variations in the media encoding, the number of streams that can be sent from the mixer will vary over time. If the stream selection and adaptation only looks at short term measurements there will be many rapid fluctuations in the number of media streams being sent. This will reduce the quality of the user experience. Therefore a certain amount of low pass filtering (hysteresis) is required to avoid disturbing fluctuations in the number of streams. Thus there is a trade-off between avoiding too large fluctuations while still avoiding serious congestion issues that also affect the quality.

The main steps in the method are shown in Figure 3 below. As for the method shown in the flow-chart of Figure 2, Figure 3 is presented as a single timeline, whereas a "real" system will in general be dynamic so that certain steps may be may be repeated. At step 301 a priority value is assigned to each quality level of each media and for each number of the variable number of participating clients to which the media can be distributed. At step 302 a priority list is determined, in order of priority, specifying the media quality levels and numbers of additional participating clients to which the media can be distributed. At step 303 feedback is received indicating the QoS of the media streams being distributed. At step 304, based on the feedback, the mixer determines if the distribution of the media streams needs to be adjusted to maintain the quality levels/number of participants of a higher priority on the list. At step 305 the distribution of the media streams is adjusted accordingly. Alternatively, at step 306 based on the feedback, the mixer determines that an adjustment can be made towards including quality levels/number of participants of a lower priority on the list, and at step 307 the adjustment is made accordingly. If a media stream uses retransmission, e.g. using RTP Retransmission (see IETF RFC 4588), bit-rate typically increases temporarily in the event of random loss or congestion for that stream. In the case of adaptation at the client, the client can adjust stream bit- rate to accommodate both the stream and the retransmission within negotiated bandwidth. Adjusting the stream bit-rate may not even be necessary if head-room for retransmission is provided that can accommodate both original and retransmitted packets. Stream bit-rate adjustment is not possible for adaptation at the mixer when the mixer does not use transcoding. However, retransmission without stream bit-rate adjustment using some head-room for re-transmission (as for the client adaptation case above) can still be used, and the same considerations regarding selection of media streams can be applied also to this case by considering the aggregate of the media stream and retransmitted stream as a single unit in the selection/prioritization process.

Any multi-media system that consumes a large amount of bandwidth will have a startup issue. From a user perspective it is desirable to immediately arrive at the highest possible bit-rate needed by the application and also supported by the network path. This problem is no different for an aggregate of media streams than for a single stream at the same bit-rate as the aggregate.

Figure 4 is a schematic illustration showing the principal functional components in a mixer/conference focus 40. The mixer 40, includes an input-output module 42 where media streams are received from clients and where media streams are distributed out to clients. In addition the input-output module 42 sends out probing signals to ascertain available QoS/bit-rate and also receives feedback signals, such as RTCP and/or transport level feedback from the network indicating the QoS of each media stream. The mixer 40 also includes a processor 46 and a memory 44. Priority values have been assigned to each quality level of each media and for each number of a variable number of participating clients to which the media can be distributed. The processor 46 is configured to determine a priority list, in order of priority value, comprising all of the assigned priority values and specifying the media quality levels and numbers of additional participating clients to which the media can be distributed to which each priority value has been assigned. The memory 44 stores the priority list 45. The processor 46 also includes a media distribution function 47 that obtains and analyses the QoS feedback of the media streams being distributed, and, based on the feedback, adjusts the distribution of the media streams in accordance with the priorities on the priority list 45. Figure 5 is a schematic illustration showing the principal functional components in a client 50. The client 50 includes an input-output module 52 where media streams are received from and sent to the conference focus or mixer. In addition the input-output module 52 sends out probing signals to ascertain available QoS/bit-rate and also receives feedback signals, such as RTCP and/or transport level feedback from the network indicating the QoS of each media stream. The client 50 also includes a processor 56 and a memory 54. Priority values have been assigned to each quality level of each media and for each number of a variable number of participating clients to which the media can be distributed. The processor 56 is configured to determine a priority list, in order of priority value, comprising all of the assigned priority values and specifying the quality levels of the media streams to which each priority value has been assigned. The memory 54 stores the priority list 55. The processor 56 also includes a media preparation function 57 that obtains and analyses the QoS feedback of each media stream being sent to the conference focus, and, responsive to the feedback, adjusts the bit-rate of the media streams in accordance with the priorities on the list.

Claims

CLAIMS:

1 . A method of adapting the bit-rates of a plurality of media streams transmitted by a client entity in a m ulti-media conference, wherein the media streams have associated quality levels, at least one of the media streams having a plurality of associated quality levels, the method comprising:

assigning a priority value on a scale of priority values to each quality level of each media stream;

determining a list, in order of priority value, comprising all of the assigned priority values and for each assigned priority value specifying the quality levels of the media streams to which the priority value has been assigned;

obtaining feedback for each media stream indicating a QoS for the media stream; and

based on the feedback, adjusting the bit-rate of one or more of the media streams in accordance with the priorities on the list.

2. The method of claim 1 , wherein adjusting the bit-rate comprises either reducing the bit-rate of a stream to maintain the quality levels indicated by a higher priority on the list, or increasing the bit rate of one or more of the media streams to include a quality level indicated by a lower priority on the list.

3. The method of claim 1 or claim 2, wherein obtaining feedback also includes probing to identify whether resources are available to increase the bit-rate of a media stream.

4. A method of adapting media streams distributed by a mixer in a multi-media conference, wherein at least one of the media has a plurality of associated quality levels, and at least one of the media can be distributed to a variable number of participating clients, the method comprising:

assigning a priority value, on a scale of priority values, to each quality level of each media and for each number of the variable number of participating clients to which the media can be distributed;

determining a list, in order of priority value, comprising all of the assigned priority values and for each assigned priority value specifying the media quality levels to which the priority value has been assigned and numbers of additional participating clients to which the media can be distributed;

receiving feedback indicating the QoS of the media streams being distributed; and

based on the feedback, adjusting the distribution of the media streams in accordance with the priorities on the list.

5. The method of claim 4, wherein the adjusting includes adjusting the number of participating clients to which a media is being distributed and/or signalling a source of one or more of the media to provide the media at a changed quality level.

6. The method of claim 4 or claim 5, wherein the adjusting includes adjusting the number of and/or quality level of the media streams being distributed to individual participating clients.

7. The method of claim 6 wherein the adjusting comprises selecting one or more participating clients for receiving one or more media streams based on a list that prioritizes the importance of each participating client.

8. The method of claim 6 wherein the adjusting is performed in response to instructions based on signalling between the mixer and the participating clients.

9. The method of any of claims 4 to 8, wherein the distribution of the media streams is adjusted to maintain the quality levels of the media streams and/or number of participants receiving the media streams indicated by a higher priority on the list, or to include quality levels of the media streams and/or number of participants receiving the media streams indicated by a lower priority on the list.

10. The method of any preceding claim wherein the priority list is constructed such that the bit rate required for the media streams increases with a decreasing priority.

1 1 . The method of any preceding claim wherein for each media at least three quality threshold levels are defined, and wherein the list comprises the media threshold levels in order of priority.

12. The method of claim 1 1 , wherein the threshold levels comprise: a minimum quality threshold, a fair quality threshold and a maximum quality threshold.

13. The method of claim 1 1 or claim 12, wherein obtaining feedback also includes probing to identify whether resources are available to increase the bit-rate of a media stream.

14. The method of claim 13 wherein the method further comprises increasing the bit-rate of a media stream above a threshold level towards the next threshold level when the probing indicates resources are available.

15. Apparatus for partici pati ng i n a m u lti-media conference, the apparatus comprising an input/output for transmitting a plurality of media streams, each media stream having one or more associated quality levels and wherein a priority value on a scale of priority values has been assigned to each quality level of each media stream, the apparatus being configured to determine a list, in order of priority, comprising all of the assigned priority values and specifying the quality levels of the media streams to which each priority value has been assigned, and to obtain feedback relating to each media stream indicating a QoS for the media stream, and, responsive to the feedback, to adjust the bit-rate of one or more of the media streams in accordance with the priorities on the list.

16. The apparatus of claim 15 further configured to adjust the bit-rate by either reducing the bit-rate of a stream to maintain the quality levels indicated by a higher priority on the list, or to increase the bit rate of one or more of the media streams to include a quality level indicated by a lower priority on the list.

17. The apparatus of claim 15 or claim 16, further configured to perform a probe operation to identify whether resources are available to increase the bit-rate of a media stream.

18. A mixer apparatus for distributing streams of media to clients participating in a multi-media conference, wherein at least one of the media has a plurality of associated quality levels, at least one of the media can be distributed to a variable number of participating clients and a priority value, on a scale of priority values, has been assigned to each quality level of each media and for each number of the variable number of participating clients to which the media can be distributed, the mixer apparatus being configured to determine a list, in order of priority value, comprising all of the assigned priority values and specifying the media quality levels and numbers of additional participating clients to which the media can be distributed to which each priority value has been assigned, to obtain feedback indicating the QoS of the media streams being distributed, and, based on the feedback, to adjust the distribution of the media streams in accordance with the priorities on the list.

19. The mixer apparatus of claim 18, configured to adjust the number of participating clients to which a media is being distributed and/or to send a signal to a source of one or more of the media to provide the media at a changed quality level.

20. The apparatus of claim 18 or claim 19, configured to adjust the number of and/or quality level of the media streams being distributed to individual participating clients.

21 . The mixer apparatus of claim 18 or claim 19, configured to distribute the media streams to maintain the quality levels of the med ia streams and/or number of participants receiving the media streams indicated by a higher priority on the list, or to include quality levels of the media streams and/or number of participants receiving the media stream indicated by a lower priority on the list.