WO2004056123A1

WO2004056123A1 - Quality based signal processing

Info

Publication number: WO2004056123A1
Application number: PCT/GB2003/005249
Authority: WO
Inventors: David Sneddon Hands
Original assignee: British Telecommunications PLC
Current assignee: British Telecommunications PLC
Priority date: 2002-12-17
Filing date: 2003-12-02
Publication date: 2004-07-01
Anticipated expiration: 2005-06-17
Also published as: AU2003285577A8; GB0229396D0; AU2003285577A1; WO2004056123A8

Abstract

A signal is generated at a predetermined encoding rate by an encoder (11), and the encoded signalis stored in a buffer (12). Measurement is made of the quality of the signal stored in the buffer, to determine whether a different encoding rate, for example using more or less data compression, would be appropriate. If the quality of the buffered signal falls within a predefined range, a control unit (14) causes the buffered signal (12) to be transmitted, but if it falls outside that range the signal is re-encoded at a different rate. This allows bandwidth to be allocated dynamically according to the demands of the signals being transmitted.

Description

SIGNAL PROCESSING

This invention relates to the control of the transmission of a signal such that the quality of the signal transmitted is maintained within acceptable limits. It is of particular application to digitised audio-visual signals, but may also be used for other transmissions such as simple audio signals.

Existing video services can be monitored for quality, and to some extent the quality of video services delivered over multiple channels can be controlled using statistical multiplexing. This technique operates by examining statistical redundancy in the image represented by the video bit-stream, for example due to repetition of identical pixel values, to determine the optimum data requirements per video channel, given total available bandwidth. However, there is no unified method in existence that provides an end-to-end feedback and control mechanism for optimising the delivery of video services. A number of industrial and academic researchers have been developing realtime perceptual video quality measurement tools. These tools, typically implemented as software instruments, aim to provide video quality measurements that are representative of human judgements. Examples of such tools are described in the present applicant's International Patent Specifications W097/32428, W099/21 1 73, WO00/01 165, WO00/48407 and W001 /75799.

Perceptual models that can be used for real-time video quality measurement are described in Bretillon P, Montard N, Baina J, Goudezeune G: "Quality meter and digital television applications" Visual Communications and Image Processing 2000, 20-23 June 2000, SPIE-lnt. Soc. Opt. Eng, Proceedings of the SPIE - The International Society for Optical Engineering, vol.4067, pt.1 -3 pp 780-90, and in the applicant's patent specification W099/21 173 mentioned above.

Existing quality measurement systems rely on feedback from the receiving end. This only possible where an uplink exists. Even where this is the case, it causes delay in the time taken to adjust the transmission to accommodate such deterioration. If the transmission is not in real time, repetition of poor quality parts of the transmission may be possible, but this incurs further delay. The feedback and repetition all place extra demands on the available bandwidth. According to the present invention, there is provided a method of processing a signal wherein the signal is generated at a predetermined encoding rate, a measurement is made of the quality of the generated signal before transmission, and the encoding rate of the signal is adjusted such that the quality of the output signal falls within a predefined range

According to another aspect, a signal transmission system comprises a quality measurement unit and a transmission control unit, wherein the quality measurement unit has means for measuring the quality of a signal to be transmitted, and provides an input to the transmission control unit based on said measurement, and the transmission control unit has means for determining a suitable encoding rate for transmission of a signal on the basis of the said input.

This allows measurement from a real-time quality metric to be used at the transmission end to optimise the delivery of video services.

The invention may be embodied in a multimedia server arranged to provide audio-visual or other data to client servers, either on a one to one basis (on demand, network based) or on a one-to-many basis (broadcast). It may be provisioned using a buffer store, or directly into a video codec (coder-decoder), as described in the applicant company's United Kingdom patent application 0228556.7 filed on December 6^th 2002. In general it is the video part of a multimedia signal that is bandwidth hungry and it is here that real gains can be made. However, the invention also allows optimisation between audio, video, data and other channels, for example selecting a lower audio quality to make bits availabkle for use by the video signal.

The invention requires real-time quality measurement tools to be located at one or more points in the delivery chain of the signal - most notably at the source - the server generating it. Measurement may also take place at other points in the delivery chain, such as the transmitters, intermediate switching points, receivers, or the destination device (client server), to allow the effect of the transmission medium itself to be included in the processing. The quality metric is preferably a perceptual model, but some other real-time metric (e.g. Peak Signal to Noise ratio estimations) may be used.

A feedback system may also be included, in which measurement is carried out at the destination (client) unit. Feedback from the destination unit to the control unit can be used to indicate whether network problems affect video quality delivered to the end user. The control unit can use feedback data to modify quality-critical properties at the encoder, such as bit-rate, frame rate or resolution. In a broadcast system, one or more of the destination units can be used to provide such feedback. This invention directly addresses the problem of quality optimisation for bandwidth-hungry video services. Using quality measurements taken at the receiver or client, the invention may also provide quality information that may be used by collaborating parties (e.g. network providers and service providers) to negotiate and monitor quality limits. The invention can be used by service providers to define the quality range for delivering one or more video channel, and using this preset information, together with video quality measurements taken prior to transmission, to control the encoder bit-rate. Where applicable, feedback data is used to address changes in video quality introduced during transmission (e.g. due to congestion in the network). In addition to defining (controlling) the quality of video prior to transmission, the present invention may also use the quality information extracted to provide feedback to a network or service provider regarding the perceptual quality of service (P-QoS) at the client or receiver, or regarding the perceptual quality of service at the source or video server. The remainder of this specification uses perceptual models as the basis for quality measurement, although it should be noted that the invention is not limited to such models, and is capable of using alternative, non-perceptual, methods of quality measurement such as peak signal-to -noise ratio (PSNR).

Thus the quality of the signal is controlled prior to transmission, based on specific quality measures. This invention utilises these quality measures to provide information relating to output quality at the receiver or client, information relating to the performance of the transmission network, and quality optimisation at the source.

The invention may therefore be used to increase the coding rate where a low coding rate (high data compression) is causing significant loss of quality - for example jerky or blocky pictures. Note that in circumstances where bandwidth is restricted, the process may be used to reduce the coding rate if the control unit determines that the measured quality value is better than the minimum required, for example in order to gain spare bandwidth for use by another channel. In a single- channel situation, using a narrowband connection, the control unit may be used to optimise the balance between quality and download time.

Embodiments of the invention will now be described, by way of example, with reference to the drawings, which represent schematically the various functional elements which co-operate in the invention.

Figure 1 shows a system according to the invention, in which measures made of the signal before transmission control the mode of transmission.

Figure 2 shows another system according to the invention, in which additional feedback from the receiving end is also used Figure 1 shows a server for generating encoded video or other signals. An encoder 1 1 accesses source material and encodes it for transmission at a first, default, data rate. This default data rate may vary from time to time, depending on the bandwidth available to the transmitter 1 5 and the number of separate signals currently requiring transmission. The control component of the invention works as follows. A video encoder

1 1 is linked to a control unit 14. The encoder 1 1 is responsible for compressing a video bit-stream prior to transmission. The control unit 14 contains information regarding the upper and lower quality limits acceptable for the video service and a record of the current bit-rate used by the encoder 1 1 . The control unit 14 can read and set (or write) the bit-rate at which the video encoder 1 1 operates.

Video services are delivered by firstly setting (manually, could be a default value) the bit-rate of the encoder 1 1 . The encoder 1 1 then begins compressing the video data prior to transmission. Once the video data has been encoded, it is placed into a temporary data buffer 1 2. A real-time quality measurement tool 13, for example the real-time video quality model commercially available from Rohde 6k Schwarz, analyses the quality of the video data held in the buffer 12. Quality measurements, which may be taken every frame or every few frames, are then passed to the control unit 14. The control unit 14 uses these quality measurements to test whether the quality of the video service meets the performance criteria set by the operator.

If the quality measurement, as passed to the control unit 14, is greater than the upper quality threshold limit, the control unit changes the bit-rate downward by some pre-determined amount (e.g. 5%). For example, if the bit-rate was initially 2 Mbit/s and the quality is measured as being greater than the upper threshold limit, the control unit sets the encoder to operate at 1 .9 Mbit/s. If the quality measurement is less than the lower quality threshold limit, the control unit 14 changes the bit-rate upward by some pre-determined amount (e.g. 5%). Using the example above, if the encoder initially encodes at 2 Mbit/s and the resulting quality measure is below the lower threshold value, the control unit 14 sets the encoder 1 1 to operate at 2.1 Mbit/s. This procedure runs iteratively until the quality falls within the threshold limits. When the measured video quality is within the upper and lower threshold values, the control unit 14 sends a message to the buffer 1 2 to forward the data stored therein to the transmitter 14 for transmission to the receiver 1 6.

The invention may be used in a number of ways. Considering firstly a single live video channel delivered from a source video server across a fixed-line network to a single client. At the server side, the video data is first prepared for transmission using some form of encoding scheme. Consider a live video stream that is encoded using an MPEG-2 digital compression scheme. Prior to transmission, the encoder 1 1 places video data into a buffer 12, at some standard bit-rate (e.g. 2 Mbit/s). By placing a real-time quality measurement tool to analyse the video data stored in the buffer, a measure of quality can be obtained. The video measurement analysis is performed, for example, using the principles detailed in the applicant's International patent specification WO01 /91473, which describes a non-intrusive video quality model that operates on the frame buffer. Video data is decoded and held in a frame store. The video model then extracts information from the frame store (e.g. spatial complexity, blockiness, temporal difference), which can be used to produce a video quality measurement. Video measures may be taken on a per frame basis, to give the finest granularity, but it is also possible to take average quality measures at longer time intervals provided the quality control process does not exceed the buffer size.

Using predefined upper and lower quality threshold settings, a quality control mechanism is provided. Measurements from the video quality assessment processor 13 are sent to a quality decision control unit 14. This unit has direct access to the video encoder 1 1 and can control the bit-rate at which the video stream is encoded. If the quality measurements fall below the minimum threshold value, the control unit 14 increases the bit-rate used by the encoder 1 1 by a predetermined amount. If the maximum threshold value is exceeded the control unit 14 decreases the bit rate used by the encoder 1 1 by a predetermined amount. In either case the encoder 1 1 is instructed to re-code the previously encoded data and transmit it to the buffer 12.

A look-up table is used to define the change in bit-rate depending on how distant the current measured quality is from the desired quality. If the quality value measured falls within the threshold limits, the control unit 14 sends a 'transmit' message to the buffer mechanism 12. Only when the 'transmit' message is received by the buffer mechanism 1 2 is the video data transmitted.

If a maximum number of iterations has been reached, the bit rate that produced the optimum result (usually, but not necessarily, the most recent) is adopted. If neither of those criteria have been reached, a further iteration is performed.

The amount by which the bit-rate should be increased or decreased for the system to work with maximum effectiveness will depend on a number of factors, such as how much delay can be tolerated whilst the system hunts for the optimum bit rate, and what level of granularity is required. It should be noted that the optimum bit rate may change during the course of a transmission, because of changes in the character of the video content - fast-moving action is less tolerant of compression than a slower-moving scene - or because of changes in the amount of bandwidth available. Delay during the course of a real-time transmission is less acceptable than at the beginning of a real-time transmission, or at any time in a transmission which is not to be viewed in real time.

The implementation described above is for a single channel video service. The control mechanism is also applicable to optimise the delivery of multiple channel video services. The service provider has finite bandwidth within which these channels may be transmitted. The service provider first sets the upper and lower thresholds for each channel. Using the same basic system described above, adding differential thresholding and prioritising between channels enables the control unit 14 to optimise the quality of multiple video channels. For example, consider the situation where the maximum available bandwidth is insufficient for even the minimum quality threshold to be met across all channels. If all channels are assigned equal priority, then the control unit will try to get each channel's quality level as close to the lower minimum threshold value as possible. The service provider may assign a more stringent lower quality threshold to those channels that are of greatest value (e.g. channels that incur a high subscription charge, such as film or sports channels). The channels may also be allocated different prioritisations. For example a video service provider delivering subscription channels at several different subscription rates will want to protect the quality of the channels for which the highest rates are charged, at the expense of those charging lower rates, or which are free. Where one or more channels are assigned higher priority status than others, the control unit will first aim to get these channels' level of quality to meet the lower threshold value, and then attempt to achieve a quality level for the remaining channels as close to the lower threshold value as possible.

Figure 2 illustrates a further embodiment, in which feedback is also provided at the receiving unit where the video service terminates. The transmitter end 1 1 , 1 2, 13, 14, 15 is as before, except that the control unit 14 now has a second input, receiving feedback from the receiving unit. Prior to transmission the control 14 unit stores, for each frame for which it has the data, the following information: frame number (n) associated quality measurement MOSp_(source)(n).

At the receiver 1 6 a decoder 21 regenerates the original signal, which is then passed into a frame buffer 22 and subjected to a video quality measurement processor 23 similar to the processor 13 at at the transmitter end. Quality measures are taken from the frame buffer 22 either every frame or every few frames. The output of the quality measurement processor 23 is transmitted to the control unit 14 by a feedback unit 24. The feedback unit 24 sends a message, using for example an IP connection, to the control unit 14 located at the video source. The feedback message contains the following information: identity of receiver (e.g. IP address), frame number for which the quality measurement was extracted, n quality measurement value (MOSp,_dest))(n) The control unit 14, which stores a record of frame number and associated quality measurement prior to transmission, then retrives the previously stored quality measure at source (MOSp_(source)) relating to the specified frame, and records the following information received from the feedback unit 24: quality measure at destination (MOSp_{dest)) (n), from which is calculated: change in quality during transmission (MOSp_(source) (n) - MOSp_(dest)) (n).

The output of the control unit 14 therefore depends on inputs from two quality measurement processors 13, 23. This feedback information can be used to examine the extent of transmission errors on video quality, to monitor service level agreements or vendor agreements and ensure any quality guarantees promised by a network operator) are met.

The feedback information may also be used by the control unit 14 to amend the opertation of the encoder 1 2 to adjust the encoding properties of the transmitted video data to accommodate network problems. For example, in heavily congested networks, a reduction in video frame rate can have a minimal impact on quality but greatly reduces the bandwidth necessary to transmit the video data. Similarly, by applying knowledge regarding tolerances to reductions in image resolution, the control unit can change the resolution at which the encoder operates to reduce the bandwidth requirements of the video transmission process. The control unit 14 in this situation has the ability to set appropriate video encoding properties.

Claims

1 . A method of processing a signal wherein the signal is generated at a predetermined encoding rate, a measurement is made of the quality of the generated signal before transmission, and the encoding rate of the signal is adjusted such that the quality of the output signal falls within a predefined range.

2. A method according to claim 1 , wherein the signal is a video signal.

3. A method according to claim 1 or claim 2, wherein the quality of signal stored in a buffer is measured, and a determination is made as to whether the signal stored in the buffer is to be transmitted, or is to be re-encoded at a different data rate.

4. A method according to claim 1 or claim 2, wherein a quality measurement is made of the output of a coder/decoder (codec), and the coding rate of the codec is controlled in response to the quality measurement.

5. A method according to claim 1 , 2, 3, or 4, wherein additional quality measurements are made at one or more receivers, and fed back to control the encoding rate.

6. A method according to claim 1 , 2, 3, 4 or 5, wherein the quality metric is a perceptual model.

7. A signal transmission system comprising a quality measurement unit and a transmission control unit, wherein the quality measurement unit has means for measuring the quality of a signal to be transmitted, and provides an input to the transmission control unit based on said measurement, and the transmission control unit has means for determining a suitable encoding rate for transmission of a signal on the basis of the said input.

8. A signal transmission system according to claim 7, comprising an encoding unit controlled by the transmission control unit to generate a signal at a predetermined encoding rate, and a buffer store for storing the signal so generated, wherein the quality measurement unit measures a quality of the signal stored in the buffer, and determines whether the signal stored in the buffer is to be tranmsitted or re-encoded at a different data rate.

9. A signal transmission system according to claim 7, comprising a variable rate coder/decoder unit (codec), measurement means for analysing the quality of the output of the codec, and control means for controlling the coding rate of the codec in response to the quality measured by the measurement means.

10. A signal transmission system according to claim 7, 8 or 9, comprising means for receiving quality measures, fed back to the control means from additional measurement means provided at one or more receivers, for controlling the encoding rate.

1 1 . A signal transmission system according to claim 7, 8, 9, or 10, wherein the quality metric is a perceptual model.