CN1315059C - Method and device for decoding image sequence using virtual image frame reordering - Google Patents

Abstract

In some cases, the image display system also needs to continuously display the pictures stored in the same buffer. To solve the above problem, the present invention solves the problem of reordering image decoding sequences (decoder) into image display sequences (display orders) by inserting virtual image pictures, achieving minimum hardware and software requirements.

Description

Method and device for decoding image sequence using virtual image frame reordering

technical field

The invention relates to an image display system, in particular to the arrangement of compressed image decoding and display sequence in the image display system.

Background technique

The CCITT/ISO (International Telegraph and Telephone Consultative Committee/International Organization for Standardization) organization has formulated standard formats for compression and decompression algorithms for still and moving digital images, including JPEG (Joint Photo Coding Expert Group), MPEG (Motion Picture Expert Group) ), H.261, etc. These standards have been widely used in video conferencing, VCD (video disc), DVD (digital video disc), video, on-demand video satellite image transmission, etc., as well as other applications that require the transmission of digital video images. The MPEG standard is formulated by the MPEG committee under the ISO, and its format includes animation, sound, and coding representation methods that combine the two.

Generally speaking, in image processing, data compression is carried out by means of motion estimation, comparing the pixels of the image frame to be compressed with the pixels of the previous image frame to obtain an estimated motion vector , then when transmitting the image frame, it is not necessary to transmit the complete image frame, but only the motion vector and the difference data between the macroblock of the previous image frame and the macroblock of the current image frame.

In the method of motion estimation, image pictures are divided into I pictures (intra-coded pictures), P pictures (predictively coded pictures), and B pictures (bidirectionally predictively coded pictures). Among them, the I picture does not need to do motion estimation during encoding, the P picture will use forward motion estimation (forward motion estimation) during the encoding process, and use the I picture or other P pictures as a reference, and the B picture will use forward motion estimation. For backward motion estimation, I-pictures or P-pictures need to be used as references. I pictures, P pictures, and B pictures form a series of ordered image picture sequences in the standard MPEG2 algorithm, called picture groups (Group of Pictures), such as I, B, B, P, B, B, P, B, B, P, B, B, I.... After the image screen is encoded, it is sent to the image display system for display.

After the image display system receives the image compression sequence, it must first decompress to form an image decoding sequence (decode order), and must be reordered when outputting the image display sequence (display order). In the process of general decoding and display, this reordering action is that the decoding sequence will be sequentially stored in different decoding buffers (decoding buffer), and when displaying, it will be stored in different decoding buffers according to the display order. Frames, in order to achieve the purpose of rearranging the image display sequence. However, in some cases, the display system must continuously display the pictures in a certain decoding buffer. In this way, the above-mentioned method cannot be used to reorder the image decoding sequence into an image display sequence. For example, some optical disc display systems provide disc preview. The function of digest view is convenient for users to quickly browse the content of the CD, and the reduced versions of different image sequences or different time points of the same image sequence must be displayed at the same time. In order to support the preview function, the picture currently being decoded must be displayed together with other previously displayed pictures, which means that they are all stored in the same display buffer, and the display system must continuously display this display buffer. screen. The sequence currently being decoded must also be stored in the display buffer in the order displayed, as shown in Table 1:

Table 1 Relationship between decoding sequence and display sequence time 1 2 3 4 5 6 7 8 9 10 11 12 13 14 decoding sequence I0 P3 B1 B2 P6 B4 B5 I9 B7 B8 P12 B10 B11 display sequence I0 B1 B2 P3 B4 B5 P6 B7 B8 I9 B10 B11 P12

Assuming that all buffers have the same size, and the image size can be controlled by the system itself, for example, 1/3 or 1/4 of the original size, as can be seen from Table 1, after I0 is decoded and stored in a decoding buffer, P3 must be performed Decoding movement, and at this time, I0 is stored in the display buffer after being scaled down. When the system completes the decoding and storage of B1 and B2 in sequence, the reduced size P3 should be stored in the display. In the buffer, in the process of decoding and displaying, the I-picture or P-picture used as a reference picture will be stored twice, one is stored in the normal size in the decoding buffer, and the other is stored in the reduced size in the The display buffer is used for display purposes, and the B picture is directly sent to the display buffer after decoding.

The known technology adopts the practice of decoding twice, and stores it in the decoding buffer with a normal size after decoding once, and stores it in the display buffer with a reduced size after decoding once, but it must be retrieved in the image decoding sequence for the second decoding. This picture makes the decoding control method more complicated. Another known technique is to perform the resizing function with additional direct memory access (DMA) hardware, and directly transfer the image frame originally stored in the decoding buffer to the display buffer, but this will increase the hardware cost. Another known technique is to use the system's RISC to perform the work of reading the decoding buffer and writing the display buffer, but the RISC needs to spend more I/O cycles (I/ O cycle), will reduce the performance of the RISC computer.

Since the image decoding sequence must be reordered when it is output as an image display sequence, there may be situations where the reference picture must be decoded twice, or additional direct memory access hardware is required, or the RISC I/O cycle is increased. The image display system sometimes also needs to continuously display images stored in the same buffer. In order to solve the above problems, the present invention solves the problem of reordering the image decoding sequence into the image display sequence by inserting virtual image frames, and achieves the minimum hardware and software requirements.

Contents of the invention

The object of the present invention is to provide a method and device for reordering an image decoding sequence into an image display sequence by inserting virtual image frames into the image decoding sequence, so as to reduce the complexity of hardware and software and reduce the cost.

The present invention provides a method for reordering an image decoding sequence into an image display sequence, wherein the image decoding sequence is composed of an I picture, a P picture and a B picture, and the method performs the following steps after receiving a compressed image sequence: (1) judging the first picture frame of the compressed picture picture sequence; (2) if the first picture picture is an I picture, decoding the first picture picture and storing it in the first buffer; and (3) according to a preset It is assumed that the first virtual image frame is obtained in a manner, and the first virtual image frame is transmitted to the second buffer for image display.

The present invention also provides a method for reordering an image decoding sequence into an image display sequence, wherein the image decoding sequence is composed of I pictures, P pictures and B pictures, and after receiving a compressed image picture sequence, the method includes the following steps: Steps: (1) detecting the first image frame of the compressed image frame sequence; (2) if the first image frame is a P frame, decoding the first image frame and storing it in the first buffer; and (3) according to A first virtual image frame is obtained in a preset manner, and the first virtual image frame is sent to a second buffer for image display.

The present invention also provides a method for reordering an image decoding sequence into an image display sequence, wherein the image decoding sequence is composed of I pictures, P pictures and B pictures, and after receiving a compressed image picture sequence, the method includes the following steps: Steps: (1) judge the first image frame of the compressed image frame sequence; (2) if the first image frame is an I frame, obtain the first virtual image frame according to a preset method, and transmit the first virtual image frame to the first buffer for image display; and (3) decoding the first image frame and storing it in the second buffer.

The present invention also provides a method for reordering an image decoding sequence into an image display sequence, wherein the image decoding sequence is composed of I picture, P picture and B picture, after receiving a compressed image sequence, the method includes The following steps: (1) detect the first image frame of the compressed image frame sequence; (2) if the first image frame is a P frame, obtain a first virtual image frame according to a preset method, and use the first virtual image frame Send to the first buffer for image display; and decode the first image frame and store it in the second buffer.

The present invention also provides a device for reordering an image decoding sequence into an image display sequence, wherein the image decoding sequence is composed of I picture, P picture and B picture, and the device receives an image decoding sequence, including the following devices:

A parameter generating device, used to generate a control parameter;

A motion compensation device (motion compensation device, MC) outputs a virtual image frame according to a predetermined method;

A memory, wherein the memory includes:

A first buffer for storing the first decoded image frame of the image decoding sequence;

a second register for storing the virtual image frame;

A third buffer for storing the second decoded image frame of the image decoding sequence;

A display controller (display controller) is used for connecting with the second register and displaying the virtual image frame.

Description of drawings

Figure 1a, 1b is a diagram of the device of the present invention

Fig. 2a, 2b, 3a, 3b are flow charts of the present invention

Fig. 4 is the storage content of each buffer in the specific embodiment of the present invention

Icon Component Symbol Explanation

100: parameter generator 105: control parameter

110: Motion compensator 115: Virtual image screen

120: Bus 130: Memory

131: First buffer 132: Second buffer

133: The third register 140: Display controller

Detailed ways

Please refer to Fig. 1 (a), the present invention is a device for reordering image decoding sequences into image display sequences, including: a parameter generating device 100 for generating control parameters 105; a motion compensation device 110, which is Utilize the first decoded image frame pre-stored in the first buffer 131, and cooperate with the control parameter 105 to generate a virtual image frame 115; the bus 120 is connected to each device of the present invention; the memory 130, see FIG. 2 ( b), wherein the memory 130 includes a first buffer 131 for storing the first decoded image frame of the image decoding sequence; a second buffer 132 for storing the virtual image frame 115; a third buffer 133 for storing The second decoded image frame of the image decoding sequence; the display controller 140 , configured to connect to the second buffer 132 and display the virtual image frame 115 . Wherein the image frames are all transmitted through the bus 120, and the display controller 140 receives the virtual image frame 115 and the decoded image frame, and can execute, for example: on-screen cutting menu (OSD), subpicture (subpicture), picture mixing (highlight) mixing function) and other functions, and transmit the image screen to a video display device, such as a TV.

It should be noted that the image frame of the compressed image frame sequence mentioned in the present invention can be in the form of a frame or a field, and the virtual image frame can also be a frame or a field. (field) screen form. The corresponding control parameters 105, taking MPEG2 as an example, include:

a. The picture coding type (picture_coding_type) of the virtual image picture is P picture

b. The macroblock of the virtual frame, its frame motion type (frame_motion_type), the embodiment is "frame-based";

c. The macroblock of the virtual field, its field motion type (field motion type), the embodiment is "Field_based";

d. The motion vector (motion vector) is defined by the user, and the preferred embodiment is 0; and

e. coded-block pattern (coded-block pattern), the embodiment is 0.

f. The virtual map field obtains the predicted value from the map field with the same spatiality (parity) as itself

The control parameter may further include a scale factor, so the length and width of the virtual image frame may be different from the size of the general image frame.

In the following descriptions and illustrations, blocks with the same numbers represent the same functions.

The implementation steps of the present invention are as follows, please see Fig. 2a: after step 200 receives a compressed image frame sequence, step 205 judges that the first image frame of the compressed image frame sequence is an I frame, a B frame or a P frame, and in step 205 if the first image frame An image frame is an I frame. Step 210 decodes the first image frame and stores it in a first buffer 131 . And, step 215 utilizes the decoded image frame pre-stored in the third buffer 133, cooperates with the control parameter 105 to generate the first virtual image frame 115, and transmits the first virtual image frame 115 to the second buffer 132 for image display .

The present invention further includes the following steps: step 220 judges the second image frame; in step 220 if the second image frame is a P frame, step 222 decodes the second image frame, and stores the second image frame in the third buffer 133 and Step 223 utilizes the decoded image frame pre-stored in the first buffer 131 and cooperates with the control parameter 105 to generate a second virtual image frame, and transmits the second virtual image frame to the second buffer 132 for image display.

If the second image frame is an I frame in step 220 , step 225 decodes the second image frame and stores the second image frame in the third buffer 133 . And step 226 utilizes the decoded image frame pre-stored in the first buffer 131 and cooperates with the control parameter 105 to generate a second virtual image frame, and transmits the second virtual image frame to the second buffer 132 for image display.

It must be noted that, in step 205, if the first image frame is a P frame, as shown in FIG. 2b, other implementation steps are the same as when the first image frame is an I frame.

In addition, steps 210 and 215 in FIG. 2a can be interchanged, that is, the present invention can first generate a virtual image frame 115, and then decode the first image frame, as shown in steps 305 and 310 in FIG. 3a. The steps of other embodiments are the same as those in Fig. 2a. Similarly, after step 220 in FIG. 2 a , step 222 to decode the image frame and step 223 to generate the second virtual image frame can also be interchanged, and step 225 and step 226 can also be interchanged. Similarly, after step 220 in FIG. 3 a , step 222 and step 223 can be interchanged, and step 225 and step 226 can be interchanged.

Steps 210 and 215 in FIG. 2b can also be interchanged, as shown in steps 305 and 310 in FIG. 3b. However, after step 220 in FIG. 2 b , steps 222 and 223 can be interchanged, and steps 225 and 226 can be interchanged. Similarly in FIG. 3b, step 222 and step 223 can be interchanged, and step 225 and step 226 can be interchanged.

Now enumerate a preferred embodiment of the present invention as follows, please see Fig. 4, when receiving an image frame P6, decode the image frame P6 and store in the first cache memory 131, and store in the 3rd cache memory 133 originally at the same time The P3 of P3 cooperates with the control parameter 105 to generate a virtual image frame of P3 and send it to the second buffer 132 , and then the received B4 and B5 are decoded and directly sent to the second buffer for display by the display controller 140 . Then when an image frame I9 is received, I9 is decoded and stored in the third buffer, and the P6 previously stored in the first buffer 131 is matched with the control parameter 105 to generate a virtual frame of P6 and sent to the second buffer 132, and the subsequent received B7 and B8 are also directly sent to the second buffer after being decoded, and the display is controlled by the display controller 140 .

Please continue to refer to FIG. 4, when an image frame P12 is received, the image frame P12 is decoded and stored in the first buffer 131, and at the same time, the I9 previously stored in the third buffer 133 is combined with the control parameter 105 to generate a virtual image of I9. The image frames are sent to the second buffer 132 , and the received B10 and B11 are decoded and directly sent to the second buffer for display by the display controller 140 . When the image picture P15 is received later, P15 is decoded and stored in the third buffer, and the P12 originally stored in the first buffer 131 is combined with the control parameter 105 to generate a virtual picture of P12 and send it to the second buffer 132. Control the display by the display controller 140.

See Table 2 for the relationship between the decoding sequence and the display sequence after inserting the virtual image picture.

Table 2 Relationship between decoding sequence and display sequence after inserting virtual image picture time 1 2 3 4 5 6 7 decoding sequence I0 (I0_vp), P3 B1 B2 (P3_vp), P6 B4 B5 display sequence I0 B1 B2 P3 B4 B5 time 8 9 10 11 12 13 14 decoding sequence (P6_vp), I9 B7 B8 (I9_vp), P12 B10 B11 (P12_vp), P15 display sequence P6 B7 B8 I9 B10 B11 P12

In Table 2, the motion compensator 110 performs a function similar to DMA, and writes the image frames pre-stored in one buffer into another buffer. In addition, it should be noted that when receiving a reference image frame (I-frame or P-frame), the device of the present invention will store the received frame into the first or third buffer, covering the first or third buffer Older data in the register, as shown in Figure 4, when the P6 picture is received, the system will store P6 in the original buffer stored in I0, and will not overwrite the buffer where P3 is located.

The present invention uses the motion compensator 110 to generate a virtual image frame, eliminating the need for additional DMA hardware, or increasing the RISC computer output and input operation cycle, thereby reducing the complexity of hardware and software and achieving the purpose of reducing costs.

Although the present invention is disclosed above with a preferred embodiment, it is not intended to limit the present invention to anyone skilled in the art. Various modifications and changes should be made without departing from the spirit and scope of the present invention. Therefore, the protection scope of the present invention shall prevail in the protection scope of the appended claims.

Claims (17)

1. one kind be the method for image display sequence with picture decoding sequence rearrangement, and wherein this picture decoding sequence is made up of I picture, P picture and B picture, and this method comprises the following steps: behind compressed image picture sequence of reception

(1.1) judge first image frame of this compressed image picture sequence;

(1.2) if this first image frame is the I picture, this first image frame of decoding, and be stored in first buffer; And

(1.3) utilize a decoded picture picture, cooperate a controlled variable that is produced by a parameter generating device to obtain the first virtual image picture, this first virtual image picture is sent to second buffer, be used for image and show.

2. the method for claim 1, wherein in the step of (1.3), this decoded picture picture be to be stored in one the 3rd buffer.

3. the method for claim 1 wherein also comprises after the step of (1.3):

(3.1) judge second image frame;

When (3.2) being the P picture as if this second image frame, this second image frame of decoding is stored in the 3rd buffer with this second image frame; And

(3.3) utilize decoded this first image frame, cooperate controlled variable, this second virtual image picture is sent to this second buffer, be used for image and show to obtain one second virtual image picture.

4. the method for claim 1 wherein also comprises after the step of (1.3):

(4.1) judge second image frame;

When (4.2) being the I picture as if this second image frame, this second image frame of decoding is stored in the 3rd buffer with this second image frame; And

(4.3) utilize decoded this first image frame, cooperate this controlled variable, this second virtual image picture is sent to this second buffer, be used for image and show to obtain the second virtual image picture.

5. one kind is the method for an image display sequence with the rearrangement of picture decoding sequence, and wherein this picture decoding sequence is made up of I picture, P picture and B picture, and this method comprises the following steps: after receiving a compressed image picture sequence

(5.1) detect first image frame of this compressed image picture sequence;

(5.2) if this first image frame is the P picture, this first image frame of decoding, and be stored in first buffer; And

(5.3) utilize a decoded picture picture, cooperate a controlled variable that is produced by a parameter generating device to obtain the first virtual image picture, this first virtual image picture is sent to second buffer, be used for image and show.

6. method as claimed in claim 5, wherein in the step of (5.3), this decoded picture picture be to be stored in one the 3rd buffer.

7. method as claimed in claim 5 wherein also comprises after the step of (5.3):

(7.1) judge second image frame;

When (7.2) being the P picture as if this second image frame, this second image frame of decoding is stored in the 3rd buffer with this second image frame; And

(7.3) utilize decoded this first image frame, cooperate controlled variable, this second virtual image picture is sent to this second buffer, be used for image and show to obtain the second virtual image picture.

8. method as claimed in claim 5 wherein also comprises after the step of (5.3):

(8.1) judge second image frame;

When (8.2) being the I picture as if this second image frame, this second image frame of decoding is stored in one the 3rd buffer with this second image frame; And

(8.3) utilize decoded this first image frame, cooperate controlled variable, this second virtual image picture is sent to this second buffer, be used for image and show to obtain the second virtual image picture.

9. one kind is the method for an image display sequence with the rearrangement of picture decoding sequence, and wherein this picture decoding sequence is made up of I picture, P picture and B picture, and this method comprises the following steps: after receiving a compressed image picture sequence

(9.1) judge first image frame of this compressed image picture sequence;

(9.2) if this first image frame is the I picture, utilize a decoded picture picture, cooperation obtains the first virtual image picture by the controlled variable that a parameter generating device is produced, and this first virtual image picture is sent to first buffer, is used for image and shows; And

(9.3) this first image frame of decoding, and be stored in second buffer.

10. method as claimed in claim 9, wherein in the step of (9.2), this decoded picture picture be to be stored in one the 3rd buffer.

11. method as claimed in claim 9 wherein also comprises after the step of (9.3):

(11.1) judge second image frame;

When (11.2) being the P picture as if this second image frame, this second image frame of decoding is stored in the 3rd buffer with this second image frame; And

(11.3) utilize decoded this first image frame, cooperate controlled variable, this second virtual image picture is sent to this first buffer, be used for image and show to obtain the second virtual image picture.

12. method as claimed in claim 9 wherein also comprises after the step of (9.3):

(12.1) judge one second image frame;

When (12.2) being the I picture as if this second image frame, this second image frame of decoding is stored in the 3rd buffer with this second image frame; And

(12.3) utilize decoded this first image frame, cooperate this controlled variable, this second virtual image picture is sent to this first buffer, be used for image and show to obtain the second virtual image picture.

13. one kind is the method for an image display sequence with picture decoding sequence rearrangement, wherein this picture decoding sequence is made up of I picture, P picture and B picture, and this method comprises the following steps: after receiving a compressed image picture sequence

(13.1) detect first image frame of this compressed image picture sequence;

(13.2) if this first image frame is the P picture, utilize a decoded picture picture, cooperation obtains the first virtual image picture by the controlled variable that a parameter generating device is produced, and this first virtual image picture is sent to first buffer, is used for image and shows; And

(13.3) this first image frame of decoding, and be stored in second buffer.

14. method as claimed in claim 13, wherein in the step of (13.2), this decoded picture picture be stored in one the 3rd buffer.

15. method as claimed in claim 13 wherein also comprises after the step of (13.3):

(15.1) judge second image frame;

When (15.2) being the P picture as if this second image frame, this second image frame of decoding is stored in one the 3rd buffer with this second image frame; And

(15.3) utilize a decoded image frame, cooperate controlled variable, this second virtual image picture is sent to this first buffer, be used for image and show to obtain the second virtual image picture.

16. method as claimed in claim 13 wherein also comprises after the step of (13.3):

(16.1) judge second image frame;

When (16.2) being the I picture as if this second image frame, this second image frame of decoding is stored in the 3rd buffer with this second image frame; And

(16.3) utilize decoded this first image frame, cooperate controlled variable, this second virtual image picture is sent to this first buffer, be used for image and show to obtain the second virtual image picture.

17. one kind is the device of an image display sequence with picture decoding sequence rearrangement, wherein this picture decoding sequence is made up of I picture, P picture and B picture, and this device receives a picture decoding sequence, comprises array apparatus down:

One parameter generating device is used to produce a controlled variable;

One storer, wherein this storer comprises:

One first buffer is used to store the first decoded picture picture of this picture decoding sequence;

One second buffer is used to store a virtual image picture; And

One the 3rd buffer is used to store the second decoded picture picture of this picture decoding sequence;

One motion compensator unit according to this controlled variable and this first decoded picture picture, is exported this virtual image picture;

One display controller is used to connect this second buffer and shows this virtual image picture.

Images